getPartitionColumnStatistics is not supported

hale0923 · 2022年12月22日 13:47

【详述】通过 StarRocks-2.5.0RC 访问 AWS 存放在 S3 的数据，AWS 使用 Glue 管理元数据。最终数据无法正常查询，可正常获取库、表等信息
【业务影响】
【StarRocks版本】StarRocks-2.5.0RC
【集群规模】1fe+1be
【附件】
2022-12-22 21:36:07,480 ERROR (starrocks-mysql-nio-pool-6|524) [CachingHiveMetastore.get():357] Error occurred when loading cache
com.google.common.util.concurrent.UncheckedExecutionException: com.starrocks.connector.exception.StarRocksConnectorException: Failed to get database [INFORMATION_SCHEMA] from meta store: Database information_schema not found. (Service: AWSGlue; Status Code: 400; Error Code: EntityNotFoundException; Request ID: 92412062-2a31-4713-baed-5f9bf0d229c2; Proxy: null)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4952) ~[spark-dpp-1.0.0.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.get(CachingHiveMetastore.java:355) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.getDb(CachingHiveMetastore.java:176) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetastoreOperations.getDb(HiveMetastoreOperations.java:41) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetadata.getDb(HiveMetadata.java:74) ~[starrocks-fe.jar:?]
at com.starrocks.server.MetadataMgr.lambda$getDb$1(MetadataMgr.java:118) ~[starrocks-fe.jar:?]
at java.util.Optional.map(Optional.java:215) ~[?:1.8.0_312]
at com.starrocks.server.MetadataMgr.getDb(MetadataMgr.java:118) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.resolveTable(QueryAnalyzer.java:740) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.access$100(QueryAnalyzer.java:66) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.resolveTableRef(QueryAnalyzer.java:245) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:166) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:83) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:212) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:88) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryRelation(QueryAnalyzer.java:106) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:100) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:83) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:35) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:88) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.analyze(QueryAnalyzer.java:76) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.visitQueryStatement(Analyzer.java:325) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.visitQueryStatement(Analyzer.java:115) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:35) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:41) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.analyze(Analyzer.java:117) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer.analyze(Analyzer.java:112) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:54) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:37) ~[starrocks-fe.jar:?]
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:370) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:312) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:427) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:673) ~[starrocks-fe.jar:?]
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:55) ~[starrocks-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]
Caused by: com.starrocks.connector.exception.StarRocksConnectorException: Failed to get database [INFORMATION_SCHEMA] from meta store: Database information_schema not found. (Service: AWSGlue; Status Code: 400; Error Code: EntityNotFoundException; Request ID: 92412062-2a31-4713-baed-5f9bf0d229c2; Proxy: null)
at com.starrocks.connector.hive.HiveMetaClient.getDb(HiveMetaClient.java:161) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetastore.getDb(HiveMetastore.java:51) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.loadDb(CachingHiveMetastore.java:180) ~[starrocks-fe.jar:?]
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:168) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:192) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4952) ~[spark-dpp-1.0.0.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.get(CachingHiveMetastore.java:355) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.getDb(CachingHiveMetastore.java:176) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.loadDb(CachingHiveMetastore.java:180) ~[starrocks-fe.jar:?]
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:168) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:192) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[spark-dpp-1.0.0.jar:?]
… 40 more
2022-12-22 21:36:07,480 ERROR (starrocks-mysql-nio-pool-6|524) [HiveMetadata.getDb():76] Failed to get hive database [aws_catalog.INFORMATION_SCHEMA]
com.starrocks.connector.exception.StarRocksConnectorException: Failed to get database [INFORMATION_SCHEMA] from meta store: Database information_schema not found. (Service: AWSGlue; Status Code: 400; Error Code: EntityNotFoundException; Request ID: 92412062-2a31-4713-baed-5f9bf0d229c2; Proxy: null)
at com.starrocks.connector.hive.HiveMetaClient.getDb(HiveMetaClient.java:161) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetastore.getDb(HiveMetastore.java:51) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.loadDb(CachingHiveMetastore.java:180) ~[starrocks-fe.jar:?]
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:168) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:192) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4952) ~[spark-dpp-1.0.0.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.get(CachingHiveMetastore.java:355) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.getDb(CachingHiveMetastore.java:176) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.loadDb(CachingHiveMetastore.java:180) ~[starrocks-fe.jar:?]
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:168) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:192) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[spark-dpp-1.0.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4952) ~[spark-dpp-1.0.0.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.get(CachingHiveMetastore.java:355) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.CachingHiveMetastore.getDb(CachingHiveMetastore.java:176) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetastoreOperations.getDb(HiveMetastoreOperations.java:41) ~[starrocks-fe.jar:?]
at com.starrocks.connector.hive.HiveMetadata.getDb(HiveMetadata.java:74) ~[starrocks-fe.jar:?]
at com.starrocks.server.MetadataMgr.lambda$getDb$1(MetadataMgr.java:118) ~[starrocks-fe.jar:?]
at java.util.Optional.map(Optional.java:215) ~[?:1.8.0_312]
at com.starrocks.server.MetadataMgr.getDb(MetadataMgr.java:118) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.resolveTable(QueryAnalyzer.java:740) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.access$100(QueryAnalyzer.java:66) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.resolveTableRef(QueryAnalyzer.java:245) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:166) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:83) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:212) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:88) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryRelation(QueryAnalyzer.java:106) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:100) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:83) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:35) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:88) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.QueryAnalyzer.analyze(QueryAnalyzer.java:76) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.visitQueryStatement(Analyzer.java:325) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.visitQueryStatement(Analyzer.java:115) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:35) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:41) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer$AnalyzerVisitor.analyze(Analyzer.java:117) ~[starrocks-fe.jar:?]
at com.starrocks.sql.analyzer.Analyzer.analyze(Analyzer.java:112) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:54) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:37) ~[starrocks-fe.jar:?]
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:370) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:312) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:427) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:673) ~[starrocks-fe.jar:?]
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:55) ~[starrocks-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]

Natsume729 · 2022年12月23日 01:47

麻烦附件贴下be.out的信息看看有没有其他报错

hale0923 · 2022年12月23日 03:10

start time: Thu Dec 22 16:35:38 CST 2022
start time: Thu Dec 22 17:27:57 CST 2022
start time: Thu Dec 22 21:25:41 CST 2022

be.out 没有其他错误信息

Natsume729 · 2022年12月23日 08:38

show database from catalog能正常打印，但查询没结果，可以检查下s3认证信息配的是不是有问题

hale0923 · 2022年12月26日 07:35

查询时出现这个是什么情况呢，已经设置了 JAVA_HOME 环境变量。在 /etc/profile 也设置了环境变量

hale0923 · 2022年12月26日 07:37

在 start_be.sh 中添加 JAVA_HOME 环境变量就可以