为了更快的定位您的问题,请提供以下信息,谢谢
【详述】在关闭元数据缓存查询分区表很慢
【背景】
【业务影响】
【StarRocks版本】例如:3.1.2
【集群规模】例如:3fe(1 follower+2observer)+3be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【联系方式】
【附件】
使用关闭元数据缓存catalog查询
“enable_metastore_cache” = “false”,
“enable_remote_file_cache” = “false”
profile信息
Planner:
- Analyzer: 159ms / 1
- CTEUniqueChecker: 0ms / 1
- CoordDeliverExec: 64ms / 1
- CoordPrepareExec: 15ms / 1
- ExecPlanBuild: 1ms / 1
- InputDependenciesChecker: 0ms / 1
- Optimizer: 44760ms / 1
- Total: 44922ms / 1
- Transformer: 1ms / 1
- TypeChecker: 0ms / 1
HMS: - getDatabase: 55ms / 1
- getPartitionColumnStatistics: 40897ms / 1 - getPartitionsByNames: 1732ms / 2
- getTable: 103ms / 1
- listPartitionNames: 726ms / 1
PARTITIONS: - LIST_FS_PARTITIONS: 1372ms / 3
Optimizer: - CostBaseOptimize: 44026ms / 1
- PhysicalRewrite: 0ms / 1
- PlanValidate: 0ms / 1
- RuleBaseOptimize: 733ms / 1
- preprocessMvs: 0ms / 1
RuleBaseOptimize:
RewriteTreeTask: - ExternalTablePartitionPrune: 732ms / 1
HMS:
PARTITIONS: - LIST_FS_PARTITIONS: 7578
getPartitionColumnStatistics: - lineitem_orc_partition: 2526 partitions
getPartitionsByNames: - lineitem_orc_partition: 2526 partitions
Execution: - QueryAllocatedMemoryUsage: 183.547 GB
- QueryCumulativeCpuTime: 49s344ms
- QueryCumulativeOperatorTime: 7s527ms
- QueryDeallocatedMemoryUsage: 183.542 GB
- QueryExecutionWallTime: 4s779ms
- QueryPeakMemoryUsage: 512.994 MB
- QuerySpillBytes: 0.000 B
- ResultDeliverTime: 0ns
在getPartitionColumnStatistics耗时较高,这点能否优化?
附件为元数据缓存开启和关闭两种情况下的profilemate_cache (93.3 KB) no_meta_cache (93.8 KB)