【详述】单表五千万数据查询很慢 20s以上
【背景】新建的表
【业务影响】查询缓慢
【StarRocks版本】2.3.0
【集群规模】例如:3fe(1 follower+2observer)+5be(fe与be混部)
【机器信息】32C/125G3 18/125G2
【附件】
- fe.log/beINFO/相应截图
- 慢查询:
- Profile信息
Query:
Summary:- Query ID: 4912c56f-7acb-11ed-b09c-0242f112ba66
- Start Time: 2022-12-13 17:48:24
- End Time: 2022-12-13 17:48:47
- Total: 23s730ms
- Query Type: Query
- Query State: EOF
- StarRocks Version: 2.3.0-RC03
- User: root
- Default Db: default_cluster:vlib3
- Sql Statement: SELECT * FROM swk_comp_result_bk WHERE RWID =‘80ef6c28-c135-454f-89a6-1fa029f6a13b’ AND BDYYID=‘104bd953-33e9-4562-80d2-961b9b3590ec’ AND BDFS=‘MATERIAL’ AND BDYQMC=‘IFLYTEK’ AND SFSC=0
- QueryCpuCost: 3s564ms
- QueryMemCost: 684.601MB
- Variables: parallel_fragment_exec_instance_num=18,pipeline_dop=0
- Collect Profile Time: 105ms
Planner: - Total: 5ms / 1
Execution Profile 4912c56f-7acb-11ed-b09c-0242f112ba66: - ExecutionTotalTime: 23s702ms
Fragment 0:- BackendNum: 1
- InstanceNum: 1
- MemoryLimit: 183.27 GB
- PeakMemoryUsage: 240.66 KB
Pipeline (id=0):- ActiveTime: 911.740us
- BlockByInputEmpty: 893
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 31
- DriverTotalTime: 23s702ms
- OverheadTime: 911.740us
- PendingTime: 23s692ms
- InputEmptyTime: 23s694ms
- FirstInputEmptyTime: 262.320ms
- FollowupInputEmptyTime: 23s432ms
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- InputEmptyTime: 23s694ms
- ScheduleCount: 924
- ScheduleTime: 8.913ms
- TotalDegreeOfParallelism: 31
- YieldByTimeLimit: 0
RESULT_SINK:
CommonMetrics:- CloseTime: 7.136us
- OperatorTotalTime: 341.188us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 40
- __MAX_OF_PushChunkNum: 40
- __MIN_OF_PushChunkNum: 0
- PushRowNum: 90
- __MAX_OF_PushRowNum: 90
- __MIN_OF_PushRowNum: 0
- PushTotalTime: 333.757us
- SetFinishedTime: 78ns
- SetFinishingTime: 216ns
UniqueMetrics:
EXCHANGE_SOURCE (plan_node_id=1):
CommonMetrics: - CloseTime: 11.280us
- OperatorTotalTime: 194.28us
- PeakMemoryUsage: 0.00
- PullChunkNum: 68
- __MAX_OF_PullChunkNum: 68
- __MIN_OF_PullChunkNum: 0
- PullRowNum: 90
- __MAX_OF_PullRowNum: 90
- __MIN_OF_PullRowNum: 0
- PullTotalTime: 124.315us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 237ns
- SetFinishingTime: 58.195us
UniqueMetrics: - BytesPassThrough: 23.13 KB
- BytesReceived: 55.26 KB
- DecompressChunkTime: 525.24us
- DeserializeChunkTime: 8.555ms
- ReceiverProcessTotalTime: 10.397ms
- RequestReceived: 41
- SenderTotalTime: 10.377ms
- SenderWaitLockTime: 439.5us
Fragment 1:
- BackendNum: 5
- InstanceNum: 5
- MemoryLimit: 137.51 GB
- PeakMemoryUsage: 3.46 GB
Pipeline (id=1):- ActiveTime: 72.285ms
- __MAX_OF_ActiveTime: 194.657ms
- __MIN_OF_ActiveTime: 18.940ms
- BlockByInputEmpty: 1.385K (1385)
- __MAX_OF_BlockByInputEmpty: 339
- __MIN_OF_BlockByInputEmpty: 4
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 31
- DriverTotalTime: 9s203ms
- __MAX_OF_DriverTotalTime: 23s693ms
- __MIN_OF_DriverTotalTime: 276.406ms
- OverheadTime: 72.285ms
- __MAX_OF_OverheadTime: 194.657ms
- __MIN_OF_OverheadTime: 18.940ms
- PendingTime: 9s122ms
- InputEmptyTime: 8s422ms
- FirstInputEmptyTime: 490.66us
- FollowupInputEmptyTime: 8s421ms
- __MAX_OF_FollowupInputEmptyTime: 22s603ms
- __MIN_OF_FollowupInputEmptyTime: 61.625ms
- __MAX_OF_InputEmptyTime: 22s603ms
- __MIN_OF_InputEmptyTime: 62.330ms
- OutputFullTime: 0ns
- PendingFinishTime: 700.102ms
- __MAX_OF_PendingFinishTime: 3s713ms
- __MIN_OF_PendingFinishTime: 135.457us
- PreconditionBlockTime: 0ns
- __MAX_OF_PendingTime: 23s657ms
- __MIN_OF_PendingTime: 71.799ms
- InputEmptyTime: 8s422ms
- ScheduleCount: 1.54K (1540)
- __MAX_OF_ScheduleCount: 370
- __MIN_OF_ScheduleCount: 5
- ScheduleTime: 8.719ms
- __MAX_OF_ScheduleTime: 17.685ms
- __MIN_OF_ScheduleTime: 5.508ms
- TotalDegreeOfParallelism: 155
- YieldByTimeLimit: 0
EXCHANGE_SINK (plan_node_id=1):
CommonMetrics:- CloseTime: 1.478us
- OperatorTotalTime: 75.709us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 40
- __MAX_OF_PushChunkNum: 9
- __MIN_OF_PushChunkNum: 0
- PushRowNum: 90
- __MAX_OF_PushRowNum: 27
- __MIN_OF_PushRowNum: 0
- PushTotalTime: 36.119us
- SetFinishedTime: 69ns
- SetFinishingTime: 38.40us
UniqueMetrics: - DestID: 1
- DestFragments: 4912c56f7acb11ed-b09c0242f112ba6c
- PartType: UNPARTITIONED
- BytesPassThrough: 23.13 KB
- __MAX_OF_BytesPassThrough: 23.13 KB
- __MIN_OF_BytesPassThrough: 0.00
- BytesSent: 55.26 KB
- __MAX_OF_BytesSent: 17.19 KB
- __MIN_OF_BytesSent: 0.00
- CompressTime: 6.983us
- NetworkTime: 4.339ms
- __MAX_OF_NetworkTime: 20.669ms
- __MIN_OF_NetworkTime: 0ns
- OverallThroughput: 491.611328125 KB/sec
- __MAX_OF_OverallThroughput: 491.611328125 KB/sec
- __MIN_OF_OverallThroughput: 0.0 /sec
- RequestSent: 34
- __MAX_OF_RequestSent: 9
- __MIN_OF_RequestSent: 0
- SerializeChunkTime: 19.477us
- ShuffleHashTime: 0ns
- UncompressedBytes: 120.46 KB
- __MAX_OF_UncompressedBytes: 39.80 KB
- __MIN_OF_UncompressedBytes: 0.00
- WaitTime: 4.264ms
CHUNK_ACCUMULATE (plan_node_id=0):
CommonMetrics: - CloseTime: 621ns
- OperatorTotalTime: 15.195us
- PeakMemoryUsage: 0.00
- PullChunkNum: 40
- __MAX_OF_PullChunkNum: 9
- __MIN_OF_PullChunkNum: 0
- PullRowNum: 90
- __MAX_OF_PullRowNum: 27
- __MIN_OF_PullRowNum: 0
- PullTotalTime: 89ns
- PushChunkNum: 42
- __MAX_OF_PushChunkNum: 11
- __MIN_OF_PushChunkNum: 0
- PushRowNum: 90
- __MAX_OF_PushRowNum: 27
- __MIN_OF_PushRowNum: 0
- PushTotalTime: 14.42us
- SetFinishedTime: 176ns
- SetFinishingTime: 264ns
UniqueMetrics:
OLAP_SCAN (plan_node_id=0):
CommonMetrics: - CloseTime: 5.448ms
- OperatorTotalTime: 74.422ms
- __MAX_OF_OperatorTotalTime: 193.13ms
- __MIN_OF_OperatorTotalTime: 22.377ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 2.642K (2642)
- __MAX_OF_PullChunkNum: 576
- __MIN_OF_PullChunkNum: 7
- PullRowNum: 90
- __MAX_OF_PullRowNum: 27
- __MIN_OF_PullRowNum: 0
- PullTotalTime: 68.973ms
- __MAX_OF_PullTotalTime: 187.167ms
- __MIN_OF_PullTotalTime: 17.932ms
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 296ns
- SetFinishingTime: 257ns
UniqueMetrics: - MorselQueueType: fixed_morsel_queue
- Predicates: 7: RWID = ‘80ef6c28-c135-454f-89a6-1fa029f6a13b’, 9: BDYYID = ‘104bd953-33e9-4562-80d2-961b9b3590ec’, 4: BDFS = ‘MATERIAL’, 62: BDYQMC = ‘IFLYTEK’, 77: SFSC = 0
- Rollup: swk_comp_result_bk
- Table: swk_comp_result_bk
- BytesRead: 11.76 MB
- __MAX_OF_BytesRead: 2.99 MB
- __MIN_OF_BytesRead: 40.00 B
- CachedPagesNum: 0
- ChunkBufferCapacity: 9.92K (9920)
- CompressedBytesRead: 5.67 GB
- __MAX_OF_CompressedBytesRead: 1.15 GB
- __MIN_OF_CompressedBytesRead: 536.68 KB
- CreateSegmentIter: 1.864ms
- __MAX_OF_CreateSegmentIter: 17.613ms
- __MIN_OF_CreateSegmentIter: 447.489us
- DefaultChunkBufferCapacity: 2.045K (2045)
- __MAX_OF_DefaultChunkBufferCapacity: 630
- __MIN_OF_DefaultChunkBufferCapacity: 314
- DictDecode: 19.892us
- IOTime: 3s799ms
- __MAX_OF_IOTime: 20s480ms
- __MIN_OF_IOTime: 1.54ms
- LateMaterialize: 2s117ms
- __MAX_OF_LateMaterialize: 8s664ms
- __MIN_OF_LateMaterialize: 27.8ms
- MorselsCount: 1.65K (1650)
- __MAX_OF_MorselsCount: 330
- __MIN_OF_MorselsCount: 5
- PeakChunkBufferSize: 599
- PushdownPredicates: 5
- RawRowsRead: 279.205K (279205)
- __MAX_OF_RawRowsRead: 71.518K (71518)
- __MIN_OF_RawRowsRead: 0
- ReadPagesNum: 102.788K (102788)
- __MAX_OF_ReadPagesNum: 21.148K (21148)
- __MIN_OF_ReadPagesNum: 56
- RowsRead: 90
- __MAX_OF_RowsRead: 27
- __MIN_OF_RowsRead: 0
- ScanTime: 4s706ms
- __MAX_OF_ScanTime: 22s378ms
- __MIN_OF_ScanTime: 7.274ms
- SegmentInit: 4s533ms
- BitmapIndexFilter: 43.65ms
- __MAX_OF_BitmapIndexFilter: 1s777ms
- __MIN_OF_BitmapIndexFilter: 6.260us
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 52.153143M (52153143)
- __MAX_OF_BloomFilterFilterRows: 10.570829M (10570829)
- __MIN_OF_BloomFilterFilterRows: 2.977K (2977)
- SegmentZoneMapFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- __MAX_OF_SegmentInit: 21s632ms
- __MIN_OF_SegmentInit: 5.916ms
- BitmapIndexFilter: 43.65ms
- SegmentRead: 14.990ms
- BlockFetch: 7.685ms
- __MAX_OF_BlockFetch: 1s141ms
- __MIN_OF_BlockFetch: 3.669us
- BlockFetchCount: 103
- __MAX_OF_BlockFetchCount: 25
- __MIN_OF_BlockFetchCount: 0
- BlockSeek: 118.431ms
- __MAX_OF_BlockSeek: 2s30ms
- __MIN_OF_BlockSeek: 16.719us
- BlockSeekCount: 481
- __MAX_OF_BlockSeekCount: 145
- __MIN_OF_BlockSeekCount: 0
- ChunkCopy: 1.266us
- DecompressT: 32.415ms
- __MAX_OF_DecompressT: 2s919ms
- __MIN_OF_DecompressT: 888.40us
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 4.471us
- PredFilterRows: 279.115K (279115)
- __MAX_OF_PredFilterRows: 71.491K (71491)
- __MIN_OF_PredFilterRows: 0
- RowsetsReadCount: 1.65K (1650)
- __MAX_OF_RowsetsReadCount: 330
- __MIN_OF_RowsetsReadCount: 1
- SegmentsReadCount: 1.65K (1650)
- __MAX_OF_SegmentsReadCount: 330
- __MIN_OF_SegmentsReadCount: 1
- TotalColumnsDataPageCount: 547.439K (547439)
- __MAX_OF_TotalColumnsDataPageCount: 110.939K (110939)
- __MIN_OF_TotalColumnsDataPageCount: 86
- __MAX_OF_SegmentRead: 1s613ms
- __MIN_OF_SegmentRead: 2.750us
- BlockFetch: 7.685ms
- TabletCount: 1.65K (1650)
- UncompressedBytesRead: 8.12 GB
- __MAX_OF_UncompressedBytesRead: 1.64 GB
- __MIN_OF_UncompressedBytesRead: 734.38 KB
Pipeline (id=0):
- ActiveTime: 441.461us
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 4.454ms
- OverheadTime: 441.461us
- PendingTime: 0ns
- InputEmptyTime: 0ns
- FirstInputEmptyTime: 0ns
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- InputEmptyTime: 0ns
- ScheduleCount: 5
- ScheduleTime: 4.13ms
- TotalDegreeOfParallelism: 5
- YieldByTimeLimit: 0
NOOP_SINK (plan_node_id=0):
CommonMetrics:- CloseTime: 925ns
- OperatorTotalTime: 1.254us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 61ns
- SetFinishingTime: 268ns
UniqueMetrics:
OLAP_SCAN_PREPARE (plan_node_id=0):
CommonMetrics: - CloseTime: 12.387us
- OperatorTotalTime: 441.416us
- PeakMemoryUsage: 0.00
- PullChunkNum: 5
- PullRowNum: 0
- PullTotalTime: 428.465us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 133ns
- SetFinishingTime: 430ns
UniqueMetrics:
- ActiveTime: 72.285ms
- 并行度:show variables like ‘%parallel_fragment_exec_instance_num%’;
parallel_fragment_exec_instance_num=18 - pipeline是否开启:show variables like ‘%pipeline%’;
- Profile信息