分别连接3个不同的fe节点,查询sql,耗时差几倍,原因是?

【详述】3个fe节点,连接不同的fe,查询sql,耗时查几倍。fe1耗时3.04 sec,fe2和f3耗时25s。
查询sql:explain SELECT DISTINCT url FROM new_frog_detail WHERE dt >= ‘2023-09-27’;
【背景】
【业务影响】
【StarRocks版本】2.2.4
【集群规模】例如:3fe(2 follower+ leader)+5be(fe与be独立部署)
【机器信息】48C/256G/万兆
【附件】

  • 并行度:
    MySQL [event_analysis]> show variables like ‘%parallel_fragment_exec_instance_num%’;
    ±------------------------------------±------+
    | Variable_name | Value |
    ±------------------------------------±------+
    | parallel_fragment_exec_instance_num | 1 |
    ±------------------------------------±------+

MySQL [event_analysis]> show variables like ‘%pipeline_dop%’;
±--------------±------+
| Variable_name | Value |
±--------------±------+
| pipeline_dop | 0 |
±--------------±------+

  • pipeline是否开启:
    MySQL [event_analysis]> show variables like ‘%pipeline%’;
    ±-----------------------±------+
    | Variable_name | Value |
    ±-----------------------±------+
    | enable_pipeline_engine | true |
    | pipeline_dop | 0 |
    | pipeline_profile_level | 1 |

fe1 profile:
Query:
Summary:
- Query ID: fa50167f-76fd-11ee-876c-f8f21e72dfc8
- Start Time: 2023-10-30 16:26:09
- End Time: 2023-10-30 16:26:12
- Total: 3s46ms
- Query Type: Query
- Query State: EOF
- StarRocks Version: 2.2.4
- User: root
- Default Db: default_cluster:event_analysis
- Sql Statement: SELECT DISTINCT url FROM new_frog_detail WHERE dt >= ‘2023-09-27’
- QueryCpuCost: 1m8s
- QueryMemCost: 2.076GB
- Variables: parallel_fragment_exec_instance_num=1,pipeline_dop=0
- Collect Profile Time: 11ms
Execution Profile fa50167f-76fd-11ee-876c-f8f21e72dfc8:
- ExecutionTotalTime: 3s38ms
Fragment 0:
- BackendNum: 1
- InstanceNum: 1
- MemoryLimit: 2.00 GB
- PeakMemoryUsage: 580.59 KB
Pipeline (id=0):
- ActiveTime: 3.389ms
- BlockByInputEmpty: 1
- BlockByOutputFull: 83
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 3s38ms
- OverheadTime: 3.389ms
- PendingTime: 3s34ms
- InputEmptyTime: 3s25ms
- FirstInputEmptyTime: 3s25ms
- FollowupInputEmptyTime: 27.728us
- OutputFullTime: 9.221ms
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 85
- ScheduleTime: 816.912us
- TotalDegreeOfParallelism: 1
- YieldByTimeLimit: 0
RESULT_SINK:
CommonMetrics:
- CloseTime: 4.69us
- OperatorTotalTime: 2.494ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 120
- PushRowNum: 5.31K (5310)
- PushTotalTime: 2.489ms
- SetFinishedTime: 51ns
- SetFinishingTime: 79ns
UniqueMetrics:
EXCHANGE_SOURCE (plan_node_id=5):
CommonMetrics:
- CloseTime: 774ns
- OperatorTotalTime: 299.985us
- PeakMemoryUsage: 0.00
- PullChunkNum: 120
- PullRowNum: 5.31K (5310)
- PullTotalTime: 294.366us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 98ns
- SetFinishingTime: 4.747us
UniqueMetrics:
- BytesPassThrough: 0.00
- BytesReceived: 165.30 KB
- DecompressChunkTime: 341.831us
- DeserializeChunkTime: 922.967us
- ReceiverProcessTotalTime: 1.643ms
- RequestReceived: 120
- SenderTotalTime: 1.622ms
- SenderWaitLockTime: 34.149us
Fragment 1:
- BackendNum: 5
- InstanceNum: 5
- MemoryLimit: 240.00 GB
- PeakMemoryUsage: 55.39 MB
Pipeline (id=2):
- ActiveTime: 417.139us
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 24
- DriverTotalTime: 3s19ms
- OverheadTime: 417.139us
- PendingTime: 3s17ms
- InputEmptyTime: 3s16ms
- FirstInputEmptyTime: 3s16ms
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 987.261us
- PreconditionBlockTime: 0ns
- ScheduleCount: 120
- ScheduleTime: 1.204ms
- TotalDegreeOfParallelism: 120
- YieldByTimeLimit: 0
EXCHANGE_SINK (plan_node_id=5):
CommonMetrics:
- CloseTime: 997ns
- OperatorTotalTime: 240.262us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 120
- PushRowNum: 5.31K (5310)
- __MAX_OF_PushRowNum: 1.09K (1090)
- __MIN_OF_PushRowNum: 25
- PushTotalTime: 17.693us
- SetFinishedTime: 65ns
- SetFinishingTime: 221.504us
UniqueMetrics:
- DestID: 5
- DestFragments: fa50167f76fd11ee-876cf8f21e72e046
- PartType: UNPARTITIONED
- BytesPassThrough: 0.00
- BytesSent: 165.30 KB
- CompressTime: 11.21us
- NetworkTime: 1.138ms
- OverallThroughput: 175.0586862564087 MB/sec
- __MAX_OF_OverallThroughput: 49.932671546936035 MB/sec
- __MIN_OF_OverallThroughput: 14.269261360168457 MB/sec
- RequestSent: 120
- SerializeChunkTime: 4.110us
- ShuffleHashTime: 0ns
- UncompressedBytes: 220.58 KB
- __MAX_OF_UncompressedBytes: 45.16 KB
- __MIN_OF_UncompressedBytes: 1.14 KB
- WaitTime: 408.214us
AGGREGATE_DISTINCT_BLOCKING_SOURCE (plan_node_id=4):
CommonMetrics:
- CloseTime: 3.475us
- OperatorTotalTime: 169.297us
- PeakMemoryUsage: 0.00
- PullChunkNum: 120
- PullRowNum: 5.31K (5310)
- __MAX_OF_PullRowNum: 1.09K (1090)
- __MIN_OF_PullRowNum: 25
- PullTotalTime: 165.610us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 110ns
- SetFinishingTime: 99ns
UniqueMetrics:
Pipeline (id=1):
- ActiveTime: 1.621ms
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 24
- DriverTotalTime: 3s20ms
- OverheadTime: 1.621ms
- PendingTime: 3s15ms
- InputEmptyTime: 3s15ms
- FirstInputEmptyTime: 3s15ms
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 120
- ScheduleTime: 3.435ms
- TotalDegreeOfParallelism: 120
- YieldByTimeLimit: 0
AGGREGATE_DISTINCT_BLOCKING_SINK (plan_node_id=4):
CommonMetrics:
- CloseTime: 8.203us
- OperatorTotalTime: 91.172us
- PeakMemoryUsage: 677.81 KB
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 120
- PushRowNum: 390.889K (390889)
- __MAX_OF_PushRowNum: 80.794K (80794)
- __MIN_OF_PushRowNum: 1.609K (1609)
- PushTotalTime: 81.638us
- SetFinishedTime: 92ns
- SetFinishingTime: 1.235us
UniqueMetrics:
- GroupingKeys: 5: url
- AggComputeTime: 77.698us
- ExprComputeTime: 1.743us
- ExprReleaseTime: 485ns
- GetResultsTime: 145.688us
- HashTableSize: 5.31K (5310)
- __MAX_OF_HashTableSize: 1.09K (1090)
- __MIN_OF_HashTableSize: 25
- InputRowCount: 390.889K (390889)
- __MAX_OF_InputRowCount: 80.794K (80794)
- __MIN_OF_InputRowCount: 1.609K (1609)
- PassThroughRowCount: 0
- ResultAggAppendTime: 0ns
- ResultGroupByAppendTime: 9.489us
- ResultIteratorTime: 0ns
- RowsReturned: 0
- StreamingTime: 0ns
LOCAL_EXCHANGE_SOURCE (pseudo_plan_node_id=-100):
CommonMetrics:
- CloseTime: 675ns
- OperatorTotalTime: 1.531ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 120
- PullRowNum: 390.889K (390889)
- __MAX_OF_PullRowNum: 80.794K (80794)
- __MIN_OF_PullRowNum: 1.609K (1609)
- PullTotalTime: 1.528ms
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 2.747us
- SetFinishingTime: 164ns
UniqueMetrics:
Pipeline (id=0):
- ActiveTime: 9.809ms
- __MAX_OF_ActiveTime: 21.256ms
- __MIN_OF_ActiveTime: 6.924ms
- BlockByInputEmpty: 13.3K (13300)
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 24
- DriverTotalTime: 3s20ms
- OverheadTime: 9.809ms
- __MAX_OF_OverheadTime: 21.256ms
- __MIN_OF_OverheadTime: 6.924ms
- PendingTime: 2s976ms
- InputEmptyTime: 2s980ms
- FirstInputEmptyTime: 1s869ms
- FollowupInputEmptyTime: 1s110ms
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 13.452K (13452)
- ScheduleTime: 33.646ms
- __MAX_OF_ScheduleTime: 49.920ms
- __MIN_OF_ScheduleTime: 21.999ms
- TotalDegreeOfParallelism: 120
- YieldByTimeLimit: 0
LOCAL_EXCHANGE_SINK (pseudo_plan_node_id=-100):
CommonMetrics:
- CloseTime: 453ns
- OperatorTotalTime: 759.848us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 14.4K (14400)
- PushRowNum: 390.889K (390889)
- __MAX_OF_PushRowNum: 80.794K (80794)
- __MIN_OF_PushRowNum: 1.609K (1609)
- PushTotalTime: 758.927us
- SetFinishedTime: 24ns
- SetFinishingTime: 442ns
UniqueMetrics:
- Type: Partition
EXCHANGE_SOURCE (plan_node_id=3):
CommonMetrics:
- CloseTime: 1.160us
- OperatorTotalTime: 2.105ms
- __MAX_OF_OperatorTotalTime: 8.322ms
- __MIN_OF_OperatorTotalTime: 1.320ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 14.4K (14400)
- PullRowNum: 390.889K (390889)
- __MAX_OF_PullRowNum: 80.794K (80794)
- __MIN_OF_PullRowNum: 1.609K (1609)
- PullTotalTime: 2.101ms
- __MAX_OF_PullTotalTime: 8.311ms
- __MIN_OF_PullTotalTime: 1.317ms
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 91ns
- SetFinishingTime: 3.387us
UniqueMetrics:
- BytesPassThrough: 0.00
- BytesReceived: 11.76 MB
- DecompressChunkTime: 3.646ms
- DeserializeChunkTime: 14.614ms
- __MAX_OF_DeserializeChunkTime: 17.957ms
- __MIN_OF_DeserializeChunkTime: 10.341ms
- ReceiverProcessTotalTime: 23.608ms
- __MAX_OF_ReceiverProcessTotalTime: 28.523ms
- __MIN_OF_ReceiverProcessTotalTime: 17.544ms
- RequestReceived: 600
- SenderTotalTime: 23.576ms
- __MAX_OF_SenderTotalTime: 28.474ms
- __MIN_OF_SenderTotalTime: 17.526ms
- SenderWaitLockTime: 64.698us
Fragment 2:
- BackendNum: 5
- InstanceNum: 120
- MemoryLimit: 240.00 GB
- PeakMemoryUsage: 13.90 GB
Pipeline (id=1):
- ActiveTime: 2.798ms
- __MAX_OF_ActiveTime: 7.960ms
- __MIN_OF_ActiveTime: 1.701ms
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 2s531ms
- __MAX_OF_DriverTotalTime: 3s6ms
- __MIN_OF_DriverTotalTime: 1s856ms
- OverheadTime: 2.798ms
- __MAX_OF_OverheadTime: 7.960ms
- __MIN_OF_OverheadTime: 1.701ms
- PendingTime: 2s527ms
- InputEmptyTime: 2s526ms
- FirstInputEmptyTime: 2s526ms
- __MAX_OF_FirstInputEmptyTime: 3s3ms
- __MIN_OF_FirstInputEmptyTime: 1s851ms
- FollowupInputEmptyTime: 0ns
- __MAX_OF_InputEmptyTime: 3s3ms
- __MIN_OF_InputEmptyTime: 1s851ms
- OutputFullTime: 0ns
- PendingFinishTime: 1.351ms
- __MAX_OF_PendingFinishTime: 20.761ms
- __MIN_OF_PendingFinishTime: 243.677us
- PreconditionBlockTime: 0ns
- __MAX_OF_PendingTime: 3s4ms
- __MIN_OF_PendingTime: 1s852ms
- ScheduleCount: 120
- ScheduleTime: 399.605us
- __MAX_OF_ScheduleTime: 25.636ms
- __MIN_OF_ScheduleTime: 55.703us
- TotalDegreeOfParallelism: 120
- YieldByTimeLimit: 0
EXCHANGE_SINK (plan_node_id=3):
CommonMetrics:
- CloseTime: 21.339us
- OperatorTotalTime: 2.305ms
- __MAX_OF_OperatorTotalTime: 6.866ms
- __MIN_OF_OperatorTotalTime: 1.466ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 120
- PushRowNum: 390.889K (390889)
- PushTotalTime: 1.51ms
- SetFinishedTime: 174ns
- SetFinishingTime: 1.232ms
UniqueMetrics:
- DestID: 3
- DestFragments: fa50167f76fd11ee-876cf8f21e72e041, fa50167f76fd11ee-876cf8f21e72e042, fa50167f76fd11ee-876cf8f21e72e043, fa50167f76fd11ee-876cf8f21e72e044, fa50167f76fd11ee-876cf8f21e72e045
- PartType: HASH_PARTITIONED
- BytesPassThrough: 0.00
- BytesSent: 11.76 MB
- CompressTime: 625.851us
- NetworkTime: 737.439us
- __MAX_OF_NetworkTime: 6.515ms
- __MIN_OF_NetworkTime: 112.413us
- OverallThroughput: 36.00132657960057 GB/sec
- __MAX_OF_OverallThroughput: 841.7662811279297 MB/sec
- __MIN_OF_OverallThroughput: 14.876653671264648 MB/sec
- RequestSent: 600
- SerializeChunkTime: 220.528us
- ShuffleHashTime: 220.607us
- UncompressedBytes: 15.27 MB
- WaitTime: 1.476ms
- __MAX_OF_WaitTime: 25.611ms
- __MIN_OF_WaitTime: 288.198us
AGGREGATE_DISTINCT_STREAMING_SOURCE (plan_node_id=2):
CommonMetrics:
- CloseTime: 12.765us
- OperatorTotalTime: 515.370us
- PeakMemoryUsage: 0.00
- PullChunkNum: 120
- PullRowNum: 390.889K (390889)
- PullTotalTime: 502.29us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 219ns
- SetFinishingTime: 355ns
UniqueMetrics:
Pipeline (id=0):
- ActiveTime: 2s482ms
- __MAX_OF_ActiveTime: 2s990ms
- __MIN_OF_ActiveTime: 1s782ms
- BlockByInputEmpty: 9.101K (9101)
- __MAX_OF_BlockByInputEmpty: 633
- __MIN_OF_BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 2s527ms
- __MAX_OF_DriverTotalTime: 3s4ms
- __MIN_OF_DriverTotalTime: 1s853ms
- OverheadTime: 2s482ms
- __MAX_OF_OverheadTime: 2s990ms
- __MIN_OF_OverheadTime: 1s782ms
- PendingTime: 34.891ms
- InputEmptyTime: 36.2ms
- FirstInputEmptyTime: 8.146ms
- __MAX_OF_FirstInputEmptyTime: 88.529ms
- __MIN_OF_FirstInputEmptyTime: 0ns
- FollowupInputEmptyTime: 27.855ms
- __MAX_OF_FollowupInputEmptyTime: 136.105ms
- __MIN_OF_FollowupInputEmptyTime: 0ns
- __MAX_OF_InputEmptyTime: 155.408ms
- __MIN_OF_InputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- __MAX_OF_PendingTime: 154.864ms
- __MIN_OF_PendingTime: 0ns
- ScheduleCount: 12.062K (12062)
- __MAX_OF_ScheduleCount: 663
- __MIN_OF_ScheduleCount: 23
- ScheduleTime: 10.914ms
- __MAX_OF_ScheduleTime: 42.403ms
- __MIN_OF_ScheduleTime: 1.576ms
- TotalDegreeOfParallelism: 120
- YieldByTimeLimit: 2.75K (2750)
AGGREGATE_DISTINCT_STREAMING_SINK (plan_node_id=2):
CommonMetrics:
- CloseTime: 10.972us
- OperatorTotalTime: 1s967ms
- __MAX_OF_OperatorTotalTime: 2s403ms
- __MIN_OF_OperatorTotalTime: 1s426ms
- PeakMemoryUsage: 41.45 MB
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 1.696391M (1696391)
- PushRowNum: 6.944909449B (6944909449)
- PushTotalTime: 1s967ms
- __MAX_OF_PushTotalTime: 2s403ms
- __MIN_OF_PushTotalTime: 1s426ms
- SetFinishedTime: 64ns
- SetFinishingTime: 591ns
UniqueMetrics:
- GroupingKeys: 5: url
- AggComputeTime: 1s822ms
- __MAX_OF_AggComputeTime: 2s224ms
- __MIN_OF_AggComputeTime: 1s302ms
- ExprComputeTime: 3.499ms
- __MAX_OF_ExprComputeTime: 19.865ms
- __MIN_OF_ExprComputeTime: 1.857ms
- ExprReleaseTime: 132.324ms
- __MAX_OF_ExprReleaseTime: 189.804ms
- __MIN_OF_ExprReleaseTime: 73.827ms
- GetResultsTime: 461.510us
- HashTableSize: 390.889K (390889)
- InputRowCount: 6.944909449B (6944909449)
- PassThroughRowCount: 0
- ResultAggAppendTime: 0ns
- ResultGroupByAppendTime: 330.657us
- ResultIteratorTime: 0ns
- RowsReturned: 0
- StreamingTime: 0ns
PROJECT (plan_node_id=1):
CommonMetrics:
- CloseTime: 1.44us
- OperatorTotalTime: 33.961ms
- __MAX_OF_OperatorTotalTime: 59.957ms
- __MIN_OF_OperatorTotalTime: 21.181ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 1.696391M (1696391)
- PullRowNum: 6.944909449B (6944909449)
- PullTotalTime: 2.180ms
- __MAX_OF_PullTotalTime: 16.549ms
- __MIN_OF_PullTotalTime: 1.166ms
- PushChunkNum: 1.696391M (1696391)
- PushRowNum: 6.944909449B (6944909449)
- PushTotalTime: 31.779ms
- __MAX_OF_PushTotalTime: 58.243ms
- __MIN_OF_PushTotalTime: 19.891ms
- SetFinishedTime: 30ns
- SetFinishingTime: 233ns
UniqueMetrics:
OLAP_SCAN (plan_node_id=0):
CommonMetrics:
- CloseTime: 1.14ms
- OperatorTotalTime: 90.958ms
- __MAX_OF_OperatorTotalTime: 157.409ms
- __MIN_OF_OperatorTotalTime: 42.286ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 1.697533M (1697533)
- PullRowNum: 6.944909449B (6944909449)
- PullTotalTime: 89.943ms
- __MAX_OF_PullTotalTime: 156.768ms
- __MIN_OF_PullTotalTime: 41.238ms
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 345ns
- SetFinishingTime: 215ns
UniqueMetrics:
- Rollup: new_frog_detail
- Table: new_frog_detail
- BytesRead: 0.00
- CachedPagesNum: 0
- CompressedBytesRead: 1.76 GB
- __MAX_OF_CompressedBytesRead: 16.41 MB
- __MIN_OF_CompressedBytesRead: 1.04 MB
- CreateSegmentIter: 107.729us
- __MAX_OF_CreateSegmentIter: 10.519ms
- __MIN_OF_CreateSegmentIter: 14.845us
- IOTime: 9.174ms
- __MAX_OF_IOTime: 34.775ms
- __MIN_OF_IOTime: 1.215ms
- PushdownPredicates: 0
- RawRowsRead: 6.944909449B (6944909449)
- __MAX_OF_RawRowsRead: 74.759718M (74759718)
- __MIN_OF_RawRowsRead: 3.972148M (3972148)
- ReadPagesNum: 850.929K (850929)
- __MAX_OF_ReadPagesNum: 9.148K (9148)
- __MIN_OF_ReadPagesNum: 487
- RowsRead: 6.944909449B (6944909449)
- __MAX_OF_RowsRead: 74.759718M (74759718)
- __MIN_OF_RowsRead: 3.972148M (3972148)
- ScanTime: 693.868ms
- __MAX_OF_ScanTime: 2s79ms
- __MIN_OF_ScanTime: 113.536ms
- SegmentInit: 643.533us
- BitmapIndexFilter: 0ns
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- __MAX_OF_SegmentInit: 17.104ms
- __MIN_OF_SegmentInit: 33.802us
- SegmentRead: 671.341ms
- BlockFetch: 476.886ms
- __MAX_OF_BlockFetch: 1s499ms
- __MIN_OF_BlockFetch: 83.271ms
- BlockFetchCount: 1.696391M (1696391)
- __MAX_OF_BlockFetchCount: 18.257K (18257)
- __MIN_OF_BlockFetchCount: 970
- BlockSeek: 189.600ms
- __MAX_OF_BlockSeek: 572.63ms
- __MIN_OF_BlockSeek: 25.94ms
- BlockSeekCount: 1.696391M (1696391)
- __MAX_OF_BlockSeekCount: 18.257K (18257)
- __MIN_OF_BlockSeekCount: 970
- ChunkCopy: 0ns
- DecompressT: 52.394ms
- __MAX_OF_DecompressT: 204.43ms
- __MIN_OF_DecompressT: 1.255ms
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 0ns
- PredFilterRows: 0
- RowsetsReadCount: 2.077K (2077)
- __MAX_OF_RowsetsReadCount: 27
- __MIN_OF_RowsetsReadCount: 1
- SegmentsReadCount: 1.433K (1433)
- __MAX_OF_SegmentsReadCount: 17
- __MIN_OF_SegmentsReadCount: 1
- TotalColumnsDataPageCount: 849.496K (849496)
- __MAX_OF_TotalColumnsDataPageCount: 9.138K (9138)
- __MIN_OF_TotalColumnsDataPageCount: 486
- __MAX_OF_SegmentRead: 2s28ms
- __MIN_OF_SegmentRead: 111.723ms
- UncompressedBytesRead: 2.09 GB
- __MAX_OF_UncompressedBytesRead: 19.69 MB
- __MIN_OF_UncompressedBytesRead: 1.30 MB

fe2 profile:
Query:
Summary:
- Query ID: cd272b7a-76d2-11ee-bfc7-f8f21e72d1c4
- Start Time: 2023-10-30 11:17:05
- End Time: 2023-10-30 11:17:30
- Total: 25s138ms
- Query Type: Query
- Query State: EOF
- StarRocks Version: 2.2.4
- User: root
- Default Db: default_cluster:event_analysis
- Sql Statement: SELECT DISTINCT url FROM new_frog_detail WHERE dt >= ‘2023-09-27’
- QueryCpuCost: 24s335ms
- QueryMemCost: 101.662MB
- Variables: parallel_fragment_exec_instance_num=1,pipeline_dop=0
- Collect Profile Time: 15ms
Execution Profile cd272b7a-76d2-11ee-bfc7-f8f21e72d1c4:
- ExecutionTotalTime: 25s123ms
Fragment 0:
- BackendNum: 1
- InstanceNum: 1
- MemoryLimit: 2.00 GB
- PeakMemoryUsage: 469.15 KB
Pipeline (id=0):
- ActiveTime: 2.233ms
- BlockByInputEmpty: 0
- BlockByOutputFull: 3
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 25s123ms
- OverheadTime: 2.233ms
- PendingTime: 25s120ms
- InputEmptyTime: 25s117ms
- FirstInputEmptyTime: 25s117ms
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 3.758ms
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 4
- ScheduleTime: 60.837us
- TotalDegreeOfParallelism: 1
- YieldByTimeLimit: 0
RESULT_SINK:
CommonMetrics:
- CloseTime: 6.415us
- OperatorTotalTime: 1.836ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 5
- PushRowNum: 5.297K (5297)
- PushTotalTime: 1.830ms
- SetFinishedTime: 138ns
- SetFinishingTime: 93ns
UniqueMetrics:
EXCHANGE_SOURCE (plan_node_id=5):
CommonMetrics:
- CloseTime: 1.595us
- OperatorTotalTime: 52.909us
- PeakMemoryUsage: 0.00
- PullChunkNum: 5
- PullRowNum: 5.297K (5297)
- PullTotalTime: 48.401us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 124ns
- SetFinishingTime: 2.789us
UniqueMetrics:
- BytesPassThrough: 0.00
- BytesReceived: 114.80 KB
- DecompressChunkTime: 171.3us
- DeserializeChunkTime: 167.837us
- ReceiverProcessTotalTime: 368.47us
- RequestReceived: 5
- SenderTotalTime: 365.742us
- SenderWaitLockTime: 1.17us
Fragment 1:
- BackendNum: 5
- InstanceNum: 5
- MemoryLimit: 10.00 GB
- PeakMemoryUsage: 2.41 MB
Pipeline (id=1):
- ActiveTime: 599.962us
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 25s115ms
- OverheadTime: 599.962us
- PendingTime: 25s114ms
- InputEmptyTime: 25s114ms
- FirstInputEmptyTime: 25s114ms
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 354.271us
- PreconditionBlockTime: 0ns
- ScheduleCount: 5
- ScheduleTime: 142.842us
- TotalDegreeOfParallelism: 5
- YieldByTimeLimit: 0
EXCHANGE_SINK (plan_node_id=5):
CommonMetrics:
- CloseTime: 73.828us
- OperatorTotalTime: 343.955us
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 5
- PushRowNum: 5.297K (5297)
- PushTotalTime: 162.508us
- SetFinishedTime: 64ns
- SetFinishingTime: 107.554us
UniqueMetrics:
- DestID: 5
- DestFragments: cd272b7a76d211ee-bfc7f8f21e72d1cf
- PartType: UNPARTITIONED
- BytesPassThrough: 0.00
- BytesSent: 114.80 KB
- CompressTime: 141.623us
- NetworkTime: 437.362us
- OverallThroughput: 209.35264492034912 MB/sec
- __MAX_OF_OverallThroughput: 91.83943367004395 MB/sec
- __MIN_OF_OverallThroughput: 0.0 /sec
- RequestSent: 5
- SerializeChunkTime: 14.968us
- ShuffleHashTime: 0ns
- UncompressedBytes: 216.82 KB
- WaitTime: 532.941us
AGGREGATE_DISTINCT_BLOCKING_SOURCE (plan_node_id=4):
CommonMetrics:
- CloseTime: 10.939us
- OperatorTotalTime: 201.494us
- PeakMemoryUsage: 0.00
- PullChunkNum: 5
- PullRowNum: 5.297K (5297)
- PullTotalTime: 190.190us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 113ns
- SetFinishingTime: 251ns
UniqueMetrics:
Pipeline (id=0):
- ActiveTime: 871.750us
- BlockByInputEmpty: 20
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 25s114ms
- OverheadTime: 871.750us
- PendingTime: 25s113ms
- InputEmptyTime: 25s113ms
- FirstInputEmptyTime: 24s176ms
- FollowupInputEmptyTime: 937.85ms
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 25
- ScheduleTime: 357.31us
- TotalDegreeOfParallelism: 5
- YieldByTimeLimit: 0
AGGREGATE_DISTINCT_BLOCKING_SINK (plan_node_id=4):
CommonMetrics:
- CloseTime: 15.38us
- OperatorTotalTime: 489.460us
- PeakMemoryUsage: 550.04 KB
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 25
- PushRowNum: 24.124K (24124)
- PushTotalTime: 471.385us
- SetFinishedTime: 23ns
- SetFinishingTime: 3.13us
UniqueMetrics:
- GroupingKeys: 5: url
- AggComputeTime: 377.427us
- ExprComputeTime: 11.83us
- ExprReleaseTime: 73.179us
- GetResultsTime: 161.138us
- HashTableSize: 5.297K (5297)
- InputRowCount: 24.124K (24124)
- PassThroughRowCount: 0
- ResultAggAppendTime: 0ns
- ResultGroupByAppendTime: 85.583us
- ResultIteratorTime: 0ns
- RowsReturned: 0
- StreamingTime: 0ns
EXCHANGE_SOURCE (plan_node_id=3):
CommonMetrics:
- CloseTime: 1.816us
- OperatorTotalTime: 334.296us
- PeakMemoryUsage: 0.00
- PullChunkNum: 25
- PullRowNum: 24.124K (24124)
- PullTotalTime: 326.196us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 225ns
- SetFinishingTime: 6.58us
UniqueMetrics:
- BytesPassThrough: 0.00
- BytesReceived: 511.51 KB
- DecompressChunkTime: 153.176us
- DeserializeChunkTime: 192.172us
- ReceiverProcessTotalTime: 420.820us
- RequestReceived: 25
- SenderTotalTime: 418.892us
- SenderWaitLockTime: 2.617us
Fragment 2:
- BackendNum: 5
- InstanceNum: 5
- MemoryLimit: 10.00 GB
- PeakMemoryUsage: 533.85 MB
Pipeline (id=1):
- ActiveTime: 2.300ms
- BlockByInputEmpty: 0
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 24s628ms
- OverheadTime: 2.300ms
- PendingTime: 24s626ms
- InputEmptyTime: 24s624ms
- FirstInputEmptyTime: 24s624ms
- FollowupInputEmptyTime: 0ns
- OutputFullTime: 0ns
- PendingFinishTime: 1.878ms
- PreconditionBlockTime: 0ns
- ScheduleCount: 5
- ScheduleTime: 120.301us
- TotalDegreeOfParallelism: 5
- YieldByTimeLimit: 0
EXCHANGE_SINK (plan_node_id=3):
CommonMetrics:
- CloseTime: 37.906us
- OperatorTotalTime: 1.642ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 5
- PushRowNum: 24.124K (24124)
- PushTotalTime: 525.885us
- SetFinishedTime: 137ns
- SetFinishingTime: 1.79ms
UniqueMetrics:
- DestID: 3
- DestFragments: cd272b7a76d211ee-bfc7f8f21e72d1ca, cd272b7a76d211ee-bfc7f8f21e72d1cb, cd272b7a76d211ee-bfc7f8f21e72d1cc, cd272b7a76d211ee-bfc7f8f21e72d1cd, cd272b7a76d211ee-bfc7f8f21e72d1ce
- PartType: HASH_PARTITIONED
- BytesPassThrough: 0.00
- BytesSent: 511.51 KB
- CompressTime: 631.59us
- NetworkTime: 1.505ms
- OverallThroughput: 578.6812677383423 MB/sec
- __MAX_OF_OverallThroughput: 231.12549781799316 MB/sec
- __MIN_OF_OverallThroughput: 39.22741985321045 MB/sec
- RequestSent: 25
- SerializeChunkTime: 54.148us
- ShuffleHashTime: 288.933us
- UncompressedBytes: 964.48 KB
- WaitTime: 1.936ms
AGGREGATE_DISTINCT_STREAMING_SOURCE (plan_node_id=2):
CommonMetrics:
- CloseTime: 11.587us
- OperatorTotalTime: 693.10us
- PeakMemoryUsage: 0.00
- PullChunkNum: 5
- PullRowNum: 24.124K (24124)
- PullTotalTime: 680.882us
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 227ns
- SetFinishingTime: 312ns
UniqueMetrics:
Pipeline (id=0):
- ActiveTime: 24s620ms
- BlockByInputEmpty: 186
- __MAX_OF_BlockByInputEmpty: 118
- __MIN_OF_BlockByInputEmpty: 3
- BlockByOutputFull: 0
- BlockByPrecondition: 0
- DegreeOfParallelism: 1
- DriverTotalTime: 24s625ms
- OverheadTime: 24s620ms
- PendingTime: 1.388ms
- InputEmptyTime: 1.440ms
- FirstInputEmptyTime: 34.429us
- FollowupInputEmptyTime: 1.405ms
- OutputFullTime: 0ns
- PendingFinishTime: 0ns
- PreconditionBlockTime: 0ns
- ScheduleCount: 1.414K (1414)
- ScheduleTime: 4.49ms
- TotalDegreeOfParallelism: 5
- YieldByTimeLimit: 1.222K (1222)
AGGREGATE_DISTINCT_STREAMING_SINK (plan_node_id=2):
CommonMetrics:
- CloseTime: 9.323us
- OperatorTotalTime: 22s510ms
- PeakMemoryUsage: 2.21 MB
- PullChunkNum: 0
- PullRowNum: 0
- PullTotalTime: 0ns
- PushChunkNum: 1.692595M (1692595)
- PushRowNum: 6.928712344B (6928712344)
- PushTotalTime: 22s510ms
- SetFinishedTime: 28ns
- SetFinishingTime: 386ns
UniqueMetrics:
- GroupingKeys: 5: url
- AggComputeTime: 21s998ms
- ExprComputeTime: 31.582ms
- ExprReleaseTime: 411.641ms
- __MAX_OF_ExprReleaseTime: 445.813ms
- __MIN_OF_ExprReleaseTime: 362.77ms
- GetResultsTime: 644.609us
- HashTableSize: 24.124K (24124)
- InputRowCount: 6.928712344B (6928712344)
- PassThroughRowCount: 0
- ResultAggAppendTime: 0ns
- ResultGroupByAppendTime: 496.824us
- ResultIteratorTime: 0ns
- RowsReturned: 0
- StreamingTime: 0ns
PROJECT (plan_node_id=1):
CommonMetrics:
- CloseTime: 1.674us
- OperatorTotalTime: 324.19ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 1.692595M (1692595)
- PullRowNum: 6.928712344B (6928712344)
- PullTotalTime: 22.181ms
- PushChunkNum: 1.692595M (1692595)
- PushRowNum: 6.928712344B (6928712344)
- PushTotalTime: 301.836ms
- SetFinishedTime: 30ns
- SetFinishingTime: 156ns
UniqueMetrics:
OLAP_SCAN (plan_node_id=0):
CommonMetrics:
- CloseTime: 1.51ms
- OperatorTotalTime: 401.339ms
- PeakMemoryUsage: 0.00
- PullChunkNum: 1.693684M (1693684)
- PullRowNum: 6.928712344B (6928712344)
- PullTotalTime: 400.287ms
- PushChunkNum: 0
- PushRowNum: 0
- PushTotalTime: 0ns
- SetFinishedTime: 266ns
- SetFinishingTime: 199ns
UniqueMetrics:
- Rollup: new_frog_detail
- Table: new_frog_detail
- BytesRead: 0.00
- CachedPagesNum: 0
- CompressedBytesRead: 1.76 GB
- __MAX_OF_CompressedBytesRead: 360.83 MB
- __MIN_OF_CompressedBytesRead: 1.52 MB
- CreateSegmentIter: 745.832us
- IOTime: 73.489ms
- __MAX_OF_IOTime: 237.244ms
- __MIN_OF_IOTime: 1.125ms
- PushdownPredicates: 0
- RawRowsRead: 6.928712344B (6928712344)
- __MAX_OF_RawRowsRead: 1.393167925B (1393167925)
- __MIN_OF_RawRowsRead: 5.270851M (5270851)
- ReadPagesNum: 849.572K (849572)
- __MAX_OF_ReadPagesNum: 170.853K (170853)
- __MIN_OF_ReadPagesNum: 645
- RowsRead: 6.928712344B (6928712344)
- __MAX_OF_RowsRead: 1.393167925B (1393167925)
- __MIN_OF_RowsRead: 5.270851M (5270851)
- ScanTime: 7s143ms
- __MAX_OF_ScanTime: 23s293ms
- __MIN_OF_ScanTime: 106.40ms
- SegmentInit: 4.407ms
- BitmapIndexFilter: 0ns
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- __MAX_OF_SegmentInit: 17.370ms
- __MIN_OF_SegmentInit: 50.407us
- SegmentRead: 7s58ms
- BlockFetch: 5s327ms
- __MAX_OF_BlockFetch: 17s515ms
- __MIN_OF_BlockFetch: 77.160ms
- BlockFetchCount: 1.692595M (1692595)
- __MAX_OF_BlockFetchCount: 340.336K (340336)
- __MIN_OF_BlockFetchCount: 1.287K (1287)
- BlockSeek: 1s701ms
- __MAX_OF_BlockSeek: 5s473ms
- __MIN_OF_BlockSeek: 26.319ms
- BlockSeekCount: 1.692595M (1692595)
- __MAX_OF_BlockSeekCount: 340.336K (340336)
- __MIN_OF_BlockSeekCount: 1.287K (1287)
- ChunkCopy: 0ns
- DecompressT: 60.907ms
- __MAX_OF_DecompressT: 191.510ms
- __MIN_OF_DecompressT: 902.150us
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 0ns
- PredFilterRows: 0
- RowsetsReadCount: 2.266K (2266)
- __MAX_OF_RowsetsReadCount: 466
- __MIN_OF_RowsetsReadCount: 1
- SegmentsReadCount: 1.646K (1646)
- __MAX_OF_SegmentsReadCount: 337
- __MIN_OF_SegmentsReadCount: 1
- TotalColumnsDataPageCount: 847.926K (847926)
- __MAX_OF_TotalColumnsDataPageCount: 170.516K (170516)
- __MIN_OF_TotalColumnsDataPageCount: 644
- __MAX_OF_SegmentRead: 23s24ms
- __MIN_OF_SegmentRead: 104.674ms
- UncompressedBytesRead: 2.08 GB
- __MAX_OF_UncompressedBytesRead: 427.63 MB
- __MIN_OF_UncompressedBytesRead: 1.78 MB

Explain String |
±------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS:5: url |
| PARTITION: UNPARTITIONED |
| |
| RESULT SINK |
| |
| 5:EXCHANGE |
| |
| PLAN FRAGMENT 1 |
| OUTPUT EXPRS: |
| PARTITION: HASH_PARTITIONED: 5: url |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 05 |
| UNPARTITIONED |
| |
| 4:AGGREGATE (merge finalize) |
| | group by: 5: url |
| | |
| 3:EXCHANGE |
| |
| PLAN FRAGMENT 2 |
| OUTPUT EXPRS: |
| PARTITION: RANDOM |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 03 |
| HASH_PARTITIONED: 5: url |
| |
| 2:AGGREGATE (update serialize) |
| | STREAMING |
| | group by: 5: url |
| | |
| 1:Project |
| | <slot 5> : 5: url |
| | |
| 0:OlapScanNode |
| TABLE: new_frog_detail |
| PREAGGREGATION: ON |
| partitions=34/182 |
| rollup: new_frog_detail |
| tabletRatio=1088/1088 |
| tabletList=305527,305531,305535,305539,305543,305547,305551,305555,305559,305563 … |
| cardinality=6945353940 |
| avgRowSize=36.37276 |
| numNodes=0

2.2.4版本太低了,已经不维护了,请升级2.5最新版本进行测试。

1赞

想确认下,2.2是否存在这样的bug?

2.5.13 未出现