【业务影响】影响
【是否存算分离】 是
【StarRocks版本】3.2.4
【集群规模】3个CN+3个FE
【机器信息】CPU虚拟核/内存/网卡，例如：16C/64G
并发高的时候，执行sql很慢，这是其中一条PROFILE，帮忙分析下
ANALYZE PROFILE FROM “eb89bf89-038d-11ef-a1a4-12a434d14977”;
±--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String |
±--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Summary |
| QueryId: eb89bf89-038d-11ef-a1a4-12a434d14977 |
| Version: 3.2.4-613f0b5 |
| State: Finished |
| TotalTime: 18s687ms |
| ExecutionTime: 18s668ms [Scan: 18s366ms (98.38%), Network: 14.323ms (0.08%), ResultDeliverTime: 0ns (0.00%), ScheduleTime: 137.233ms (0.74%)] |
| CollectProfileTime: 5ms |
| FrontendProfileMergeTime: 2.705ms |
| QueryPeakMemoryUsage: 124.407 MB, QueryAllocatedMemoryUsage: 65.271 GB |
| Top Most Time-consuming Nodes: |
| 1. OLAP_SCAN (id=0) : 18s391ms (99.92%) |
| 2. MERGE_EXCHANGE (id=3) [GATHER]: 14.545ms (0.08%) |
| 3. TOP_N (id=2) [ROW_NUMBER, TOP-N]: 82.051us (0.00%) |
| 4. RESULT_SINK: 72.965us (0.00%) |
| 5. PROJECT (id=1) : 10.403us (0.00%) |
| Top Most Memory-consuming Nodes: |
| 1. TOP_N (id=2) [ROW_NUMBER, TOP-N]: 33.398 KB |
| 2. MERGE_EXCHANGE (id=3) [GATHER]: 30.250 KB |
| NonDefaultVariables: |
| big_query_profile_threshold: 0s -> 5s |
| character_set_results: utf8 -> NULL |
| enable_adaptive_sink_dop: false -> true |
| enable_column_expr_predicate: true -> false |
| enable_scan_datacache: false -> true |
| exec_mem_limit: 2147483648 -> 5368709120 |
| io_tasks_per_scan_operator: 4 -> 8 |
| parallel_fragment_exec_instance_num: 1 -> 8 |
| query_mem_limit: 0 -> 10737418240 |
| sql_mode_v2: 32 -> 2097184 |
| Fragment 0 |
| │ BackendNum: 1 |
| │ InstancePeakMemoryUsage: 231.991 KB, InstanceAllocatedMemoryUsage: 1.174 MB |
| │ PrepareTime: 239.882us |
| └──RESULT_SINK |
| │ TotalTime: 72.965us (0.00%) [CPUTime: 72.965us] |
| │ OutputRows: 0 |
| │ SinkType: MYSQL_PROTOCAL |
| └──MERGE_EXCHANGE (id=3) [GATHER] |
| Estimates: [row: 200, cpu: 318768.65, memory: 318768.65, network: 318768.65, cost: 133435422.87] |
| TotalTime: 14.545ms (0.08%) [CPUTime: 222.411us, NetworkTime: 14.323ms] |
| OutputRows: 0 |
| PeakMemory: 30.250 KB, AllocatedMemory: 55.029 KB |
| SubordinateOperators: |
| LOCAL_EXCHANGE [Passthrough] |
| |
| Fragment 1 |
| │ BackendNum: 3 |
| │ InstancePeakMemoryUsage: 99.730 MB, InstanceAllocatedMemoryUsage: 65.270 GB |
| │ PrepareTime: 627.849us |
| └──DATA_STREAM_SINK (id=3) |
| │ PartitionType: UNPARTITIONED |
| └──TOP_N (id=2) [ROW_NUMBER, TOP-N] |
| │ Estimates: [row: 200, cpu: 52609124.38, memory: 318768.65, network: 52609124.38, cost: 132160348.26] |
| │ TotalTime: 82.051us (0.00%) [CPUTime: 82.051us] |
| │ OutputRows: 0 |
| │ PeakMemory: 33.398 KB, AllocatedMemory: 125.000 KB |
| │ OrderByExprs: [<slot 3> 3: table_time] |
| │ SubordinateOperators: |
| │ LOCAL_EXCHANGE [Passthrough] |
| └──PROJECT (id=1) |
| │ Estimates: [row: ?, cpu: ?, memory: ?, network: ?, cost: ?] |
| │ TotalTime: 10.403us (0.00%) [CPUTime: 10.403us] |
| │ OutputRows: 0 |
| │ Expression: [1: table_pk1, 2: table_pk2, 3: table_time, 5: table_source, …] |
| └──OLAP_SCAN (id=0) |
| Estimates: [row: 33007, cpu: 52609124.38, memory: 0.00, network: 0.00, cost: 26304562.19] |
| TotalTime: 18s391ms (99.92%) [CPUTime: 25.628ms, ScanTime: 18s366ms] |
| OutputRows: 0 |
| Table: : iot_tcp_log_pk |
| Detail Timers: [ScanTime = IOTaskExecTime + IOTaskWaitTime] |
| IOTaskExecTime: 5s162ms [min=14.621ms, max=14s963ms] |
| LateMaterialize: 3s954ms [min=1s557ms, max=10s95ms] |
| IOTaskWaitTime: 590.192ms [min=34.051us, max=4s888ms]
