为了更快的定位您的问题,请提供以下信息,谢谢
【详述】使用grouping sets吃内存过大
【背景】
【是否存算分离】否
【StarRocks版本】例如:3.3.3
【集群规模】例如:3fe(1 follower+2observer)+30be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:128C/1T/万兆
| Fragment 0 |
| │ BackendNum: 24 |
| │ InstancePeakMemoryUsage: 25.639 GB, InstanceAllocatedMemoryUsage: 63.063 TB |
| │ PrepareTime: 740.071ms |
| └──OLAP_TABLE_SINK |
| │ TotalTime: 109.767ms (0.04%) [CPUTime: 109.767ms] |
| │ OutputRows: 196.621K (196621) |
| │ PartitionType: RANDOM |
| │ Table: dwm_qb_car_all |
| └──PROJECT (id=80) |
| │ Estimates: [row: ?, cpu: ?, memory: ?, network: ?, cost: ?] |
| │ TotalTime: 14.601ms (0.01%) [CPUTime: 14.601ms] |
| │ OutputRows: 196.621K (196621) |
| │ Expression: [96: service_manager_code, 477: flag_p_gq_xxz, 820: if, 822: if, …] |
| └──AGGREGATION (id=79) [finalize, merge] |
| │ Estimates: [row: 155242, cpu: 1233418785.81, memory: 1363315936.59, network: 0.00, cost: 22836545643.17] |
| │ TotalTime: 17s258ms (6.38%) [CPUTime: 17s258ms] |
| │ OutputRows: 196.621K (196621) |
| │ PeakMemory: 52.314 GB, AllocatedMemory: 1.547 TB |
| │ AggExprs: [sum(1466: sum), sum(1467: sum), sum(1468: sum), sum(1469: sum), …] |
| │ GroupingExprs: [989: expr, 795: tjcomcode1, 820: if, 824: ifnull, …] |
| │ SubordinateOperators: |
| │ NOOP |
| │ SPILL_PROCESS |
| └──EXCHANGE (id=78) [SHUFFLE] |
| Estimates: [row: 164657, cpu: 123341878.58, memory: 0.00, network: 123341878.58, cost: 19493204377.08] |
| TotalTime: 43s993ms (16.26%) [CPUTime: 33s407ms, NetworkTime: 10s586ms] |
| OutputRows: 44.555M (44555199) |
| PeakMemory: 91.212 GB, AllocatedMemory: 3.464 TB |
| Detail Timers: |
| ClosureBlockTime: 900.908ms [min=17.291ms, max=34s56ms] |
| OverallTime: 30s250ms [min=6s517ms, max=1m6s] |
| ShuffleChunkAppendTime: 443.563ms [min=0ns, max=16s903ms] |
| |
| Fragment 1 |
| │ BackendNum: 24 |
| │ InstancePeakMemoryUsage: 41.636 GB, InstanceAllocatedMemoryUsage: 79.784 TB |
| │ PrepareTime: 239.629ms |
| └──DATA_STREAM_SINK (id=78) |
| │ PartitionType: HASH_PARTITIONED |
| │ PartitionExprs: [989: expr, 795: tjcomcode1, 820: if, 824: ifnull, …]