【详述】某个复杂关联查询SQL,两个单独查询耗时大概为2s(1数据量)/2s(7数据量),两个union起来为6s,查看profile, 发现耗时主要集中在EXCHANGE_NODE,但查看步骤未发现具体耗时
【背景】做过哪些操作?
【业务影响】
【StarRocks版本】例如: 2.2.2 e27e2aa
【集群规模】3fe(1 follower+2observer)+3be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【联系方式】为了在解决问题过程中能及时联系到您获取一些日志信息,请补充下您的联系方式,例如:社区群4-小李或者邮箱,谢谢
【附件】
- Profile信息
- 并行度:show variables like ‘%parallel_fragment_exec_instance_num%’;
- pipeline是否开启:show variables like ‘%pipeline%’;
- 执行计划:explain costs + sql
- be节点cpu和内存使用率截图
Instance 06846cf4-9544-11ed-a125-bc97e113c9ff (host=TNetworkAddress(hostname:10.14.xxx.xxx, port:19060)):(Active: 5s514ms[5514374605ns], % non-child: 0.00%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 20.00 GB
- PeakMemoryUsage: 81.23 KB
- RowsProduced: 3
DataStreamSender (dst_id=1269, dst_fragments=[06846cf4954411ed-a125bc97e113cbd3]):(Active: 317.565us[317565ns], % non-child: 0.01%)
- PartType: UNPARTITIONED
- BytesSent: 1.21 KB
- CompressTime: 10.831us
- IgnoreRows: 0
- OverallThroughput: 3.7238235473632812 MB/sec
- SendRequestTime: 25.779us
- SerializeChunkTime: 15.309us
- ShuffleDispatchTime: 0ns
- ShuffleHashTime: 0ns
- UncompressedBytes: 3.79 KB
- WaitResponseTime: 240.835us
SORT_NODE (id=1268):(Active: 5s514ms[5514162360ns], % non-child: 0.01%)
- SortKeys: 6416: if ASC, 6311: round DESC
- SortType: TopN
- BuildingTime: 232.875us
- ChunksSorter: 398.936us
- MergingTime: 0ns
- OutputTime: 72.426us
- PeakMemoryUsage: 0.00
- RowsReturned: 3
- RowsReturnedRate: 0
- SortFilterCost: 0ns
- SortFilterRows: 0
- SortingTime: 0ns
PROJECT_NODE (id=1267):(Active: 5s513ms[5513650170ns], % non-child: 0.00%)
- CommonSubExprComputeTime: 327ns
- ExprComputeTime: 55.526us
- PeakMemoryUsage: 0.00
- RowsReturned: 3
- RowsReturnedRate: 0
UNION_NODE (id=0):(Active: 5s513ms[5513460533ns], % non-child: 0.00%)
- PeakMemoryUsage: 0.00
- RowsReturned: 3
- RowsReturnedRate: 0
EXCHANGE_NODE (id=1266):(Active: 3.596us[3596ns], % non-child: 0.00%)
- BytesPassThrough: 0.00
- BytesReceived: 405.00 B
- DecompressChunkTime: 6.193us
- DeserializeChunkTime: 78.941us
- PeakMemoryUsage: 0.00
- ReceiverProcessTotalTime: 88.273us
- RequestReceived: 1
- RowsReturned: 2
- RowsReturnedRate: 556.173K /sec
- SenderTotalTime: 88.69us
- SenderWaitLockTime: 155ns
EXCHANGE_NODE (id=633):(Active: 5s513ms[5513322810ns], % non-child: 93.49%)
- BytesPassThrough: 0.00
- BytesReceived: 862.00 B
- DecompressChunkTime: 1.957us
- DeserializeChunkTime: 109.177us
- PeakMemoryUsage: 0.00
- ReceiverProcessTotalTime: 117.262us
- RequestReceived: 1
- RowsReturned: 1
- RowsReturnedRate: 0
- SenderTotalTime: 117.70us
- SenderWaitLockTime: 96ns