【详述】问题详细描述
对6亿条数据求uv操作,普通查询count(distinct vid) 要 56s, vid是String 字母的字符串
创建物化视图语句
CREATE MATERIALIZED VIEW bitmap_uv AS
select dt,hours,channel,sub_channel,country_id,site,lang,bitmap_union(to_bitmap(bit_vid)) from mds_log_weblog_click_bak_bitmap group by dt,hours,channel,sub_channel,country_id,site,lang
创建物化视图后,
count(distinct bit_vid) 查询要56s ,bit_vid 是int型
.text-only,.text-card-text{white-space: pre;}.rich-text-paragraph{min-height: 15px;}2021-11-05 10:39:24,153 [query] |Client=127.0.0.1:38202|User=root|Db=default_cluster:hive_test|State=EOF|Time=54647|ScanBytes=3108051208|ScanRows=41859638|ReturnRows=2994862|StmtId=4905|QueryId=75ceb70e-3de1-11ec-ae6c-bc97e14b7ae4|IsQuery=true|feIp=|Stmt=select dt,hours,channel,sub_channel,country_id,site,lang,count(distinct bit_vid ) from mds_log_weblog_click_bak_bitmap group by dt,hours,channel,sub_channel,country_id,site,lang
.text-only,.text-card-text{white-space: pre;}.rich-text-paragraph{min-height: 15px;}2021-11-05 10:40:41,324 [slow_query] |Client=127.0.0.1:38202|User=root|Db=default_cluster:hive_test|State=EOF|Time=56843|ScanBytes=25519632583|ScanRows=639082263|ReturnRows=2994862|StmtId=4907|QueryId=a27efe01-3de1-11ec-ae6c-bc97e14b7ae4|IsQuery=true|feIp=[](|Stmt=select dt,hours,channel,sub_channel,country_id,site,lang,count(distinct vid ) from mds_log_weblog_click_bak_bitmap group by dt,hours,channel,sub_channel,country_id,site,lang
Query:
Summary:
- Query ID: 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae4
- Start Time: 2021-11-05 11:28:18
- End Time: 2021-11-05 11:29:12
- Total: 54s342ms
- Query Type: Query
- Query State: EOF
- Doris Version: 1.18.2
- User: root
- Default Db: default_cluster:hive_test
- Sql Statement: select dt,hours,channel,sub_channel,country_id,site,lang,count(distinct bit_vid )
from mds_log_weblog_click_bak_bitmap group by dt,hours,channel,sub_channel,country_id,site,lang
Execution Profile 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae4:(Active: 54s341ms[54341620208ns], % non-child: 100.00%)
Fragment 0:
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7aeb (host=TNetworkAddress(hostname:, port:9060)):(Active: 54s332ms[54332355402ns], % non-child: 1.94%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 32.00 KB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 2.994862M (2994862)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataBufferSender (dst_fragment_instance_id=6b3c8cac-3de8-11ec-ae6c-bc97e14b7aeb):
- AppendBatchTime: 1s36ms
- ResultRendTime: 202.932ms
- TupleConvertTime: 819.895ms
- NumSentRows: 2.994862M (2994862)
EXCHANGE_NODE (id=4):(Active: 53s276ms[53276511684ns], % non-child: 98.04%)
- BytesReceived: 93.10 MB
- ConvertRowBatchTime: 0ns
- DataArrivalWaitTime: 0ns
- DecompressRowBatchTimer: 331.915ms
- DeserializeChunkMetaTimer: 0ns
- DeserializeRowBatchTimer: 73.889ms
- FirstBatchArrivalWaitTime: 0ns
- PeakMemoryUsage: 0.00
- RequestReceived: 732.00 B
- RowsReturned: 2.994862M (2994862)
- RowsReturnedRate: 56.213K /sec
- SenderTotalTime: 411.320ms
- SenderWaitLockTime: 363.18us
- SendersBlockedTotalTimer(): 0ns
Fragment 1:
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7aea (host=TNetworkAddress(hostname:, port:9060)):(Active: 54s325ms[54325433243ns], % non-child: 0.00%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 246.53 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 998.488K (998488)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=4, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7aeb]):(Active: 109.962ms[109962178ns], % non-child: 0.20%)
- PartType: UNPARTITIONED
- BytesSent: 31.06 MB
- CompressTime: 82.315ms
- IgnoreRows: 0
- OverallThroughput: 282.4879140853882 MB/sec
- PeakMemoryUsage: 1.14 KB
- SendRequestTime: 10.369ms
- SerializeBatchTime: 11.412ms
- ShuffleDispatchTime: 0ns
- ShuffleHashTime: 0ns
- UncompressedBytes: 61.56 MB
- WaitResponseTime: 2.39ms
AGGREGATION_NODE (id=3):(Active: 54s213ms[54213455102ns], % non-child: 16.33%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count( bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid))
- AggComputeTime: 5s5ms
- ExprComputeTime: 4.358ms
- ExprReleaseTime: 2s12ms
- GetResultsTime: 1s847ms
- HashTableSize: 998.488K (998488)
- InputRowCount: 2.525463M (2525463)
- PassThroughRowCount: 0
- PeakMemoryUsage: 246.50 MB
- ResultAggAppendTime: 1s684ms
- ResultGroupByAppendTime: 145.74ms
- ResultIteratorTime: 16.439ms
- RowsReturned: 998.488K (998488)
- RowsReturnedRate: 18.417K /sec
- StreamingTime: 0ns
EXCHANGE_NODE (id=2):(Active: 45s340ms[45340746069ns], % non-child: 83.44%)
- BytesReceived: 329.46 MB
- ConvertRowBatchTime: 0ns
- DataArrivalWaitTime: 0ns
- DecompressRowBatchTimer: 564.717ms
- DeserializeChunkMetaTimer: 0ns
- DeserializeRowBatchTimer: 12s564ms
- FirstBatchArrivalWaitTime: 0ns
- PeakMemoryUsage: 0.00
- RequestReceived: 750.00 B
- RowsReturned: 2.525463M (2525463)
- RowsReturnedRate: 55.699K /sec
- SenderTotalTime: 13s135ms
- SenderWaitLockTime: 359.485us
- SendersBlockedTotalTimer(): 0ns
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae8 (host=TNetworkAddress(hostname:, port:9060)):(Active: 54s134ms[54134918624ns], % non-child: 0.00%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 246.53 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 998.871K (998871)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=4, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7aeb]):(Active: 103.820ms[103820690ns], % non-child: 0.19%)
- PartType: UNPARTITIONED
- BytesSent: 31.05 MB
- CompressTime: 78.828ms
- IgnoreRows: 0
- OverallThroughput: 299.02871227264404 MB/sec
- PeakMemoryUsage: 1.14 KB
- SendRequestTime: 13.530ms
- SerializeBatchTime: 7.64ms
- ShuffleDispatchTime: 0ns
- ShuffleHashTime: 0ns
- UncompressedBytes: 61.57 MB
- WaitResponseTime: 1.462ms
AGGREGATION_NODE (id=3):(Active: 54s29ms[54029124733ns], % non-child: 17.31%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count( bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid))
- AggComputeTime: 5s403ms
- ExprComputeTime: 3.846ms
- ExprReleaseTime: 2s304ms
- GetResultsTime: 1s692ms
- HashTableSize: 998.871K (998871)
- InputRowCount: 2.526008M (2526008)
- PassThroughRowCount: 0
- PeakMemoryUsage: 246.50 MB
- ResultAggAppendTime: 1s536ms
- ResultGroupByAppendTime: 142.753ms
- ResultIteratorTime: 12.374ms
- RowsReturned: 998.871K (998871)
- RowsReturnedRate: 18.487K /sec
- StreamingTime: 0ns
EXCHANGE_NODE (id=2):(Active: 44s623ms[44623971806ns], % non-child: 82.12%)
- BytesReceived: 327.39 MB
- ConvertRowBatchTime: 0ns
- DataArrivalWaitTime: 0ns
- DecompressRowBatchTimer: 579.818ms
- DeserializeChunkMetaTimer: 0ns
- DeserializeRowBatchTimer: 15s596ms
- FirstBatchArrivalWaitTime: 0ns
- PeakMemoryUsage: 0.00
- RequestReceived: 758.00 B
- RowsReturned: 2.526008M (2526008)
- RowsReturnedRate: 56.606K /sec
- SenderTotalTime: 16s182ms
- SenderWaitLockTime: 340.976us
- SendersBlockedTotalTimer(): 0ns
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae9 (host=TNetworkAddress(hostname:, port:9060)):(Active: 54s66ms[54066285477ns], % non-child: 0.00%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 246.53 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 997.503K (997503)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=4, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7aeb]):(Active: 98.741ms[98741202ns], % non-child: 0.18%)
- PartType: UNPARTITIONED
- BytesSent: 30.99 MB
- CompressTime: 80.245ms
- IgnoreRows: 0
- OverallThroughput: 313.8319101333618 MB/sec
- PeakMemoryUsage: 1.14 KB
- SendRequestTime: 7.373ms
- SerializeBatchTime: 7.142ms
- ShuffleDispatchTime: 0ns
- ShuffleHashTime: 0ns
- UncompressedBytes: 61.49 MB
- WaitResponseTime: 795.669us
AGGREGATION_NODE (id=3):(Active: 53s965ms[53965912080ns], % non-child: 17.21%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count( bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid))
- AggComputeTime: 5s528ms
- ExprComputeTime: 3.809ms
- ExprReleaseTime: 2s193ms
- GetResultsTime: 1s625ms
- HashTableSize: 997.503K (997503)
- InputRowCount: 2.52158M (2521580)
- PassThroughRowCount: 0
- PeakMemoryUsage: 246.50 MB
- ResultAggAppendTime: 1s443ms
- ResultGroupByAppendTime: 164.699ms
- ResultIteratorTime: 15.990ms
- RowsReturned: 997.503K (997503)
- RowsReturnedRate: 18.483K /sec
- StreamingTime: 0ns
EXCHANGE_NODE (id=2):(Active: 44s613ms[44613529193ns], % non-child: 82.10%)
- BytesReceived: 335.60 MB
- ConvertRowBatchTime: 0ns
- DataArrivalWaitTime: 0ns
- DecompressRowBatchTimer: 579.245ms
- DeserializeChunkMetaTimer: 0ns
- DeserializeRowBatchTimer: 13s103ms
- FirstBatchArrivalWaitTime: 0ns
- PeakMemoryUsage: 0.00
- RequestReceived: 747.00 B
- RowsReturned: 2.52158M (2521580)
- RowsReturnedRate: 56.52K /sec
- SenderTotalTime: 13s689ms
- SenderWaitLockTime: 357.31us
- SendersBlockedTotalTimer(): 0ns
Fragment 2:
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae5 (host=TNetworkAddress(hostname:, port:9060)):(Active: 52s343ms[52343920938ns], % non-child: 4.06%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 692.09 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 2.533894M (2533894)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=2, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7ae8, 6b3c8cac3de811ec-ae6cbc97e14b7ae9, 6b3c8cac3de811ec-ae6cbc97e14b7aea]):(Active: 9s208ms[9208234944ns], % non-child: 16.95%)
- PartType: HASH_PARTITIONED
- BytesSent: 331.16 MB
- CompressTime: 662.401ms
- IgnoreRows: 0
- OverallThroughput: 35.96354103088379 MB/sec
- PeakMemoryUsage: 3.00 KB
- SendRequestTime: 112.519ms
- SerializeBatchTime: 1s974ms
- ShuffleDispatchTime: 9s161ms
- ShuffleHashTime: 102.736ms
- UncompressedBytes: 470.83 MB
- WaitResponseTime: 30.973ms
AGGREGATION_NODE (id=1):(Active: 40s928ms[40928618796ns], % non-child: 69.61%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid)
- AggComputeTime: 23s865ms
- ExprComputeTime: 11.283ms
- ExprReleaseTime: 5s317ms
- GetResultsTime: 8s581ms
- HashTableSize: 2.487577M (2487577)
- InputRowCount: 13.797397M (13797397)
- PassThroughRowCount: 46.317K (46317)
- PeakMemoryUsage: 464.50 MB
- ResultAggAppendTime: 7s608ms
- ResultGroupByAppendTime: 931.227ms
- ResultIteratorTime: 35.872ms
- RowsReturned: 2.533894M (2533894)
- RowsReturnedRate: 61.91K /sec
- StreamingTime: 18.11ms
OLAP_SCAN_NODE (id=0):(Active: 3s99ms[3099954005ns], % non-child: 0.00%)
- Table: mds_log_weblog_click_bak_bitmap
- Rollup: bitmap_uv
- BytesRead: 1.80 GB
- NumDiskAccess: 0
- PeakMemoryUsage: 247.18 MB
- PerReadThreadRawHdfsThroughput: 0.0 /sec
- RowsRead: 0
- RowsReturned: 13.797397M (13797397)
- RowsReturnedRate: 4.450839M /sec
- ScanTime: 41s730ms
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 0ns
- MaterializeTupleTime(): 0ns
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 0
- TabletCount : 232
- TotalRawReadTime(): 0ns
- TotalReadThroughput: 35.48159980773926 MB/sec
MERGE:
- aggr: 29s995ms
- union: 28s708ms
SCAN:(Active: 40s353ms[40353425000ns], % non-child: 74.26%)
- CachedPagesNum: 0
- CompressedBytesRead: 980.68 MB
- IOTime: 318.891ms
- PushdownPredicates: 0
- RawRowsRead: 13.797397M (13797397)
- ReaderInit: 168.839ms
- CaptureRowset: 42.777ms
- SegmentInit: 220.970ms
- BitmapIndexFilter: 0ns
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- SegmentRead: 39s168ms
- BlockFetch: 38s777ms
- BlockFetchCount: 3.627K (3627)
- BlockSeek: 484.979ms
- BlockSeekCount: 490
- ChunkCopy: 0ns
- DecompressT: 1s268ms
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 0ns
- PredFilterRows: 0
- TotalPagesNum: 27.89K (27890)
- UncompressedBytesRead: 1.15 GB
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae7 (host=TNetworkAddress(hostname:, port:9060)):(Active: 49s593ms[49593695615ns], % non-child: 3.81%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 668.69 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 2.526568M (2526568)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=2, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7ae8, 6b3c8cac3de811ec-ae6cbc97e14b7ae9, 6b3c8cac3de811ec-ae6cbc97e14b7aea]):(Active: 7s823ms[7823245891ns], % non-child: 14.40%)
- PartType: HASH_PARTITIONED
- BytesSent: 330.91 MB
- CompressTime: 684.580ms
- IgnoreRows: 0
- OverallThroughput: 42.29789638519287 MB/sec
- PeakMemoryUsage: 3.00 KB
- SendRequestTime: 103.815ms
- SerializeBatchTime: 1s887ms
- ShuffleDispatchTime: 7s784ms
- ShuffleHashTime: 106.627ms
- UncompressedBytes: 470.04 MB
- WaitResponseTime: 26.890ms
AGGREGATION_NODE (id=1):(Active: 39s701ms[39701989479ns], % non-child: 67.59%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid)
- AggComputeTime: 23s483ms
- ExprComputeTime: 12.547ms
- ExprReleaseTime: 5s120ms
- GetResultsTime: 8s49ms
- HashTableSize: 2.489085M (2489085)
- InputRowCount: 13.764943M (13764943)
- PassThroughRowCount: 37.483K (37483)
- PeakMemoryUsage: 465.00 MB
- ResultAggAppendTime: 7s134ms
- ResultGroupByAppendTime: 878.176ms
- ResultIteratorTime: 29.489ms
- RowsReturned: 2.526568M (2526568)
- RowsReturnedRate: 63.638K /sec
- StreamingTime: 31.305ms
OLAP_SCAN_NODE (id=0):(Active: 2s971ms[2971348246ns], % non-child: 0.00%)
- Table: mds_log_weblog_click_bak_bitmap
- Rollup: bitmap_uv
- BytesRead: 1.80 GB
- NumDiskAccess: 0
- PeakMemoryUsage: 224.43 MB
- PerReadThreadRawHdfsThroughput: 0.0 /sec
- RowsRead: 0
- RowsReturned: 13.764943M (13764943)
- RowsReturnedRate: 4.632557M /sec
- ScanTime: 39s874ms
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 0ns
- MaterializeTupleTime(): 0ns
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 0
- TabletCount : 232
- TotalRawReadTime(): 0ns
- TotalReadThroughput: 37.17123508453369 MB/sec
MERGE:
- aggr: 30s680ms
- union: 29s332ms
SCAN:(Active: 38s468ms[38468737704ns], % non-child: 70.79%)
- CachedPagesNum: 0
- CompressedBytesRead: 978.53 MB
- IOTime: 329.34ms
- PushdownPredicates: 0
- RawRowsRead: 13.764943M (13764943)
- ReaderInit: 158.102ms
- CaptureRowset: 36.175ms
- SegmentInit: 225.483ms
- BitmapIndexFilter: 0ns
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- SegmentRead: 37s281ms
- BlockFetch: 36s880ms
- BlockFetchCount: 3.617K (3617)
- BlockSeek: 530.785ms
- BlockSeekCount: 497
- ChunkCopy: 0ns
- DecompressT: 1s385ms
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 0ns
- PredFilterRows: 0
- TotalPagesNum: 27.869K (27869)
- UncompressedBytesRead: 1.15 GB
Instance 6b3c8cac-3de8-11ec-ae6c-bc97e14b7ae6 (host=TNetworkAddress(hostname:, port:9060)):(Active: 48s250ms[48250868144ns], % non-child: 3.80%)
- AverageThreadTokens: 4607182418800017400.00
- MemoryLimit: 4.00 GB
- PeakMemoryUsage: 682.85 MB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 2.512589M (2512589)
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=2, dst_fragments=[6b3c8cac3de811ec-ae6cbc97e14b7ae8, 6b3c8cac3de811ec-ae6cbc97e14b7ae9, 6b3c8cac3de811ec-ae6cbc97e14b7aea]):(Active: 8s42ms[8042553142ns], % non-child: 14.80%)
- PartType: HASH_PARTITIONED
- BytesSent: 330.39 MB
- CompressTime: 701.575ms
- IgnoreRows: 0
- OverallThroughput: 41.079702377319336 MB/sec
- PeakMemoryUsage: 3.00 KB
- SendRequestTime: 140.120ms
- SerializeBatchTime: 2s147ms
- ShuffleDispatchTime: 8s0ms
- ShuffleHashTime: 108.150ms
- UncompressedBytes: 468.99 MB
- WaitResponseTime: 23.515ms
AGGREGATION_NODE (id=1):(Active: 38s145ms[38145462494ns], % non-child: 65.01%)
- GroupingKeys: dt, hours, channel, sub_channel, country_id, site, lang
- AggregateFunctions: bitmap_union_count(default_cluster:hive_test.mds_log_weblog_click_bak_bitmap.mv_bitmap_union_bit_vid)
- AggComputeTime: 23s114ms
- ExprComputeTime: 12.803ms
- ExprReleaseTime: 5s105ms
- GetResultsTime: 7s55ms
- HashTableSize: 2.488611M (2488611)
- InputRowCount: 14.401528M (14401528)
- PassThroughRowCount: 23.978K (23978)
- PeakMemoryUsage: 465.00 MB
- ResultAggAppendTime: 6s257ms
- ResultGroupByAppendTime: 764.197ms
- ResultIteratorTime: 28.349ms
- RowsReturned: 2.512589M (2512589)
- RowsReturnedRate: 65.868K /sec
- StreamingTime: 9.882ms
OLAP_SCAN_NODE (id=0):(Active: 2s815ms[2815798203ns], % non-child: 0.00%)
- Table: mds_log_weblog_click_bak_bitmap
- Rollup: bitmap_uv
- BytesRead: 1.87 GB
- NumDiskAccess: 0
- PeakMemoryUsage: 237.32 MB
- PerReadThreadRawHdfsThroughput: 0.0 /sec
- RowsRead: 0
- RowsReturned: 14.401528M (14401528)
- RowsReturnedRate: 5.114545M /sec
- ScanTime: 37s710ms
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 0ns
- MaterializeTupleTime(): 0ns
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 0
- TabletCount : 232
- TotalRawReadTime(): 0ns
- TotalReadThroughput: 39.79592227935791 MB/sec
MERGE:
- aggr: 33s252ms
- union: 31s678ms
SCAN:(Active: 36s78ms[36078672100ns], % non-child: 66.39%)
- CachedPagesNum: 0
- CompressedBytesRead: 1008.69 MB
- IOTime: 472.280ms
- PushdownPredicates: 0
- RawRowsRead: 14.401528M (14401528)
- ReaderInit: 144.363ms
- CaptureRowset: 22.481ms
- SegmentInit: 203.167ms
- BitmapIndexFilter: 0ns
- BitmapIndexFilterRows: 0
- BloomFilterFilterRows: 0
- ShortKeyFilterRows: 0
- ZoneMapIndexFilterRows: 0
- SegmentRead: 34s847ms
- BlockFetch: 34s260ms
- BlockFetchCount: 3.791K (3791)
- BlockSeek: 691.559ms
- BlockSeekCount: 503
- ChunkCopy: 0ns
- DecompressT: 1s380ms
- DelVecFilterRows: 0
- IndexLoad: 0ns
- PredFilter: 0ns
- PredFilterRows: 0
- TotalPagesNum: 28.805K (28805)
- UncompressedBytesRead: 1.18 GB
【StarRocks版本】1.18.2
【集群规模】3be（fe与be混部）
【机器信息】32c,128G