解析json后,使用json的key来分组开窗,会导致be崩溃

【详述】
导致be崩溃的sql:
SELECT
*
from (
SELECT
a
, b
, ROW_NUMBER() over(partition by pro.key order by b) as num
from (
SELECT ‘{“aa”:“sss”,“dd”:123}’ as a, 1 as b
union all
SELECT ‘{“aa”:“sss”,“dd”:123}’ as a, 2 as b
) t, LATERAL JSON_EACH(a) as pro
) t
where num = 1
;
只要执行where num=1的时候就会导致be宕机。
如果在外层包一层,则不会报错,不会报错的sql如下:
SELECT
*
from (
select
*
, ROW_NUMBER() over(partition by c order by b) as num
from (
SELECT
a
, b
, pro.key as c
from (
SELECT ‘{“aa”:“sss”,“dd”:123}’ as a, 1 as b
union all
SELECT ‘{“aa”:“sss”,“dd”:123}’ as a, 2 as b
) t, LATERAL JSON_EACH(a) as pro
) t
) t
where num = 1
;
这时候执行where num=1就不会导致be宕机

【背景】无
【业务影响】
【StarRocks版本】2.5.3
【集群规模】3fe+5be

您好,报错的sql导致be crash能看下挂掉be的be.out文件吗?

be.out 文件的报错信息

query_id:502547ea-d1cb-11ed-aa09-1866dafb1278, fragment_instance:502547ea-d1cb-11ed-aa09-1866dafb127a
tracker:process consumption: 1269937160
tracker:query_pool consumption: 33200
tracker:load consumption: 0
tracker:metadata consumption: 218344958
tracker:tablet_metadata consumption: 141187743
tracker:rowset_metadata consumption: 76640944
tracker:segment_metadata consumption: 115617
tracker:column_metadata consumption: 400654
tracker:tablet_schema consumption: 559775
tracker:segment_zonemap consumption: 37980
tracker:short_key_index consumption: 10978
tracker:column_zonemap_index consumption: 68990
tracker:ordinal_index consumption: 160304
tracker:bitmap_index consumption: 0
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 0
tracker:schema_change consumption: 0
tracker:column_pool consumption: 367773170
tracker:page_cache consumption: 46997056
tracker:update consumption: 0
tracker:chunk_allocator consumption: 36516568
tracker:clone consumption: 0
tracker:consistency consumption: 0
*** Aborted at 1680490667 (unix time) try “date -d @1680490667” if you are using GNU date ***
PC: @ 0x2f3c2c1 ZN9starrocks10vectorized20PartitionHashMapBase33append_chunk_for_one_nullable_keyIN5phmap13flat_hash_mapINS_5SliceEPNS0_15PartitionChunksENS0_17SliceHashWithSeedILNS0_9PhmapSeedE0EEENS0_10SliceEqualESaISt4pairIKS5_S7_EEEEZNS0_40PartitionHashMapWithOneNullableStringKeyISG_E12append_chunkIZNS_8pipeline25LocalPartitionTopnContext29push_one_chunk_to_partitionerEPNS_12RuntimeStateERKSt10shared_ptrINS0_5ChunkEEEUlmE_ZNSL_29push_one_chunk_to_partitionerESN_SS_EUlmSS_E0_EEbSQ_RKSt6vectorISO_INS0_6ColumnEESaISX_EEPNS_7MemPoolEPNS_10ObjectPoolEOT_OT0_EUljE_ZNSJ_IST_SU_EEbSQ_S11_S13_S15_S17_S19_EUlRSD_E0_ST_SU_EEvRS16_RS6_SQ_PKNS0_14NullableColumnES19_OT1_S15_OT2_OT3
*** SIGSEGV (@0x656d6954) received by PID 26525 (TID 0x7f12c554c700) from PID 1701669204; stack trace: ***
@ 0x5769222 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7f1361a51852 os::Linux::chained_handler()
@ 0x7f1361a58676 JVM_handle_linux_signal
@ 0x7f1361a4e653 signalHandler()
@ 0x7f1360f17630 (unknown)
@ 0x2f3c2c1 ZN9starrocks10vectorized20PartitionHashMapBase33append_chunk_for_one_nullable_keyIN5phmap13flat_hash_mapINS_5SliceEPNS0_15PartitionChunksENS0_17SliceHashWithSeedILNS0_9PhmapSeedE0EEENS0_10SliceEqualESaISt4pairIKS5_S7_EEEEZNS0_40PartitionHashMapWithOneNullableStringKeyISG_E12append_chunkIZNS_8pipeline25LocalPartitionTopnContext29push_one_chunk_to_partitionerEPNS_12RuntimeStateERKSt10shared_ptrINS0_5ChunkEEEUlmE_ZNSL_29push_one_chunk_to_partitionerESN_SS_EUlmSS_E0_EEbSQ_RKSt6vectorISO_INS0_6ColumnEESaISX_EEPNS_7MemPoolEPNS_10ObjectPoolEOT_OT0_EUljE_ZNSJ_IST_SU_EEbSQ_S11_S13_S15_S17_S19_EUlRSD_E0_ST_SU_EEvRS16_RS6_SQ_PKNS0_14NullableColumnES19_OT1_S15_OT2_OT3
@ 0x2f44bc4 ZN9starrocks10vectorized17ChunksPartitioner5offerIZNS_8pipeline25LocalPartitionTopnContext29push_one_chunk_to_partitionerEPNS_12RuntimeStateERKSt10shared_ptrINS0_5ChunkEEEUlmE_ZNS4_29push_one_chunk_to_partitionerES6_SB_EUlmSB_E0_EENS_6StatusESB_OT_OT0
@ 0x2f50330 starrocks::pipeline::LocalPartitionTopnContext::push_one_chunk_to_partitioner()
@ 0x2f27b59 starrocks::pipeline::LocalPartitionTopnSinkOperator::push_chunk()
@ 0x2c57e06 starrocks::pipeline::PipelineDriver::process()
@ 0x4e075e7 starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x4812f8d starrocks::ThreadPool::dispatch_thread()
@ 0x480dd1a starrocks::thread::supervise_thread()
@ 0x7f1360f0fea5 start_thread
@ 0x7f136052ab0d __clone
@ 0x0 (unknown)

大佬,这个问题有定位吗?

请稍等,正在处理,之后我会把修复 pr 贴到这里。

PR: https://github.com/StarRocks/starrocks/pull/21277

大佬,这个在2.5的呢个小版本修复呢?