为了更快的定位您的问题,请提供以下信息,谢谢
【详述】查询具有复杂结构的hive外表,BE直接crash
hive建表语句
CREATE EXTERNAL TABLE test.t_multi_tag (
deviceid STRING COMMENT '设备D',
id BIGINT COMMENT '设备映射后的唯一ID',
tag_values_single MAP<STRING,STRING> COMMENT '单值型标签',
tag_values_multi MAP<STRING,ARRAY<STRUCT<t:STRING,v:STRING,w:INT>>> COMMENT '多值性标签(string)',
tag_values_bitmap MAP<STRING,ARRAY<STRUCT<t:STRING,v:BINARY,w:INT>>> COMMENT '多值性标签(bitmap)'
)
SR查询语句
select deviceid, tag_values_single['lastlogin_time'],tag_values_bitmap['pay_roleid_90d'] from hive_catalog.test.t_multi_tag
where deviceid='00baa04b3e19729f825bd6121cad98def167ea6d72c8364e0570dd2c8f4f6d53'
and tag_values_single['lastlogin_time']>'2024-11-18 00:00:00' limit 10 ;
【背景】
【业务影响】
【是否存算分离】
【StarRocks版本】3.3.3
【集群规模】3fe(1 follower+2observer)+5be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【联系方式】
【附件】
- fe.log/beINFO/相应截图
- 慢查询:
- Profile信息
- 并行度:show variables like ‘%parallel_fragment_exec_instance_num%’;
- pipeline是否开启:show variables like ‘%pipeline%’;
- be节点cpu和内存使用率截图
- 查询报错:
- query_dump,怎么获取query_dump文件
- be crash
- be.out
F20241122 18:10:52.023367 140141792061184 logical_type_infra.h:170] Check failed: false Unknown type: 0
3.3.3 RELEASE (build 312ed45)
query_id:0bad881f-a8ba-11ef-a8de-52540027bef0, fragment_instance:0bad881f-a8ba-11ef-a8de-52540027bef1
tracker:process consumption: 38226973280
tracker:query_pool consumption: 354848
tracker:query_pool/connector_scan consumption: 0
tracker:load consumption: 0
tracker:metadata consumption: 1283030118
tracker:tablet_metadata consumption: 167727939
tracker:rowset_metadata consumption: 358516513
tracker:segment_metadata consumption: 111048539
tracker:column_metadata consumption: 645737127
tracker:tablet_schema consumption: 1559715
tracker:segment_zonemap consumption: 42375538
tracker:short_key_index consumption: 41278157
tracker:column_zonemap_index consumption: 139736623
tracker:ordinal_index consumption: 223054840
tracker:bitmap_index consumption: 2901792
tracker:bloom_filter_index consumption: 1129056
tracker:compaction consumption: 0
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 16635155152
tracker:jit_cache consumption: 53960
tracker:update consumption: 2628382071
tracker:chunk_allocator consumption: 1002351456
tracker:clone consumption: 0
tracker:consistency consumption: 0
tracker:datacache consumption: 0
tracker:replication consumption: 0
*** Aborted at 1732270252 (unix time) try "date -d @1732270252" if you are using GNU date ***
PC: @ 0x7f779691a387 (/usr/lib64/libc-2.17.so (deleted)+0x36386)
*** SIGABRT (@0x3ee00059fe1) received by PID 368609 (TID 0x7f754dbbe700) from PID 368609; stack trace: ***
@ 0x7f77975e020b (/usr/lib64/libpthread-2.17.so (deleted)+0x620a)
@ 0x79c3320 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
@ 0x7f77975e9630 (/usr/lib64/libpthread-2.17.so (deleted)+0xf62f)
@ 0x7f779691a387 (/usr/lib64/libc-2.17.so (deleted)+0x36386)
@ 0x7f779691ba78 (/usr/lib64/libc-2.17.so (deleted)+0x37a77)
@ 0x3523833 starrocks::failure_function()
@ 0x79b7c1a google::LogMessage::Fail()
@ 0x79b8fe5 google::LogMessageFatal::~LogMessageFatal()
@ 0x3e6e735 auto starrocks::type_dispatch_column<starrocks::ColumnBuilder, starrocks::TypeDescriptor, unsigned long>(starrocks::LogicalType, starrocks::ColumnBuilder, starrocks::TypeDescriptor, unsigned long)
@ 0x3e65a05 starrocks::ColumnHelper::create_column(starrocks::TypeDescriptor const&, bool, bool, unsigned long, bool)
@ 0x5e22d64 starrocks::serde::ProtobufChunkDeserializer::deserialize(std::basic_string_view<char, std::char_traits<char> >, long*)
@ 0x3752739 starrocks::DataStreamRecvr::SenderQueue::_deserialize_chunk(starrocks::ChunkPB const&, starrocks::Chunk*, starrocks::DataStreamRecvr::Metrics&, starrocks::faststring*)
@ 0x3756d2d starrocks::DataStreamRecvr::PipelineSenderQueue::get_chunk(starrocks::Chunk**, int)
@ 0x37279d0 starrocks::DataStreamRecvr::get_chunk_for_pipeline(std::unique_ptr<starrocks::Chunk, std::default_delete<starrocks::Chunk> >*, int)
@ 0x4214e28 starrocks::pipeline::ExchangeSourceOperator::pull_chunk(starrocks::RuntimeState*)
@ 0x42de751 starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int)
@ 0x42ceb96 starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x382b71b starrocks::ThreadPool::dispatch_thread()
@ 0x3824c46 starrocks::Thread::supervise_thread(void*)
@ 0x7f77975e1ea5 (/usr/lib64/libpthread-2.17.so (deleted)+0x7ea4)
F20241122 18:10:55.972492 140138354820864 tablet_updates.cpp:875] submit apply task failed: Runtime error: Could not create thread: Resource temporarily unavailable tablet:10066 #version:121 [132204 132308@119 132308.1] pending: rowsets:4
151231 [seg:1 row:5414 del:1 bytes:17473363 row_size:0 compaction_score:250978228 compaction_level:-1 partial_update_by_column:false]
153226 [seg:1 row:344 del:0 bytes:274721 row_size:0 compaction_score:268160735 compaction_level:-1 partial_update_by_column:false]
153229 [seg:1 row:872 del:0 bytes:1342154 row_size:0 compaction_score:267093302 compaction_level:-1 partial_update_by_column:false]
153242 [seg:1 row:3097 del:0 bytes:7873998 row_size:0 compaction_score:260561458 compaction_level:-1 partial_update_by_column:false]
@ 0x7f77969e2b0d (/usr/lib64/libc-2.17.so (deleted)+0xfeb0c)
- 外表查询报错
- be.out和fe.warn.log