为了更快的定位您的问题，请提供以下信息，谢谢
【详述】be节点挂掉
【背景】
【业务影响】
【StarRocks版本】例如：3.0.2
【集群规模】例如：3fe（3 follower）+12be（fe与be混部）
【机器信息】8c32g
【联系方式】18840044216
【附件】
3.0.2 RELEASE (build c833698)
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000
tracker:process consumption: 3750107696
tracker:query_pool consumption: 0
tracker:load consumption: 0
tracker:metadata consumption: 309353316
tracker:tablet_metadata consumption: 0
tracker:rowset_metadata consumption: 0
tracker:segment_metadata consumption: 0
tracker:column_metadata consumption: 0
tracker:tablet_schema consumption: 0
tracker:segment_zonemap consumption: 0
tracker:short_key_index consumption: 0
tracker:column_zonemap_index consumption: 0
tracker:ordinal_index consumption: 0
tracker:bitmap_index consumption: 0
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 0
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 538909005
tracker:chunk_allocator consumption: 0
tracker:clone consumption: 0
tracker:consistency consumption: 0
start time: Thu Aug 31 07:20:01 CST 2023
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object
3.0.2 RELEASE (build c833698)
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000
tracker:process consumption: 2018356960
tracker:query_pool consumption: 0
tracker:load consumption: 0
tracker:metadata consumption: 203666819
tracker:tablet_metadata consumption: 0
tracker:rowset_metadata consumption: 0
tracker:segment_metadata consumption: 0
tracker:column_metadata consumption: 0
tracker:tablet_schema consumption: 0
tracker:segment_zonemap consumption: 0
tracker:short_key_index consumption: 0
tracker:column_zonemap_index consumption: 0
tracker:ordinal_index consumption: 0
tracker:bitmap_index consumption: 0
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 0
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 658923284
tracker:chunk_allocator consumption: 0
tracker:clone consumption: 0
tracker:consistency consumption: 0
start time: Thu Aug 31 10:15:17 CST 2023
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object
3.0.2 RELEASE (build c833698)
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000
tracker:process consumption: 3523878704
tracker:query_pool consumption: 0
tracker:load consumption: 0
tracker:metadata consumption: 232824807
tracker:tablet_metadata consumption: 0
tracker:rowset_metadata consumption: 0
tracker:segment_metadata consumption: 0
tracker:column_metadata consumption: 0
tracker:tablet_schema consumption: 0
tracker:segment_zonemap consumption: 0
tracker:short_key_index consumption: 0
tracker:column_zonemap_index consumption: 0
tracker:ordinal_index consumption: 0
tracker:bitmap_index consumption: 0
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 0
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 608839325
tracker:chunk_allocator consumption: 0
tracker:clone consumption: 0
tracker:consistency consumption: 0
start time: Thu Aug 31 11:34:45 CST 2023
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/jni-packages/starrocks-jdbc-bridge-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Java HotSpot™ 64-Bit Server VM warning: You have loaded library /opt/module/starrocks/be/lib/hadoop/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It’s highly recommended that you fix the library with 'execstack -c ', or link it with ‘-z noexecstack’.
start time: Thu Aug 31 11:50:27 CST 2023
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/jni-packages/starrocks-jdbc-bridge-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Java HotSpot™ 64-Bit Server VM warning: You have loaded library /opt/module/starrocks/be/lib/hadoop/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It’s highly recommended that you fix the library with 'execstack -c ', or link it with ‘-z noexecstack’.
3.0.2 RELEASE (build c833698)
query_id:1ae489c3-5c53-11ee-b1a9-00163e0c0e88, fragment_instance:1ae489c3-5c53-11ee-b1a9-00163e0c0e97
tracker:process consumption: 6456963416
tracker:query_pool consumption: 320672240
tracker:load consumption: 0
tracker:metadata consumption: 1028960704
tracker:tablet_metadata consumption: 558158714
tracker:rowset_metadata consumption: 213183283
tracker:segment_metadata consumption: 30157798
tracker:column_metadata consumption: 227460909
tracker:tablet_schema consumption: 47284746
tracker:segment_zonemap consumption: 17820577
tracker:short_key_index consumption: 7458131
tracker:column_zonemap_index consumption: 26897317
tracker:ordinal_index consumption: 99538712
tracker:bitmap_index consumption: 8623392
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 71865616
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 1037905432
tracker:chunk_allocator consumption: 2142060160
tracker:clone consumption: 0
tracker:consistency consumption: 0
*** Aborted at 1695722251 (unix time) try “date -d @1695722251” if you are using GNU date ***
PC: @ 0x7fb9a644cd10 __memmove_ssse3_back
*** SIGSEGV (@0x0) received by PID 21160 (TID 0x7fb936b5f700) from PID 0; stack trace: ***
@ 0x62d7062 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fb9a78fbf2b os::Linux::chained_handler()
@ 0x7fb9a79009bc JVM_handle_linux_signal
@ 0x7fb9a78f38e8 signalHandler()
@ 0x7fb9a6ddb630 (unknown)
@ 0x7fb9a644cd10 __memmove_ssse3_back
@ 0x30b7474 starrocks::FixedLengthColumnBase<>::append()
@ 0x569a7d2 starrocks::NullableColumn::append()
@ 0x3a98d6d starrocks::ArrayAggAggregateFunctionV2::finalize_to_column()
@ 0x3a7951f starrocks::AggregateFunctionBatchHelper<>::batch_finalize()
@ 0x358a7ed ZZN9starrocks10Aggregator25convert_hash_map_to_chunkEiPSt10shared_ptrINS_5ChunkEEPbENKUlRT_E_clIKSt10unique_ptrINS_38AggHashMapWithOneStringKeyWithNullableIN5phmap13flat_hash_mapINS_5SliceEPhNS_17SliceHashWithSeedILNS_9PhmapSeedE1EEENS_10SliceEqualESaISt4pairIKSE_SF_EEEELb1EEESt14default_deleteISP_EEEEDaS7
@ 0x358aba2 (unknown)
@ 0x3532ece starrocks::Aggregator::convert_hash_map_to_chunk()
@ 0x34463e2 starrocks::pipeline::AggregateBlockingSourceOperator::pull_chunk()
@ 0x310f4f2 starrocks::pipeline::PipelineDriver::process()
@ 0x578d84b starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x50e5222 starrocks::ThreadPool::dispatch_thread()
@ 0x50dfd1a starrocks:
:supervise_thread()
@ 0x7fb9a6dd3ea5 start_thread
@ 0x7fb9a63eeb0d __clone
@ 0x0 (unknown)
start time: Tue Sep 26 17:59:12 CST 2023
修了，升级3.0最新小版本
新节点又有挂的，看着好像不太一样，请问也是这个原因吗？
start time: Thu Aug 31 11:50:41 CST 2023
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/jni-packages/starrocks-jdbc-bridge-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/starrocks/be/lib/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Java HotSpot™ 64-Bit Server VM warning: You have loaded library /opt/module/starrocks/be/lib/hadoop/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It’s highly recommended that you fix the library with 'execstack -c ', or link it with ‘-z noexecstack’.
3.0.2 RELEASE (build c833698)
query_id:f387f5fe-5d04-11ee-ac06-00163e09a0ba, fragment_instance:f387f5fe-5d04-11ee-ac06-00163e09a0cc
tracker:process consumption: 7379739296
tracker:query_pool consumption: 286054056
tracker:load consumption: 0
tracker:metadata consumption: 1266185623
tracker:tablet_metadata consumption: 535116716
tracker:rowset_metadata consumption: 217885999
tracker:segment_metadata consumption: 91316130
tracker:column_metadata consumption: 421866778
tracker:tablet_schema consumption: 22019596
tracker:segment_zonemap consumption: 63431914
tracker:short_key_index consumption: 20104569
tracker:column_zonemap_index consumption: 57530306
tracker:ordinal_index consumption: 205717240
tracker:bitmap_index consumption: 13061296
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 125579672
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 1659310552
tracker:chunk_allocator consumption: 2147681792
tracker:clone consumption: 0
tracker:consistency consumption: 0
*** Aborted at 1695798634 (unix time) try “date -d @1695798634” if you are using GNU date ***
PC: @ 0x7fbaa1a73d10 __memmove_ssse3_back
*** SIGSEGV (@0x0) received by PID 16767 (TID 0x7fba1d962700) from PID 0; stack trace: ***
@ 0x62d7062 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fbaa2f22f2b os::Linux::chained_handler()
@ 0x7fbaa2f279bc JVM_handle_linux_signal
@ 0x7fbaa2f1a8e8 signalHandler()
@ 0x7fbaa2402630 (unknown)
@ 0x7fbaa1a73d10 __memmove_ssse3_back
@ 0x30b7474 starrocks::FixedLengthColumnBase<>::append()
@ 0x569a7d2 starrocks::NullableColumn::append()
@ 0x3a98d6d starrocks::ArrayAggAggregateFunctionV2::finalize_to_column()
@ 0x3a7951f starrocks::AggregateFunctionBatchHelper<>::batch_finalize()
@ 0x358a7ed ZZN9starrocks10Aggregator25convert_hash_map_to_chunkEiPSt10shared_ptrINS_5ChunkEEPbENKUlRT_E_clIKSt10unique_ptrINS_38AggHashMapWithOneStringKeyWithNullableIN5phmap13flat_hash_mapINS_5SliceEPhNS_17SliceHashWithSeedILNS_9PhmapSeedE1EEENS_10SliceEqualESaISt4pairIKSE_SF_EEEELb1EEESt14default_deleteISP_EEEEDaS7
@ 0x358aba2 (unknown)
@ 0x3532ece starrocks::Aggregator::convert_hash_map_to_chunk()
@ 0x34463e2 starrocks::pipeline::AggregateBlockingSourceOperator::pull_chunk()
@ 0x310f4f2 starrocks::pipeline::PipelineDriver::process()
@ 0x578d84b starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x50e5222 starrocks::ThreadPool::dispatch_thread()
@ 0x50dfd1a starrocks::supervise_thread()
@ 0x7fbaa23faea5 start_thread
@ 0x7fbaa1a15b0d __clone
@ 0x0 (unknown)
start time: Wed Sep 27 15:14:09 CST 2023
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object
3.0.2 RELEASE (build c833698)
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000
tracker:process consumption: 3408834608
tracker:query_pool consumption: -2736
tracker:load consumption: 0
tracker:metadata consumption: 957163644
tracker:tablet_metadata consumption: 522118946
tracker:rowset_metadata consumption: 204907757
tracker:segment_metadata consumption: 30525071
tracker:column_metadata consumption: 199611870
tracker:tablet_schema consumption: 25546082
tracker:segment_zonemap consumption: 25865573
tracker:short_key_index consumption: 2642863
tracker:column_zonemap_index consumption: 8026158
tracker:ordinal_index consumption: 128680688
tracker:bitmap_index consumption: 3937600
tracker:bloom_filter_index consumption: 0
tracker:compaction consumption: 9902176
tracker:schema_change consumption: 0
tracker:column_pool consumption: 0
tracker:page_cache consumption: 0
tracker:update consumption: 843366350
tracker:chunk_allocator consumption: 2044116672
tracker:clone consumption: 0
tracker:consistency consumption: 0
*** Aborted at 1695805998 (unix time) try “date -d @1695805998” if you are using GNU date ***
PC: @ 0x6d44d17 je_large_dalloc
*** SIGSEGV (@0x0) received by PID 2784 (TID 0x7fa6cc3a6700) from PID 0; stack trace: ***
@ 0x62d7062 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fa845edc982 os::Linux::chained_handler()
@ 0x7fa845ee2ae6 JVM_handle_linux_signal
@ 0x7fa845ed9403 signalHandler()
@ 0x7fa845412630 (unknown)
@ 0x6d44d17 je_large_dalloc
@ 0x6331785 LZ4F_compressBegin_usingCDict
@ 0x502c8a4 starrocks::Lz4fBlockCompression::_compress()
@ 0x502d35c starrocks::Lz4fBlockCompression::compress()
@ 0x4ca01e6 starrocks::PageIO::compress_page_body()
@ 0x4c5190b starrocks::ScalarColumnWriter::finish_current_page()
@ 0x4c53214 starrocks::ScalarColumnWriter::append()
@ 0x4c54a75 starrocks::ScalarColumnWriter::append()
@ 0x4c54b4f starrocks::StringColumnWriter::append()
@ 0x483d1e6 starrocks::SegmentWriter::append_chunk()
@ 0x4ecb0f0 starrocks::VerticalRowsetWriter::add_columns()
@ 0x4935666 starrocks::RowsetMergerImpl<>::_do_merge_vertically()
@ 0x4936f24 starrocks::RowsetMergerImpl<>::do_merge()
@ 0x492a2cf starrocks::compaction_merge_rowsets()
@ 0x47da196 starrocks::TabletUpdates::_do_compaction()
@ 0x47db59a starrocks::TabletUpdates::compaction()
@ 0x4736053 starrocks::StorageEngine::_perform_update_compaction()
@ 0x49eb09e starrocks::StorageEngine::_update_compaction_thread_callback()
@ 0x8ad1ac0 execute_native_thread_routine
@ 0x7fa84540aea5 start_thread
@ 0x7fa844a25b0d __clone
@ 0x0 (unknown)
start time: Wed Sep 27 17:26:58 CST 2023