【3.1.7】BE节点异常宕机

版本3.1.7
其中一个BE节点挂了
be.out输出

3.1.7 RELEASE (build 7e897e2)
query_id:934aaaa0-0391-11ef-8173-28e424bb02b1, fragment_instance:934aaaa0-0391-11ef-8173-28e424bb03e3
tracker:process consumption: 150200632065
tracker:query_pool consumption: 103225417486
tracker:load consumption: 37267258
tracker:metadata consumption: 1176624796
tracker:tablet_metadata consumption: 56696467
tracker:rowset_metadata consumption: 59492827
tracker:segment_metadata consumption: 115509155
tracker:column_metadata consumption: 944926347
tracker:tablet_schema consumption: 5429443
tracker:segment_zonemap consumption: 57183219
tracker:short_key_index consumption: 45066677
tracker:column_zonemap_index consumption: 216223059
tracker:ordinal_index consumption: 510744200
tracker:bitmap_index consumption: 598528
tracker:bloom_filter_index consumption: 950560
tracker:compaction consumption: 173298192
tracker:schema_change consumption: 0
tracker:column_pool consumption: 16782163346
tracker:page_cache consumption: 21930504832
tracker:update consumption: 1824352396
tracker:chunk_allocator consumption: 2101096152
tracker:clone consumption: 0
tracker:consistency consumption: 0
tracker:replication consumption: 0
*** Aborted at 1714110941 (unix time) try "date -d @1714110941" if you are using GNU date ***
PC: @     0x7fef6a9b2cf5 __memcpy_ssse3_back
*** SIGSEGV (@0x0) received by PID 3434 (TID 0x7fee55ddc700) from PID 0; stack trace: ***
    @          0x6457542 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7fef6c082e9b os::Linux::chained_handler()
    @     0x7fef6c087a5d JVM_handle_linux_signal
    @     0x7fef6c07a858 signalHandler()
    @     0x7fef6b5625d0 (unknown)
    @     0x7fef6a9b2cf5 __memcpy_ssse3_back
    @          0x2ac8ee2 std::vector<>::_M_range_insert<>()
    @          0x2acd184 starrocks::BinaryColumnBase<>::append()
    @          0x563d61a starrocks::NullableColumn::append()
    @          0x561f0c2 starrocks::Chunk::append()
    @          0x4711d70 starrocks::ChunkAccumulator::push()
    @          0x2e1fa51 starrocks::pipeline::NLJoinBuildChunkStreamBuilder::init()
    @          0x2e1fd07 starrocks::pipeline::NLJoinContext::finish_one_right_sinker()
    @          0x2e2857a starrocks::pipeline::NLJoinBuildOperator::set_finishing()
    @          0x2b005c7 starrocks::pipeline::PipelineDriver::_mark_operator_finishing()
    @          0x2b006d9 starrocks::pipeline::PipelineDriver::_mark_operator_finished()
    @          0x2b00d1b starrocks::pipeline::PipelineDriver::_mark_operator_cancelled()
    @          0x2b0137a starrocks::pipeline::PipelineDriver::cancel_operators()
    @          0x5741184 starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x500c91a starrocks::ThreadPool::dispatch_thread()
    @          0x50073aa starrocks::Thread::supervise_thread()
    @     0x7fef6b55add5 start_thread
    @     0x7fef6a95cead __clone
    @                0x0 (unknown)

be.WARNING输出:

W0426 13:55:41.464787  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_3, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1996390(O) -> hash_join_build_534_0x7fd9c1c15010(X)(HashJoiner=0x7fd9c1996d90)] cancels operator hash_join_build_534_0x7fd9c1c15010(X)(HashJoiner=0x7fd9c1996d90) with finished error runtime state is cancelled
W0426 13:55:41.464802  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_4, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c2be2510(O) -> hash_join_build_534_0x7fd9c3405a10(X)(HashJoiner=0x7fd9c3405790)] cancels operator hash_join_build_534_0x7fd9c3405a10(X)(HashJoiner=0x7fd9c3405790) with finished error runtime state is cancelled
W0426 13:55:41.464819  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_5, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c3406e10(O) -> hash_join_build_534_0x7fd9c3408490(X)(HashJoiner=0x7fd9c3408210)] cancels operator hash_join_build_534_0x7fd9c3408490(X)(HashJoiner=0x7fd9c3408210) with finished error runtime state is cancelled
W0426 13:55:41.464843  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_6, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c3409390(O) -> hash_join_build_534_0x7fd9c38c8510(X)(HashJoiner=0x7fd9c3409d90)] cancels operator hash_join_build_534_0x7fd9c38c8510(X)(HashJoiner=0x7fd9c3409d90) with finished error runtime state is cancelled
W0426 13:55:41.464859  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_7, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c3ae9790(O) -> hash_join_build_534_0x7fd9c3aea410(X)(HashJoiner=0x7fd9c3aea190)] cancels operator hash_join_build_534_0x7fd9c3aea410(X)(HashJoiner=0x7fd9c3aea190) with finished error runtime state is cancelled
W0426 13:55:41.464875  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_8, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c3aeb310(O) -> hash_join_build_534_0x7fd9c3aebf90(X)(HashJoiner=0x7fd9c3aebd10)] cancels operator hash_join_build_534_0x7fd9c3aebf90(X)(HashJoiner=0x7fd9c3aebd10) with finished error runtime state is cancelled
W0426 13:55:41.464901  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_9, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c69cac90(O) -> hash_join_build_534_0x7fd9c69cb910(X)(HashJoiner=0x7fd9c69cb690)] cancels operator hash_join_build_534_0x7fd9c69cb910(X)(HashJoiner=0x7fd9c69cb690) with finished error runtime state is cancelled
W0426 13:55:41.464934  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_10, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c69cc810(O) -> hash_join_build_534_0x7fd9c69cd490(X)(HashJoiner=0x7fd9c69cd210)] cancels operator hash_join_build_534_0x7fd9c69cd490(X)(HashJoiner=0x7fd9c69cd210) with finished error runtime state is cancelled
W0426 13:55:41.464951  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_11, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c69ce390(O) -> hash_join_build_534_0x7fd9c1ad1010(X)(HashJoiner=0x7fd9c69ced90)] cancels operator hash_join_build_534_0x7fd9c1ad1010(X)(HashJoiner=0x7fd9c69ced90) with finished error runtime state is cancelled
W0426 13:55:41.464964  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_12, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1ad1f10(O) -> hash_join_build_534_0x7fd9c1ad2b90(X)(HashJoiner=0x7fd9c1ad2910)] cancels operator hash_join_build_534_0x7fd9c1ad2b90(X)(HashJoiner=0x7fd9c1ad2910) with finished error runtime state is cancelled
W0426 13:55:41.464987  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_13, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1ad3a90(O) -> hash_join_build_534_0x7fd9c1ad4710(X)(HashJoiner=0x7fd9c1ad4490)] cancels operator hash_join_build_534_0x7fd9c1ad4710(X)(HashJoiner=0x7fd9c1ad4490) with finished error runtime state is cancelled
W0426 13:55:41.465004  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_14, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1ad5610(O) -> hash_join_build_534_0x7fd9c1afa290(X)(HashJoiner=0x7fd9c1afa010)] cancels operator hash_join_build_534_0x7fd9c1afa290(X)(HashJoiner=0x7fd9c1afa010) with finished error runtime state is cancelled
W0426 13:55:41.465020  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_15, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1afb190(O) -> hash_join_build_534_0x7fd9c1afbe10(X)(HashJoiner=0x7fd9c1afbb90)] cancels operator hash_join_build_534_0x7fd9c1afbe10(X)(HashJoiner=0x7fd9c1afbb90) with finished error runtime state is cancelled
W0426 13:55:41.465035  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_16, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1afcd10(O) -> hash_join_build_534_0x7fd9c1afd990(X)(HashJoiner=0x7fd9c1afd710)] cancels operator hash_join_build_534_0x7fd9c1afd990(X)(HashJoiner=0x7fd9c1afd710) with finished error runtime state is cancelled
W0426 13:55:41.465057  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_17, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1afe890(O) -> hash_join_build_534_0x7fd9c1b58510(X)(HashJoiner=0x7fd9c1b58290)] cancels operator hash_join_build_534_0x7fd9c1b58510(X)(HashJoiner=0x7fd9c1b58290) with finished error runtime state is cancelled
W0426 13:55:41.465073  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_18, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b59410(O) -> hash_join_build_534_0x7fd9c1b5a090(X)(HashJoiner=0x7fd9c1b59e10)] cancels operator hash_join_build_534_0x7fd9c1b5a090(X)(HashJoiner=0x7fd9c1b59e10) with finished error runtime state is cancelled
W0426 13:55:41.465086  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_19, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b5af90(O) -> hash_join_build_534_0x7fd9c1b5bc10(X)(HashJoiner=0x7fd9c1b5b990)] cancels operator hash_join_build_534_0x7fd9c1b5bc10(X)(HashJoiner=0x7fd9c1b5b990) with finished error runtime state is cancelled
W0426 13:55:41.465101  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_20, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b5cb10(O) -> hash_join_build_534_0x7fd9c1b6c790(X)(HashJoiner=0x7fd9c1b6c510)] cancels operator hash_join_build_534_0x7fd9c1b6c790(X)(HashJoiner=0x7fd9c1b6c510) with finished error runtime state is cancelled
W0426 13:55:41.465128  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_21, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b6d690(O) -> hash_join_build_534_0x7fd9c1b6e310(X)(HashJoiner=0x7fd9c1b6e090)] cancels operator hash_join_build_534_0x7fd9c1b6e310(X)(HashJoiner=0x7fd9c1b6e090) with finished error runtime state is cancelled
W0426 13:55:41.465143  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_22, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b6f210(O) -> hash_join_build_534_0x7fd9c1b6fe90(X)(HashJoiner=0x7fd9c1b6fc10)] cancels operator hash_join_build_534_0x7fd9c1b6fe90(X)(HashJoiner=0x7fd9c1b6fc10) with finished error runtime state is cancelled
W0426 13:55:41.465168  3970 pipeline_driver.cpp:714] fragment_id 934aaaa0-0391-11ef-8173-28e424bb071f driver query_id=934aaaa0-0391-11ef-8173-28e424bb02b1 fragment_id=934aaaa0-0391-11ef-8173-28e424bb071f driver=driver_533_23, status=INPUT_EMPTY, operator-chain: [exchange_source_533_0x7fd9c1b70d90(X) -> hash_join_build_534_0x7fd9c1b71a10(X)(HashJoiner=0x7fd9c1b71790)] cancels operator hash_join_build_534_0x7fd9c1b71a10(X)(HashJoiner=0x7fd9c1b71790) with finished error runtime state is cancelled