【详述】
存算分离集群 CN 进程因 SIGABRT 崩溃,导致节点完全失效。该崩溃由文件系统初始化过程中未捕获的 std::system_error(EDEADLK) 异常触发。
【业务影响】
CN 宕机，导致导入异常，查询失败
【是否存算分离】
是
【StarRocks版本】
4.1.0
【集群规模】例如：3fe（1 leader+2 follower）+3be（fe与be混部）
【机器信息】CPU虚拟核/内存/网卡，例如：48C/64G/万兆
【联系方式】
【附件】
-cn.out
terminate called after throwing an instance of ‘std::system_error’
what(): Resource deadlock avoided
odeon-4.1.0 RELEASE (build 83bdad9 distro centos arch x86_64)
query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000, plan_node_id:-1
*** Aborted at 1785132601 (unix time) try “date -d @1785132601” if you are using GNU date ***
PC: @ 0x7f83a5bca387 __GI_raise
*** SIGABRT (@0x3e80000c2b7) received by PID 49847 (TID 0x7f80794bd700) LWP(52079) from PID 49847; stack trace: ***
@ 0x7f83a812720b __pthread_once_slow
@ 0xfcff254 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
@ 0x7f83a8130630 (/usr/lib64/libpthread-2.17.so+0xf62f)
@ 0x7f83a5bca387 __GI_raise
@ 0x7f83a5bcba78 __GI_abort
@ 0x5c5a8a3 __gnu_cxx::__verbose_terminate_handler() [clone .cold]
@ 0x12d3cef6 __cxxabiv1::__terminate(void ()())
@ 0x5c5a73b std::terminate()
@ 0x12d3d093 __cxa_throw
@ 0x9e0bc58 __wrap___cxa_throw
@ 0x5c5cef3 std::__throw_system_error(int)
@ 0x55c55bf starrocks::StarOSWorker::new_shared_filesystem(std::basic_string_view<char, std::char_traits >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits<char@
@ 0x9c9af60 starrocks::StarOSWorker::build_filesystem_from_shard_info(staros::starlet::ShardInfo const&, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::a@
@ 0x9c9d055 starrocks::StarOSWorker::get_shard_filesystem(unsigned long, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std:@
@ 0xc591351 starrocks::StarletFileSystem::new_random_access_file(starrocks::RandomAccessFileOptions const&, starrocks::FileInfo const&)
@ 0xc58c6dd starrocks::StarletFileSystem::new_random_access_file(starrocks::RandomAccessFileOptions const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
@ 0x95d201c starrocks::ProtobufFile::load(google::protobuf::Message, bool)
@ 0x903d208 starrocks::lake::TabletManager::load_tablet_metadata(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, long, std::shared_ptrstarrocks::FileSystem const&)
@ 0x9042f1a starrocks::lake::TabletManager::get_tablet_metadata(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, starrocks::lake::CacheOptions const&, long, std::shared_ptrstarrocks::FileSystem const&)
@ 0x9043685 starrocks::lake::TabletManager::get_tablet_metadata(long, long, starrocks::lake::CacheOptions const&, long, std::shared_ptrstarrocks::FileSystem const&)
@ 0x9045e7f starrocks::lake::TabletManager::get_tablet(long, long, bool, bool)
@ 0x99608ad starrocks::lake::TabletParallelCompactionManager::create_parallel_tasks(long, long, long, starrocks::TabletParallelConfig const&, std::shared_ptrstarrocks::lake::CompactionTaskCallback, bool, starrocks::ThreadPool*, std::function<bool ()> const&, std::fu@
@ 0x9946200 starrocks::lake::CompactionScheduler::process_parallel_compaction(starrocks::CompactRequest const*, starrocks::CompactResponse*, std::shared_ptrstarrocks::lake::CompactionTaskCallback const&)
@ 0x9949e5e starrocks::lake::CompactionScheduler::compact(google::protobuf::RpcController*, starrocks::CompactRequest const*, starrocks::CompactResponse*, google::protobuf::Closure*)
@ 0x9cb59dd starrocks::LakeServiceImpl::compact(google::protobuf::RpcController*, starrocks::CompactRequest const*, starrocks::CompactResponse*, google::protobuf::Closure*)
@ 0xfff732f brpc::policy::ProcessRpcRequest(brpc::InputMessageBase*)
@ 0xff17cc7 brpc::ProcessInputMessage(void*)
@ 0xff1901e brpc::InputMessenger::OnNewMessages(brpc::Socket*)
@ 0xff06f7d brpc::Socket::ProcessEvent(void*)
@ 0xfed48d7 bthread::TaskGroup::task_runner(long)
@ 0x1003c7e1 bthread_make_fcontext