执行insert into select 语句，会发生Query cancelled by crash of backends.

执行下述语句：
MySQL [AIEOM]> insert into AIEOM.sf_test_user_tag_string select distinct obj.userSid as id,100000000000007 as tagId,‘WEB’ as tagValue, ‘2023-04-21’ as tagDate,‘2023-04-21 21:53:03’ as updateTime from servicecloud.aiopsUser as obj left join AIEOM.user_tag_int as A1 on obj.userSid = A1.id and A1.tagId = 100000000000047 and A1.tagValue<=0 where ( A1.id is not null ) limit 500000;
ERROR 1064 (HY000): Query cancelled by crash of backends.
之后be就会挂掉

image1130×208 19.1 KB

image741×159 10.5 KB

be.out的报错信息如下：
*** Aborted at 1682141245 (unix time) try “date -d @1682141245” if you are using GNU date ***
PC: @ 0x25e3987 starrocks::vectorized::DistinctStreamingNode::get_next()
*** SIGSEGV (@0x0) received by PID 24814 (TID 0x7f0e98f05700) from PID 0; stack trace: ***
@ 0x3fa3ad2 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7f0f17da3630 (unknown)
@ 0x25e3987 starrocks::vectorized::DistinctStreamingNode::get_next()
@ 0x258b5c2 starrocks::vectorized::DistinctBlockingNode::open()
@ 0x274dadb starrocks::vectorized::ProjectNode::open()
@ 0x2053644 starrocks::PlanFragmentExecutor::_open_internal_vectorized()
@ 0x2054347 starrocks::PlanFragmentExecutor::open()
@ 0x1fd8fab starrocks::FragmentExecState::execute()
@ 0x1fdd5dc starrocks::FragmentMgr::exec_actual()
@ 0x1fdde81 _ZNSt17_Function_handlerIFvvEZN9starrocks11FragmentMgr18exec_plan_fragmentERKNS1_23TExecPlanFragmentParamsERKSt8functionIFvPNS1_20PlanFragmentExecutorEEESC_EUlvE_E9_M_invokeERKSt9_Any_data
@ 0x2132549 starrocks::ThreadPool::dispatch_thread()
@ 0x212e0fa starrocks::supervise_thread()
@ 0x7f0f17d9bea5 start_thread
@ 0x7f0f173b6b0d __clone
@ 0x0 (unknown)

您好 您是什么版本的集群？

你好，是2.3.0

@yuchen1019 可以帮忙分析一下这个问题的原因吗？
当前集群环境，1fe，3be，3be内存128G，每个节点的内存占用60%左右，cpu占用50%左右

1. 表使用的是更新模型，执行以上sql后，be会随机挂机，有时候是1个，有时候是2个节点。（换成主键模型也是一样）
2. 把distinct语句去掉以后，就能正常执行了，源表数据量不大，才几万
3. 集群负载低的情况下，也会出现这个问题，貌似和集群资源没太大关系似的
4. 这个集群的Starrocks表很多，数据量也很大
5. 同样的sql在测试环境跑没有问题。

您好 是哪个版本 麻烦发下be crash时对应的be.out日志文件

是2.3.0版本
be.out的日志如下：
start time: Mon Apr 24 15:41:07 CST 2023
*** Aborted at 1683688232 (unix time) try “date -d @1683688232” if you are using GNU date ***
PC: @ 0x25e3987 starrocks::vectorized::DistinctStreamingNode::get_next()
*** SIGSEGV (@0x0) received by PID 17293 (TID 0x7f9e32104700) from PID 0; stack trace: ***
@ 0x3fa3ad2 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7f9eb170a630 (unknown)
@ 0x25e3987 starrocks::vectorized::DistinctStreamingNode::get_next()
@ 0x258b5c2 starrocks::vectorized::DistinctBlockingNode::open()
@ 0x274dadb starrocks::vectorized::ProjectNode::open()
@ 0x2053644 starrocks::PlanFragmentExecutor::_open_internal_vectorized()
@ 0x2054347 starrocks::PlanFragmentExecutor::open()
@ 0x1fd8fab starrocks::FragmentExecState::execute()
@ 0x1fdd5dc starrocks::FragmentMgr::exec_actual()
@ 0x1fdde81 _ZNSt17_Function_handlerIFvvEZN9starrocks11FragmentMgr18exec_plan_fragmentERKNS1_23TExecPlanFragmentParamsERKSt8functionIFvPNS1_20PlanFragmentExecutorEEESC_EUlvE_E9_M_invokeERKSt9_Any_data
@ 0x2132549 starrocks::ThreadPool::dispatch_thread()
@ 0x212e0fa starrocks::supervise_thread()
@ 0x7f9eb1702ea5 start_thread
@ 0x7f9eb0d1d9fd __clone
@ 0x0 (unknown)