starrocks3.5版本BE打core

为了更快的定位您的问题,请提供以下信息,谢谢
【详述】starrocks3.5 BE宕机Core文件,61服务器,2026-07-04 22:17发生一次;60服务器,2026-07-07 23:17发生一次
【背景】没有做操作
【业务影响】be宕机写入任务会报错
【是否存算分离】否
【StarRocks版本】3.5.12
【集群规模】3fe(2 follower+2leader)+16be(fe与be分开部署)
【机器信息】64C/512G/万兆
【联系方式】社区群29-小欧,谢谢
【附件】

  • be.out
  1. start time: Wed Mar 11 13:53:38 CST 2026, server uptime: 13:53:38 up 1628 days, 12 min, 1 user, load average: 0.08, 0.15, 0.10
  2. Run with JEMALLOC_CONF: 'percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000,metadata_thp:auto,backgroun
  3. d_thread:true,prof:true,prof_active:false’
  4. F20260707 23:17:43.121201 140160101123840 timer_thread.cpp:303] Check failed: version.load(butil::memory_order_relaxed) == id_version
  5. ** + 2 (2 vs. 5) **
  6. 3.5.13 RELEASE (build dca7c0a distro centos arch x86_64)
  7. query_id:00000000-0000-0000-0000-000000000000, fragment_instance:00000000-0000-0000-0000-000000000000, plan_node_id:-1
  8. [1783437463.126][thread: 140160101123840] je_mallctl execute purge success
  9. [1783437463.126][thread: 140160101123840] je_mallctl execute dontdump success
  10. ***** Aborted at 1783437472 (unix time) try “date -d @1783437472” if you are using GNU date *****
  11. PC: @ 0x7f7c12645387 __GI_raise
  12. ***** SIGABRT (@0x266c1) received by PID 157377 (TID 0x7f79910a0700) LWP(158823) from PID 157377; stack trace: *****
  13. ** @ 0x7f7c14ba120b __pthread_once_slow**
  14. ** @ 0xbe3e254 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)**
  15. ** @ 0x7f7c14baa630 (/usr/lib64/libpthread-2.17.so+0xf62f)**
  16. ** @ 0x7f7c12645387 __GI_raise**
  17. ** @ 0x7f7c12646a78 __GI_abort**
  18. ** @ 0x4d2f5a3 starrocks::failure_function()**
  19. ** @ 0xbe3304a google::LogMessage::Fail()**
  20. ** @ 0xbe34a84 google::LogMessageFatal::~LogMessageFatal()**
  21. ** @ 0xc01ba22 bthread::TimerThread::Task::try_delete()**
  22. ** @ 0xc01c183 bthread::TimerThread::run()**
  23. ** @ 0xc01cdc0 bthread::TimerThread::run_this(void*)**
  24. ** @ 0x7f7c14ba2ea5 start_thread**
  25. ** @ 0x7f7c1270d96d __clone**
  26. start time: Wed Jul 8 00:02:09 CST 2026, server uptime: 00:02:09 up 1746 days, 10:21, 0 users, load average: 211.56, 1045.50, 128
  27. 4.37
  28. Run with JEMALLOC_CONF: 'percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000,metadata_thp:auto,backgroun
  29. d_thread:true,prof:true,prof_active:false’
    [New LWP 158065]
    [New LWP 158569]
    [New LWP 157853]
    [New LWP 157845]
    [New LWP 157873]
    [New LWP 158056]
    [New LWP 157871]
    Core was generated by `/opt/StarRocks/be/lib/starrocks_be’.
    Program terminated with signal 6, Aborted.
    #0 0x00007f7c126d485d in ?? ()

######## be.WARNING ######## 的日志

F20260707 23:17:43.121201 140160101123840 timer_thread.cpp:303] Check failed: version.load(butil::memory_order_relaxed) == id_version + 2 (2 vs. 5)
W20260707 23:17:44.904889 139148559906560 heartbeat_server.cpp:100] Fail to handle heartbeat: Shut down in progress: BE is shutting down cached master info: TMasterInfo(network_address=TNetworkAddress(hostname=10.6.201.104, port=9020), cluster_id=298722942, epoch=6, token=, backend_ip=10.6.201.60, http_port=8030, heartbeat_flags=0, backend_id=62514, min_active_txn_id=13456172, run_mode=SHARED_NOTHING, disabled_disks=[], decommissioned_disks=[], encrypted=0, stop_regular_tablet_report=1, node_type=Backend) received master info: TMasterInfo(network_address=TNetworkAddress(hostname=10.6.201.104, port=9020), cluster_id=298722942, epoch=6, token=, backend_ip=10.6.201.60, http_port=8030, heartbeat_flags=0, backend_id=62514, min_active_txn_id=13456179, run_mode=SHARED_NOTHING, disabled_disks=[], decommissioned_disks=[], encrypted=0, stop_regular_tablet_report=1, node_type=Backend)
W20260707 23:17:49.905300 139148559906560 heartbeat_server.cpp:100] Fail to handle heartbeat: Shut down in progress: BE is shutting down cached master info: TMasterInfo(network_address=TNetworkAddress(hostname=10.6.201.104, port=9020), cluster_id=298722942, epoch=6, token=, backend_ip=10.6.201.60, http_port=8030, heartbeat_flags=0, backend_id=62514, min_active_txn_id=13456172, run_mode=SHARED_NOTHING, disabled_disks=[], decommissioned_disks=[], encrypted=0, stop_regular_tablet_report=1, node_type=Backend) received master info: TMasterInfo(network_address=TNetworkAddress(hostname=10.6.201.104, port=9020), cluster_id=298722942, epoch=6, token=, backend_ip=10.6.201.60, http_port=8030, heartbeat_flags=0, backend_id=62514, min_active_txn_id=13456183, run_mode=SHARED_NOTHING, disabled_disks=[], decommissioned_disks=[], encrypted=0, stop_regular_tablet_report=1, node_type=Backend)
W20260708 00:02:21.383314 139852250072832 fragment_context.cpp:200] [Driver] Canceled, query_id=3c7caddf-7a1d-11f1-8cb6-6805cae5bce0, instance_id=3c7caddf-7a1d-11f1-8cb6-6805cae5bce3, reason=transmit chunk rpc failed [dest_instance_id=3c7caddf-7a1d-11f1-8cb6-6805cae5bce3] [dest=10.6.201.60:8060] detail:brpc failed, error=Host is down, error_text=[E112]Not connected to 10.6.201.60:8060 yet, server_id=1836 [R1][E112]Not connected to 10.6.201.60:8060 yet, server_id=1836 [R2][E112]Not connected to 10.6.201.60:8060 yet, server_id=1836 [R3][E112]Not connected to 10.6.201.60:8060 yet, server_id=1836

系统负载指标情况:




be.out还有其它Crash堆栈吗

有后续了吗,遇到了同样的问题

我是准备升级了到3.5.19版本了!

你现在的版本是多少,我当前的版本是:3.5.15-5abb1cb

3.5.13

cn.out 输出

start time: Thu Jul 16 02:06:15 UTC 2026, server uptime: 02:06:15 up 210 days, 22:30, 0 users, load average: 86.03, 78.32, 70.51
Run with JEMALLOC_CONF: ‘percpu_arena:percpu,oversize_threshold:0,muzzy_decay_ms:5000,dirty_decay_ms:5000,metadata_thp:auto,background_thread:true,prof:true,prof_active:false’
: Number of CPUs detected is not deterministic. Per-CPU arena disabled.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/starrocks/cn/lib/hadoop/common/log4j-slf4j-impl-2.23.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/starrocks/be/lib/hadoop/common/log4j-slf4j-impl-2.23.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.