LockManager detects slow lock

为了更快的定位您的问题,请提供以下信息,谢谢
【详述】fe先报错OOM,之后fe宕机,重启fe之后,fe的CPU飙升100%,查看fe的日志,里面有大量slow lock相关的日志

【背景】feOOM宕机后重启fe,fe的CPU100%,过了半小时再次重启,fe恢复
【业务影响】集群不可用
【是否存算分离】否
【StarRocks版本】3.3.3-312ed45
【集群规模】例如:1fe + 4be
【机器信息】CPU虚拟核/内存/网卡,例如:8C/64G/万兆
【联系方式】StarRocks社区群17-Golden
【附件】

  • fe.log/beINFO日志:
2024-12-20 09:08:20.525+08:00 WARN (ReportHandler|172) [LockManager.logSlowLockTrace():398] LockManager detects slow lock : {"owners":[{"id":36,"name":"tablet checker","heldFor":1339,"waitTime":2703,"locker":"TabletChecker.doCheck():285","stack":["java.base@11.0.18/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1323)","java.base@11.0.18/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)","app//com.starrocks.common.CloseableLock.lock(CloseableLock.java:27)","app//com.starrocks.common.CloseableLock.lock(CloseableLock.java:37)","app//com.starrocks.clone.TabletChecker.getTabletHealthStatusWithPriority(TabletChecker.java:802)","app//com.starrocks.clone.TabletChecker.doCheckOnePartition(TabletChecker.java:424)","app//com.starrocks.clone.TabletChecker.doCheck(TabletChecker.java:350)","app//com.starrocks.clone.TabletChecker.checkNonUrgentTablets(TabletChecker.java:238)","app//com.starrocks.clone.TabletChecker.checkAllTablets(TabletChecker.java:230)","app//com.starrocks.clone.TabletChecker.runAfterCatalogReady(TabletChecker.java:216)","app//com.starrocks.common.util.FrontendDaemon.runOneCycle(FrontendDaemon.java:72)","app//com.starrocks.common.util.Daemon.run(Daemon.java:107)"]}],"waiter":[{"id":172,"name":"ReportHandler","heldFor":"","waitTime":4042,"locker":"ReportHandler.handleUpdateTableSchema():1575"},{"id":24,"name":"PUBLISH_VERSION","heldFor":"","waitTime":4028,"locker":"DatabaseTransactionMgr.canTxnFinished():883"},{"id":35,"name":"tablet scheduler","heldFor":"","waitTime":2770,"locker":"DiskAndTabletLoadReBalancer.getPartitionStats():1625"}]}
2024-12-20 09:08:20.525+08:00 WARN (PUBLISH_VERSION|24) [LockManager.logSlowLockTrace():398] LockManager detects slow lock : {"owners":[{"id":36,"name":"tablet checker","heldFor":1339,"waitTime":2703,"locker":"TabletChecker.doCheck():285","stack":["app//com.starrocks.system.SystemInfoService.getAvailableBackendIds(SystemInfoService.java:892)","app//com.starrocks.clone.TabletChecker.getStatusWhenNotEnoughAliveReplicas(TabletChecker.java:944)","app//com.starrocks.clone.TabletChecker.getTabletHealthStatusWithPriorityUnlocked(TabletChecker.java:1036)","app//com.starrocks.clone.TabletChecker.getTabletHealthStatusWithPriority(TabletChecker.java:803)","app//com.starrocks.clone.TabletChecker.doCheckOnePartition(TabletChecker.java:424)","app//com.starrocks.clone.TabletChecker.doCheck(TabletChecker.java:350)","app//com.starrocks.clone.TabletChecker.checkNonUrgentTablets(TabletChecker.java:238)","app//com.starrocks.clone.TabletChecker.checkAllTablets(TabletChecker.java:230)","app//com.starrocks.clone.TabletChecker.runAfterCatalogReady(TabletChecker.java:216)","app//com.starrocks.common.util.FrontendDaemon.runOneCycle(FrontendDaemon.java:72)","app//com.starrocks.common.util.Daemon.run(Daemon.java:107)"]}],"waiter":[{"id":172,"name":"ReportHandler","heldFor":"","waitTime":4042,"locker":"ReportHandler.handleUpdateTableSchema():1575"},{"id":24,"name":"PUBLISH_VERSION","heldFor":"","waitTime":4028,"locker":"DatabaseTransactionMgr.canTxnFinished():883"},{"id":35,"name":"tablet scheduler","heldFor":"","waitTime":2770,"locker":"DiskAndTabletLoadReBalancer.getPartitionStats():1625"}]}