为了更快的定位您的问题,请提供以下信息,谢谢
【详述】FE master 慢锁导致的无法compaction,时间长了会导致大量慢查询
【背景】无特殊操作
【业务影响】compaction 不能够正常进行
【是否存算分离】:是存算分离模式
【StarRocks版本】例如:3.2.12【之前在3.2.5上面也遇到过】
【集群规模】例如:3fe(1 follower+2observer)+4cn
【机器信息】aws 机器资源,采用EKS集群的方式部署
fe日志:
2025-02-24 04:39:44.486+08:00 INFO (DeadlockChecker|44) [LockChecker.checkSlowLocks():109] slow db locks: [{“lockState”:“readLocked”,“slowReadLockCount”:1,“dumpThreads”:“lockHoldTime: 2327990 ms;dump thread: StarMgrMetaSyncer, id: 50\n java.base@17.0.7/jdk.internal.misc.Unsafe.park(Native Method)\n java.base@17.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)\n app//io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:717)\n app//io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:159)\n app//com.staros.proto.StarManagerGrpc$StarManagerBlockingStub.listShard(StarManagerGrpc.java:2117)\n app//com.staros.client.StarClient.lambda$listShard$21(StarClient.java:521)\n app//com.staros.client.StarClient$$Lambda$2162/0x00000008018ff338.call(Unknown Source)\n app//com.staros.client.StarClient.internalRpcCall(StarClient.java:770)\n app//com.staros.client.StarClient.listShard(StarClient.java:521)\n app//com.starrocks.lake.StarOSAgent.listShard(StarOSAgent.java:499)\n app//com.starrocks.lake.StarMgrMetaSyncer.syncTableMetaInternal(StarMgrMetaSyncer.java:301)\n app//com.starrocks.lake.StarMgrMetaSyncer.syncTableMetaAndColocationInfoInternal(StarMgrMetaSyncer.java:365)\n app//com.starrocks.lake.StarMgrMetaSyncer.syncTableMetaAndColocationInfo(StarMgrMetaSyncer.java:262)\n app//com.starrocks.lake.StarMgrMetaSyncer.runAfterCatalogReady(StarMgrMetaSyncer.java:376)\n app//com.starrocks.common.util.FrontendDaemon.runOneCycle(FrontendDaemon.java:72)\n app//com.starrocks.common.util.Daemon.run(Daemon.java:107)\n;”,“lockDbName”:“sl_attribution_preview”,“lockWaiters”:[{“threadId”:6540484,“threadName”:“thrift-server-pool-227883”},{“threadId”:6467634,“threadName”:“thrift-server-pool-227276”},{“threadId”:6536163,“threadName”:“starrocks-mysql-nio-pool-106760”},{“threadId”:6535726,“threadName”:“starrocks-mysql-nio-pool-106759”},{“threadId”:6538262,“threadName”:“thrift-server-pool-227860”},{“threadId”:6507676,“threadName”:“thrift-server-pool-227620”},{“threadId”:48,“threadName”:“DynamicPartitionScheduler”},{“threadId”:28,“threadName”:“UpdateDbUsedDataQuota”},{“threadId”:56,“threadName”:“MemoryUsageTracker”},{“threadId”:24,“threadName”:“consistency checker”},{“threadId”:98,“threadName”:“COMPACTION_DISPATCH”},{“threadId”:54,“threadName”:“autovacuum”},{“threadId”:33,“threadName”:“tablet stat mgr”}]}]