【OS版本】
CentOS Linux release 8.3.2011
【StarRocks版本】
2.3.3
【集群规模】
2fe + 3be
【详述】
fe master 节点报错如下：
2022-10-25 08:57:55,012 WARN (thrift-server-pool-421|40827) [MasterImpl.finishTask():194] cannot find task. type: PUBLISH_VERSION, backendId: 10323, signature: 12778321
2022-10-25 08:58:18,480 WARN (starrocks-mysql-nio-pool-4613|43334) [Database.readLock():134] slow read lock db:10324 default_cluster:qichacha_audit_log 3443ms
java.lang.Exception: null
at com.starrocks.catalog.Database.readLock(Database.java:134) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.lock(StatementPlanner.java:172) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:104) ~[starrocks-fe.jar:?]
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:337) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:322) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:440) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:676) ~[starrocks-fe.jar:?]
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:55) ~[starrocks-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]
2022-10-25 08:59:55,653 WARN (thrift-server-pool-432|41471) [MasterImpl.finishTask():194] cannot find task. type: PUBLISH_VERSION, backendId: 10323, signature: 12778877
2022-10-25 08:59:56,266 WARN (thrift-server-pool-433|41699) [MasterImpl.finishTask():194] cannot find task. type: PUBLISH_VERSION, backendId: 10323, signature: 12778878
fe follower节点报错如下：
2022-10-25 09:00:38,518 WARN (replayer|84) [BDBJournalCursor.next():148] Catch an exception when get next JournalEntity. key:40757639
com.sleepycat.je.LockTimeoutException: (JE 7.3.7) Lock expired. Locker 2095703957 -1_replayer_ReplicaThreadLocker: waited for lock on database=40712558 LockAddr:1296090845 LSN=0x1035/0x7ebaad type=READ grant=WAIT_NEW timeoutMillis=1000 startTime=1666659637518 endTime=1666659638518
Owners: [<LockInfo locker=“688079547 -40883192_ReplayThread_ReplayTxn” type=“WRITE”/>]
Waiters: []
at com.sleepycat.je.txn.LockManager.makeTimeoutException(LockManager.java:1117) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.txn.LockManager.waitForLock(LockManager.java:606) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.txn.LockManager.lock(LockManager.java:345) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.txn.BasicLocker.lockInternal(BasicLocker.java:124) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.txn.ReplicaThreadLocker.lockInternal(ReplicaThreadLocker.java:63) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.txn.Locker.lock(Locker.java:499) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3585) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3316) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.CursorImpl.lockLNAndCheckDefunct(CursorImpl.java:2138) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.CursorImpl.searchExact(CursorImpl.java:1950) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Cursor.searchExact(Cursor.java:4194) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Cursor.searchNoDups(Cursor.java:4055) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Cursor.search(Cursor.java:3857) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Cursor.getInternal(Cursor.java:1284) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Database.get(Database.java:1271) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Database.get(Database.java:1330) ~[je-7.3.7.jar:7.3.7]
at com.starrocks.journal.bdbje.CloseSafeDatabase.get(CloseSafeDatabase.java:47) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:108) [starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr.replayJournal(GlobalStateMgr.java:1721) [starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr$4.runOneCycle(GlobalStateMgr.java:1518) [starrocks-fe.jar:?]
at com.starrocks.common.util.Daemon.run(Daemon.java:115) [starrocks-fe.jar:?]
2022-10-25 09:00:38,518 WARN (replayer|84) [GlobalStateMgr.replayJournal():1739] replay journal cost too much time: 1000 replayedJournalId: 40757638
master_fe.warn.log (424.3 KB)
follower_fe.warn.log (4.3 MB)