fe节点无法正常运行

💬 StarRocks 用户问答 日常运维
#1

为了更快的定位您的问题，请提供以下信息，谢谢
【详述】数据库使用物化视图，使用routine load 导入数据，过程中发现无法正常访问数据库，fe follow节点全down，leader 节点cpu 100%，重启所有节点后作业数据库可以进入，但依然无法正常工作。
部分日志为：
2023-12-24 11:53:48,936 INFO (UNKNOWN 10.141.6.103_9010_1701704288439(-1)|1) [TaskManager.replayCreateTaskRun():686] replayCreateTaskRun:TaskRunStatus{queryId=‘3b009df1-a16c-11ee-831a-00163e0c5db1’, taskName=‘mv-1908529’, createTime=1703319671350, finishTime=0, state=PENDING, progress=0%, dbName=‘betcenter_risk’, definition=‘insert overwrite risk_ip_black_white_summary SELECT betcenter_risk.risk_ip_black_white.ip_addr, sum(betcenter_risk.risk_ip_black_white.none_tag) AS none, sum(betcenter_risk.risk_ip_black_white.spite_tag) AS spite, sum(betcenter_risk.risk_ip_black_white.harass_tag) AS harass, sum(betcenter_risk.risk_ip_black_white.duidu_tag) AS duidu, sum(betcenter_risk.risk_ip_black_white.fraud_tag) AS fraud, sum(betcenter_risk.risk_ip_black_white.arbitrage_tag) AS arbitrage, sum(betcenter_risk.risk_ip_black_white.is_black) AS black, sum(betcenter_risk.risk_ip_black_white.is_white) AS white, sum(betcenter_risk.risk_ip_black_white.is_suspicious) AS suspicious, max(betcenter_risk.risk_ip_black_white.update_time) AS update_time
FROM betcenter_risk.risk_ip_black_white
GROUP BY betcenter_risk.risk_ip_black_white.ip_addr’, postRun=‘ANALYZE SAMPLE TABLE risk_ip_black_white_summary WITH ASYNC MODE’, user=‘root’, errorCode=0, errorMessage=‘null’, expireTime=1703416160721, priority=0, mergeRedundant=false, extraMessage={“forceRefresh”:false,“mvPartitionsToRefresh”:[],“refBasePartitionsToRefreshMap”:{},“basePartitionsToRefreshMap”:{}}}
2023-12-24 11:53:48,936 INFO (UNKNOWN 10.141.6.103_9010_1701704288439(-1)|1) [TaskManager.replayCreateTaskRun():686] replayCreateTaskRun:TaskRunStatus{queryId=‘922f4ca3-a16b-11ee-831a-00163e0c5db1’, taskName=‘mv-1908665’, createTime=1703319388121, finishTime=0, state=RUNNING, progress=0%, dbName=‘betcenter_risk’, definition=‘insert overwrite risk_ip_all_site_agg_summary SELECT a.ip_addr, a.sites, a.users, a.fail_req_users, ifnull(betcenter_risk.b.none, 0) AS none, ifnull(betcenter_risk.b.spite, 0) AS spite, ifnull(betcenter_risk.b.harass, 0) AS harass, ifnull(betcenter_risk.b.duidu, 0) AS duidu, ifnull(betcenter_risk.b.fraud, 0) AS fraud, ifnull(betcenter_risk.b.arbitrage, 0) AS arbitrage, ifnull(betcenter_risk.b.black, 0) AS black, ifnull(betcenter_risk.b.white, 0) AS white, ifnull(betcenter_risk.b.suspicious, 0) AS suspicious, if(a.update_time > (ifnull(betcenter_risk.b.update_time, ‘1970-01-01 00:00:00’)), a.update_time, ifnull(betcenter_risk.b.update_time, ‘1970-01-01 00:00:00’)) AS update_time
FROM (SELECT betcenter_risk.risa.ip_addr, sum(betcenter_risk.risa.count) AS sites, count(DISTINCT betcenter_risk.risa.users) AS users, count(DISTINCT betcenter_risk.risa.fail_req_users) AS fail_req_users, max(betcenter_risk.risa.update_time) AS update_time
FROM betcenter_risk.risk_ip_site_agg AS risa
GROUP BY betcenter_risk.risa.ip_addr) a LEFT OUTER JOIN betcenter_risk.risk_ip_black_white_summary AS b ON a.ip_addr = betcenter_risk.b.ip_addr’, postRun=‘ANALYZE SAMPLE TABLE risk_ip_all_site_agg_summary WITH ASYNC MODE’, user=‘root’, errorCode=0, errorMessage=‘null’, expireTime=1703416160666, priority=0, mergeRedundant=false, extraMessage={“forceRefresh”:false,“mvPartitionsToRefresh”:[],“refBasePartitionsToRefreshMap”:{},“basePartitionsToRefreshMap”:{}}}

【背景】存算分离，物化视图，routine load 导入数据
【业务影响】
【是否存算分离】
【StarRocks版本】例如：3.1.5
【集群规模】3fe（1 follower+2observer）+2cn
cn1.tar.gz (343.0 KB) cn2.tar.gz (364.4 KB) cn3.tar.gz (344.4 KB) fe1.tar.gz (199.2 KB) fe2.tar.gz (266.2 KB) fe3.tar.gz (585.6 KB)

#2

fe 启动后Leader节点反复报

2023-12-25 14:37:23,655 WARN (replayer|106) [BDBJournalCursor.wrapDatabaseException():102] failed to read after retried 1 times! key = 2316684, db = CloseSafeDatabase{db=2315297}
2023-12-25 14:37:27,655 WARN (replayer|106) [BDBJournalCursor.wrapDatabaseException():102] failed to read after retried 2 times! key = 2316684, db = CloseSafeDatabase{db=2315297}
2023-12-25 14:37:31,655 WARN (replayer|106) [BDBJournalCursor.wrapDatabaseException():102] failed to read after retried 3 times! key = 2316684, db = CloseSafeDatabase{db=2315297}
2023-12-25 14:37:31,656 WARN (replayer|106) [GlobalStateMgr.replayJournalInner():2319] catch exception when replaying 2316684,
com.starrocks.journal.JournalException: failed to read after retried 3 times! key = 2316684, db = CloseSafeDatabase{db=2315297}
at com.starrocks.journal.bdbje.BDBJournalCursor.wrapDatabaseException(BDBJournalCursor.java:103) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:325) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr.replayJournalInner(GlobalStateMgr.java:2296) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr$5.runOneCycle(GlobalStateMgr.java:2161) ~[starrocks-fe.jar:?]
at com.starrocks.common.util.Daemon.run(Daemon.java:115) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr$5.run(GlobalStateMgr.java:2226) ~[starrocks-fe.jar:?]
Caused by: com.sleepycat.je.LockTimeoutException: (JE 18.3.16) Lock expired. Locker 672448031 -1_replayer_ReplicaThreadLocker: waited for lock on database=2315297 LockAddr:1918955849 LSN=0x25f/0x771976 type=READ grant=WAIT_NEW timeoutMillis=1000 startTime=1703486250655 endTime=1703486251655
Owners: [<LockInfo locker=“260273002 -6808111_ReplayThread_ReplayTxn” type=“WRITE”/>]
Waiters: []

at com.sleepycat.je.txn.LockManager.makeTimeoutException(LockManager.java:1154) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.LockManager.waitForLock(LockManager.java:621) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.LockManager.lock(LockManager.java:353) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.BasicLocker.lockInternal(BasicLocker.java:125) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.rep.txn.ReplicaThreadLocker.lockInternal(ReplicaThreadLocker.java:63) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.Locker.lock(Locker.java:496) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3674) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3399) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLNAndCheckDefunct(CursorImpl.java:2177) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.searchExact(CursorImpl.java:1989) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.searchExact(Cursor.java:4210) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.searchNoDups(Cursor.java:4071) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.search(Cursor.java:3842) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.getInternal(Cursor.java:1290) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Database.get(Database.java:1403) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Database.get(Database.java:1462) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.starrocks.journal.bdbje.CloseSafeDatabase.get(CloseSafeDatabase.java:60) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:291) ~[starrocks-fe.jar:?]
... 4 more

2023-12-25 14:37:31,656 ERROR (replayer|106) [GlobalStateMgr$5.runOneCycle():2170] replayer thread catch an exception when replay journal 2316684.
com.starrocks.journal.JournalException: failed to read after retried 3 times! key = 2316684, db = CloseSafeDatabase{db=2315297}
at com.starrocks.journal.bdbje.BDBJournalCursor.wrapDatabaseException(BDBJournalCursor.java:103) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:325) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr.replayJournalInner(GlobalStateMgr.java:2296) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr$5.runOneCycle(GlobalStateMgr.java:2161) ~[starrocks-fe.jar:?]
at com.starrocks.common.util.Daemon.run(Daemon.java:115) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr$5.run(GlobalStateMgr.java:2226) ~[starrocks-fe.jar:?]
Caused by: com.sleepycat.je.LockTimeoutException: (JE 18.3.16) Lock expired. Locker 672448031 -1_replayer_ReplicaThreadLocker: waited for lock on database=2315297 LockAddr:1918955849 LSN=0x25f/0x771976 type=READ grant=WAIT_NEW timeoutMillis=1000 startTime=1703486250655 endTime=1703486251655
Owners: [<LockInfo locker=“260273002 -6808111_ReplayThread_ReplayTxn” type=“WRITE”/>]
Waiters: []

at com.sleepycat.je.txn.LockManager.makeTimeoutException(LockManager.java:1154) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.LockManager.waitForLock(LockManager.java:621) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.LockManager.lock(LockManager.java:353) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.BasicLocker.lockInternal(BasicLocker.java:125) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.rep.txn.ReplicaThreadLocker.lockInternal(ReplicaThreadLocker.java:63) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.txn.Locker.lock(Locker.java:496) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3674) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3399) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.lockLNAndCheckDefunct(CursorImpl.java:2177) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.dbi.CursorImpl.searchExact(CursorImpl.java:1989) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.searchExact(Cursor.java:4210) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.searchNoDups(Cursor.java:4071) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.search(Cursor.java:3842) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Cursor.getInternal(Cursor.java:1290) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Database.get(Database.java:1403) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.sleepycat.je.Database.get(Database.java:1462) ~[starrocks-bdb-je-18.3.16.jar:?]
at com.starrocks.journal.bdbje.CloseSafeDatabase.get(CloseSafeDatabase.java:60) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:291) ~[starrocks-fe.jar:?]
... 4 more

2023-12-25 14:37:32,248 INFO (nioEventLoopGroup-5-1|131) [RestBaseAction.handleRequest():71] receive http request. url=/metrics

#3

2023-12-25 14:56:01,630 WARN (thrift-server-pool-812|1185) [TIOStreamTransport.close():110] Error closing output stream.
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118) ~[?:1.8.0_392-392]
at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[?:1.8.0_392-392]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[?:1.8.0_392-392]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[?:1.8.0_392-392]
at java.io.FilterOutputStream.close(FilterOutputStream.java:158) ~[?:1.8.0_392-392]
at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:108) ~[libthrift-0.13.0.jar:0.13.0]
at org.apache.thrift.transport.TSocket.close(TSocket.java:235) ~[libthrift-0.13.0.jar:0.13.0]
at com.starrocks.common.SRTThreadPoolServer$WorkerProcess.run(SRTThreadPoolServer.java:326) ~[starrocks-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_392-392]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_392-392]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_392-392]
2023-12-25 14:56:01,763 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40930, 40933, 40924, 40927]
2023-12-25 14:56:01,777 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40928, 40931, 40925]
2023-12-25 14:56:01,780 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40929, 40932, 40926]
2023-12-25 14:56:01,809 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40938, 40940, 40942]
2023-12-25 14:56:01,817 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40944, 40937, 40939, 40943]
2023-12-25 14:56:01,818 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40945, 40946, 40941]
2023-12-25 14:56:01,848 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40951, 40954, 40957]
2023-12-25 14:56:01,851 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40952, 40955, 40958]
2023-12-25 14:56:01,853 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40950, 40953, 40956, 40959]
2023-12-25 14:56:01,863 INFO (statistics meta manager|34) [LocalMetastore.buildPartitions():1725] start to build 1 partitions sequentially for table statistics.table_statistic_v1 with 10 replicas
2023-12-25 14:56:01,871 INFO (statistics meta manager|34) [LocalMetastore.buildPartitionsSequentially():1757] build partitions sequentially, send task one by one, all tasks timeout 40s
2023-12-25 14:56:01,878 WARN (thrift-server-pool-809|1182) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.203, be_port:9060, http_port:8040), task_type:CREATE, signature:3814461, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,878 WARN (thrift-server-pool-808|1181) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.202, be_port:9060, http_port:8040), task_type:CREATE, signature:3814469, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,878 WARN (thrift-server-pool-808|1181) [LeaderImpl.finishTask():235] task type: CREATE, status_code: RUNTIME_ERROR, create tablet failed, backendId: 10224, signature: 3814469
2023-12-25 14:56:01,878 WARN (thrift-server-pool-823|1196) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.202, be_port:9060, http_port:8040), task_type:CREATE, signature:3814462, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,878 WARN (thrift-server-pool-810|1183) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.204, be_port:9060, http_port:8040), task_type:CREATE, signature:3814470, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,879 WARN (thrift-server-pool-822|1195) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.204, be_port:9060, http_port:8040), task_type:CREATE, signature:3814464, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,879 WARN (statistics meta manager|34) [LocalMetastore.waitForFinished():1962] fail to create tablet: 10224: [create tablet failed]
2023-12-25 14:56:01,879 WARN (thrift-server-pool-823|1196) [LeaderImpl.finishTask():235] task type: CREATE, status_code: RUNTIME_ERROR, create tablet failed, backendId: 10224, signature: 3814462
2023-12-25 14:56:01,879 WARN (thrift-server-pool-822|1195) [LeaderImpl.finishTask():235] task type: CREATE, status_code: RUNTIME_ERROR, create tablet failed, backendId: 10226, signature: 3814464
2023-12-25 14:56:01,879 WARN (thrift-server-pool-809|1182) [LeaderImpl.finishTask():235] task type: CREATE, status_code: RUNTIME_ERROR, create tablet failed, backendId: 10225, signature: 3814461
2023-12-25 14:56:01,879 WARN (statistics meta manager|34) [StatisticsMetaManager.createSampleStatisticsTable():161] Failed to create sample statisticsfail to create tablet: 10224: [create tablet failed]
2023-12-25 14:56:01,879 WARN (thrift-server-pool-810|1183) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10226, signature: 3814470
2023-12-25 14:56:01,879 WARN (statistics meta manager|34) [StatisticsMetaManager.refreshStatisticsTable():338] create statistics table table_statistic_v1 failed
2023-12-25 14:56:01,879 WARN (thrift-server-pool-824|1197) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.203, be_port:9060, http_port:8040), task_type:CREATE, signature:3814463, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,880 WARN (thrift-server-pool-824|1197) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10225, signature: 3814463
2023-12-25 14:56:01,880 WARN (thrift-server-pool-825|1198) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.204, be_port:9060, http_port:8040), task_type:CREATE, signature:3814466, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,880 WARN (thrift-server-pool-825|1198) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10226, signature: 3814466
2023-12-25 14:56:01,880 WARN (thrift-server-pool-826|1199) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.202, be_port:9060, http_port:8040), task_type:CREATE, signature:3814468, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,880 WARN (thrift-server-pool-811|1184) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.203, be_port:9060, http_port:8040), task_type:CREATE, signature:3814467, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,880 WARN (thrift-server-pool-826|1199) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10224, signature: 3814468
2023-12-25 14:56:01,880 WARN (thrift-server-pool-827|1200) [LeaderImpl.finishTask():187] finish task reports bad. request: TFinishTaskRequest(backend:TBackend(host:10.141.4.203, be_port:9060, http_port:8040), task_type:CREATE, signature:3814465, task_status:TStatus(status_code:RUNTIME_ERROR, error_msgs:[create tablet failed]), report_version:17033904960000)
2023-12-25 14:56:01,881 WARN (thrift-server-pool-811|1184) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10225, signature: 3814467
2023-12-25 14:56:01,881 WARN (thrift-server-pool-827|1200) [LeaderImpl.finishTask():221] cannot find task. type: CREATE, backendId: 10225, signature: 3814465
2023-12-25 14:56:01,882 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40963, 40966, 40969, 40971]
2023-12-25 14:56:01,888 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40964, 40967, 40970]
2023-12-25 14:56:01,890 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40965, 40968, 40972]
2023-12-25 14:56:01,910 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40978, 40980, 40983]
2023-12-25 14:56:01,918 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40976, 40979, 40981, 40984]
2023-12-25 14:56:01,919 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40977, 40982, 40985]
2023-12-25 14:56:01,943 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40993, 40996, 40990]
2023-12-25 14:56:01,950 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40994, 40997, 40991]
2023-12-25 14:56:01,952 INFO (StarMgrMetaSyncer|54) [StarMgrMetaSyncer.dropTabletAndDeleteShard():117] failedTablets is [40992, 40995, 40998, 40989]

#4

fe01_1513.tar.gz (1.9 KB) fe02_1513.tar.gz (2.7 KB) fe03_1513.tar.gz (581 字节)

#5

这个问题现在解决了吗，是怎么解决的呢？

京ICP备2022026421号-1