FE节点重启后无法启动

[Felix]

【详述】服务器重启后出现FE节点无法启动，后来为了验证及解决问题只部署1FE,在第一次启动的时候正常，删除pod后待容器生成新的pod便又无法启动,除非主动删除pvc或先删除"/opt/starrocks/fe/meta"
【是否存算分离】是
【StarRocks版本】version: 3.4.3
【集群规模】例如：3fe（1 follower+2observer）+3cn
【联系方式】259513855@qq.com
【fe.log】

2025-07-14 08:02:39.288Z INFO (main|1) [StarRocksFE.start():143] StarRocks FE starting, version: 3.4.3-a01aa59
2025-07-14 08:02:39.294Z INFO (main|1) [NetUtils.getHosts():79] ipv6 link local address fe80:0:0:0:f820:11ff:fe3b:5838%eth0 is skipped
2025-07-14 08:02:39.295Z INFO (main|1) [FrontendOptions.initAddrUseIp():269] check ip address: /172.18.2.74
2025-07-14 08:02:39.295Z INFO (main|1) [FrontendOptions.initAddrUseIp():292] Use IP init local addr, IP: /172.18.2.74
2025-07-14 08:02:39.364Z INFO (main|1) [ConsistencyChecker.initWorkTime():133] parsed startDate: 1970-01-01T23:00:00.000+0000, endDate: 1970-01-01T04:00:00.000+0000
2025-07-14 08:02:39.570Z INFO (main|1) [AuthorizationMgr.initBuiltinRoleUnlocked():305] create built-in role root[-1]
2025-07-14 08:02:39.574Z INFO (main|1) [AuthorizationMgr.initBuiltinRoleUnlocked():305] create built-in role db_admin[-2]
2025-07-14 08:02:39.574Z INFO (main|1) [AuthorizationMgr.initBuiltinRoleUnlocked():305] create built-in role cluster_admin[-3]
2025-07-14 08:02:39.575Z INFO (main|1) [AuthorizationMgr.initBuiltinRoleUnlocked():305] create built-in role user_admin[-4]
2025-07-14 08:02:39.575Z INFO (main|1) [AuthorizationMgr.initBuiltinRoleUnlocked():305] create built-in role public[-5]
2025-07-14 08:02:39.984Z INFO (main|1) [NodeMgr.getHelperNodes():651] get helper nodes: [172.18.2.74:9010]
2025-07-14 08:02:39.992Z INFO (main|1) [NodeMgr.getClusterIdAndRoleOnStartup():503] Current run_mode is shared_data
2025-07-14 08:02:39.992Z INFO (main|1) [NodeMgr.getClusterIdAndRoleOnStartup():508] Got role: FOLLOWER, node name: 172.18.2.137_9010_1752479447183 and run_mode: shared_data
2025-07-14 08:02:39.994Z INFO (main|1) [BDBEnvironment.ensureHelperInLocal():340] skip check local environment because helper node and local node are identical.
2025-07-14 08:02:40.012Z INFO (main|1) [BDBEnvironment.setupEnvironment():270] start to setup bdb environment for 1 times
2025-07-14 08:02:50.331Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [StateChangeExecutor.notifyNewFETypeTransfer():63] notify new FE type transfer: UNKNOWN
2025-07-14 08:02:50.332Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [BDBEnvironment.setupEnvironment():281] replicated environment is all set, wait for state change...
2025-07-14 08:03:00.333Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [BDBEnvironment.setupEnvironment():289] state change done, current role UNKNOWN
2025-07-14 08:03:00.341Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [BDBEnvironment.setupEnvironment():293] end setup bdb environment after 1 times
2025-07-14 08:03:00.349Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.loadImage():1515] image does not exist: /opt/starrocks/fe/meta/image/v2/image.0
2025-07-14 08:03:00.398Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [StarManagerServer.start():77] starting star manager on port 6090 ...
2025-07-14 08:03:00.577Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [StarManagerServer.start():92] create default executor for grpc server ...
2025-07-14 08:03:00.848Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [StarMgrServer.loadImage():217] star mgr image does not exist in /opt/starrocks/fe/meta/image/starmgr/image.0.
2025-07-14 08:03:00.848Z INFO (stateChangeExecutor|92) [StateChangeExecutor.runOneCycle():86] begin to transfer FE type from INIT to UNKNOWN
2025-07-14 08:03:00.849Z INFO (stateChangeExecutor|92) [StateChangeExecutor.runOneCycle():184] finished to transfer FE type from INIT to UNKNOWN
2025-07-14 08:03:02.848Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:04.849Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:06.849Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:08.850Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:10.850Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:12.851Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:14.851Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:16.852Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:18.852Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:20.853Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:22.853Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:24.854Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:26.854Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:28.855Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:30.855Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:32.856Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:34.856Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:36.857Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:38.857Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:40.858Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:42.858Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:44.858Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:46.859Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:48.859Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:50.860Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:52.860Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:54.861Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:56.861Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:03:58.862Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:00.862Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:00.862Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:04:02.863Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:04.863Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:06.864Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:08.864Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:10.865Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:12.865Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:14.866Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:16.866Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:18.867Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:20.867Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:22.867Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:24.868Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:26.868Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:28.869Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:30.869Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:32.870Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:34.870Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:36.871Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:38.871Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:40.872Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:42.872Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:44.873Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:46.873Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:48.873Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:50.874Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:52.874Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:54.875Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:56.875Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:04:58.876Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:00.876Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:00.876Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:05:02.877Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:04.877Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:06.878Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:08.878Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:10.879Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:12.879Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:14.880Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:16.880Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:18.880Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:20.881Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:22.881Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:24.882Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:26.882Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:28.883Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:30.883Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:32.883Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:34.884Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:36.884Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:38.885Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:40.885Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:42.886Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:44.886Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:46.886Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:48.887Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:50.887Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:52.888Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:54.888Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:56.889Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:05:58.889Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:00.890Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:00.890Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:06:02.890Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:04.891Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:06.891Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:08.891Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:10.892Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:12.892Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:14.893Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:16.893Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:18.894Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:20.894Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:22.894Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:24.895Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:26.895Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:28.896Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:30.896Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:32.897Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:34.897Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:36.897Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:38.898Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:40.898Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:42.899Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:44.899Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:46.900Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:48.900Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:50.900Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:52.901Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:54.901Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:56.902Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:06:58.902Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:00.902Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:00.903Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:07:02.903Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:04.903Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:06.904Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:08.904Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:10.905Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:12.905Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:14.905Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:16.906Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:18.906Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:20.907Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:22.907Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:24.908Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:26.908Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:28.908Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:30.909Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:32.909Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:34.909Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:36.910Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:38.910Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:40.911Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:42.911Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:44.911Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:46.912Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:48.912Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:50.913Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:52.913Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:54.913Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:56.914Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:07:58.914Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:00.915Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:00.915Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:08:02.915Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:04.916Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:06.916Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:08.916Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:10.917Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:12.917Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:14.918Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:16.918Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false
2025-07-14 08:08:18.918Z INFO (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1206] wait globalStateMgr to be ready. FE type: INIT. is ready: false

【fe.warn.log】

2025-07-14 08:02:50.331Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [StateChangeExecutor.notifyNewFETypeTransfer():63] notify new FE type transfer: UNKNOWN
2025-07-14 08:04:00.862Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:05:00.876Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:06:00.890Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:07:00.903Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.
2025-07-14 08:08:00.915Z WARN (UNKNOWN 172.18.2.137_9010_1752479447183(-1)|1) [GlobalStateMgr.waitForReady():1210] It took too much time for FE to transfer to a stable state(LEADER/FOLLOWER), it maybe caused by one of the following reasons: 1. There are too many BDB logs to replay, because of previous failure of checkpoint(you can check the create time of image file under meta/image dir). 2. Majority voting members(LEADER or FOLLOWER) of the FE cluster haven't started completely. 3. FE node has multiple IPs, you should configure the priority_networks in fe.conf to match the ip record in meta/image/ROLE. And we don't support change the ip of FE node. Ignore this reason if you are using FQDN. 4. The time deviation between FE nodes is greater than 5s, please use ntp or other tools to keep clock synchronized. 5. The configuration of edit_log_port has changed, please reset to the original value. 6. The replayer thread may get stuck, please use jstack to find the details.