为了更快的定位您的问题,请提供以下信息,谢谢
【详述】fe挂掉 重启后报错Environment invalid because of previous exception,通过删除ALTER SYSTEM DROP 新增ALTER SYSTEM ADD ,—helper启动还是会报改错误
【背景】 删除meta文件 通过删除ALTER SYSTEM DROP 新增ALTER SYSTEM ADD ,—helper启动还是会报该错误
【业务影响】
【StarRocks版本】例如:2.4.2
【集群规模】例如:3fe +6be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【联系方式】 2296501329@qq.com
【附件】
fe节点删除了 可以启动 一直报
2023-12-23 15:10:49,519 WARN (main|1) [NodeMgr.getFeNodeTypeAndNameFromHelpers():390] failed to get fe node type from helper node: node-220:9011. response code: 400
2023-12-23 15:10:49,519 WARN (main|1) [NodeMgr.getClusterIdAndRoleOnStartup():270] current node is not added to the group. please add it first. sleep 5 seconds and retry, current helper nodes: [node-220:9011]
节点 添加 就报错 fe断开
2023-12-23 15:10:19,731 ERROR (UNKNOWN node-218_9011_1703315352798(-1)|1) [StarRocksFE.start():169] StarRocksFE start failed
com.starrocks.journal.JournalException: failed to setup environment after retried 3 times
at com.starrocks.journal.bdbje.BDBEnvironment.setupEnvironment(BDBEnvironment.java:320) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBEnvironment.setup(BDBEnvironment.java:163) ~[starrocks-fe.jar:?]
at com.starrocks.journal.bdbje.BDBEnvironment.initBDBEnvironment(BDBEnvironment.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.journal.JournalFactory.create(JournalFactory.java:18) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr.initJournal(GlobalStateMgr.java:879) ~[starrocks-fe.jar:?]
at com.starrocks.server.GlobalStateMgr.initialize(GlobalStateMgr.java:858) ~[starrocks-fe.jar:?]
at com.starrocks.StarRocksFE.start(StarRocksFE.java:115) [starrocks-fe.jar:?]
at com.starrocks.StarRocksFE.main(StarRocksFE.java:67) [starrocks-fe.jar:?]
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 7.3.7) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 7.3.7) node-218_9011_1703315352798(-1):/home/apps/StarRocks-2.2.2/fe/meta/bdb Feeder: node-220_9011_1674009224356(3). Node: node-218_9011_1703315352798(-1) is no longer a member of the group. It was explicitly removed. HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. Originally thrown by HA thread: RepNode node-218_9011_1703315352798(-1) Originally thrown by HA thread: RepNode 192.168.2.218_9011_1703315352798(-1)
at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:228) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1766) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:151) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:267) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.Environment.(Environment.java:252) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:607) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:466) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:540) ~[je-7.3.7.jar:7.3.7]
at com.starrocks.journal.bdbje.BDBEnvironment.setupEnvironment(BDBEnvironment.java:256) ~[starrocks-fe.jar:?]
… 7 more
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 7.3.7) node-218_9011_1703315352798(-1):/home/apps/StarRocks-2.2.2/fe/meta/bdb Feeder: node-220_9011_1674009224356(3). Node: node-218_9011_1703315352798(-1) is no longer a member of the group. It was explicitly removed. HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. Originally thrown by HA thread: RepNode node-218_9011_1703315352798(-1) Originally thrown by HA thread: RepNode node-218_9011_1703315352798(-1)
at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.verifyMembership(ReplicaFeederHandshake.java:334) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.execute(ReplicaFeederHandshake.java:259) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:691) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:474) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:409) ~[je-7.3.7.jar:7.3.7]
at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1873) ~[je-7.3.7.jar:7.3.7]