执行decommission 下线一个be，数据均衡异常缓慢

💬 StarRocks 用户问答
#1

【详述】执行alter system decommission backend “xxxxx:9050”;命令下线一个be，命令执行后storage目录空间释放情况如下图:

image
image1842×472 30 KB

12:30分后数据均衡的速度明显减缓，20点后几乎无空间释放

master FE fe.log日志中出现如下大大量的日志
2022-06-21 23:45:08,687 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455409, forwarded stmt id: 0
2022-06-21 23:45:08,687 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455410, forwarded stmt id: 0
2022-06-21 23:45:08,687 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455411, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455412, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455413, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455414, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455415, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455416, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455417, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455418, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455419, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455420, forwarded stmt id: 0
2022-06-21 23:45:08,688 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455421, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455422, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455423, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455424, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455425, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455426, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455427, forwarded stmt id: 0
2022-06-21 23:45:08,689 INFO (starrocks-mysql-nio-pool-40806|62330) [StmtExecutor.analyze():578] begin to analyze stmt: 8793455428, forwarded stmt id: 0

master fe fe.log日志文件如下:
fe.log20220621.tar (3).gz (60.8 MB)

待下线be节点的日志如下:
be.WARNING.tar.gz (143 字节)

下线be原tablet数量和数据大小如下:

image
image857×454 10.9 KB

【StarRocks版本】：2.1.6-d44c230
【集群规模】：3fe（3 follower）+4be（3个be与fe混部）
【机器信息】CPU虚拟核/内存/网卡，40C/128G/万兆

#2

show proc “/cluster_balance”; 看下结果

#3

集群主机重启后，目前观察速度还可以
mysql> show proc “/cluster_balance”;
±------------------±-------+
| Item | Number |
±------------------±-------+
| cluster_load_stat | 1 |
| working_slots | 4 |
| sched_stat | 1 |
| priority_repair | 0 |
| pending_tablets | 0 |
| running_tablets | 1 |
| history_tablets | 1000 |
±------------------±-------+
7 rows in set (0.00 sec)

京ICP备2022026421号-1