【详述】starrocks的fe节点宕机或滚动重启fe节点。导致表写不进去了。
【业务影响】
【是否存算分离】否
【StarRocks版本】3.0.1
【集群规模】例如：3fe（1 follower+2observer）+9be（fe与be分开部署）
联系方式，例如：社区群14-流苏如画，谢谢
【附件】
2023-11-16 18:21:20,921 INFO com.starrocks.data.load.stream.TransactionStreamLoader [] - Transaction prepared, label : 881afb56-4c43-4408-9c08-ff9915ca40d1, body : {
“Status”: “INTERNAL_ERROR”,
“Message”: “too many filtered rows”,
“ErrorURL”: “http://xxxx:8040/api/_load_error_log?file=error_log_144bca50e15393be_f5109d8d0d1becb2”
}
2023-11-16 18:21:20,923 WARN org.apache.kafka.clients.consumer.ConsumerConfig [] - The configuration ‘value.serializer’ was supplied but isn’t a known config.
2023-11-16 18:21:20,923 WARN org.apache.kafka.clients.consumer.ConsumerConfig [] - The configuration ‘key.serializer’ was supplied but isn’t a known config.
2023-11-16 18:21:20,923 INFO org.apache.kafka.common.utils.AppInfoParser [] - Kafka version: 2.4.1
2023-11-16 18:21:20,923 INFO org.apache.kafka.common.utils.AppInfoParser [] - Kafka commitId: c57222ae8cd7866b
2023-11-16 18:21:20,923 INFO org.apache.kafka.common.utils.AppInfoParser [] - Kafka startTimeMs: 1700130080923
2023-11-16 18:21:20,926 ERROR com.starrocks.data.load.stream.TransactionStreamLoader [] - Transaction prepare failed, db: dwd, table: dwd_io_cntr_transfer_real, label: 881afb56-4c43-4408-9c08-ff9915ca40d1,
responseBody: {
“Status”: “INTERNAL_ERROR”,
“Message”: “too many filtered rows”,
“ErrorURL”: “http://xxxx:8040/api/_load_error_log?file=error_log_144bca50e15393be_f5109d8d0d1becb2”
}
errorLog: Error: The row create partition failed since Runtime error: txn_id=11316990 failed. Text ‘�9�3-11-24 00:00:00’ could not be parsed at index 0… Row: [xxxxxxx, 1108354493839052800, ‘1’, ‘2’, ‘100’, 6, 2, 2023-11-15 00:10:24.238000, ‘xxxx’, 24355956, ‘1108354493839052800’, ‘1’, ‘8658229854’, ‘1700007024238’, 18.488600000000000000, 18.488600000000000000, ‘8658229854’, 1, ‘1’, 1, ‘upex_assets’, ‘mysql-bin-changelog.021027’, ‘assets_transfer_info’, ‘88230960’, ‘c’, ‘1699978224000’]
Error: The row create partition failed since Runtime error: txn_id=11316990 fai
2023-11-16 18:21:20,926 ERROR com.starrocks.connector.flink.manager.TransactionTableRegion [] - TransactionTableRegion commit failed, db: dwd, table: dwd_io_cntr_transfer_real, label: 881afb56-4c43-4408-9c08-ff9915ca40d1
com.starrocks.data.load.stream.exception.StreamLoadFailException: Failed to prepare transaction, please check taskmanager log for details, Transaction{database=‘dwd’, table=‘dwd_io_cntr_transfer_real’, label=‘881afb56-4c43-4408-9c08-ff9915ca40d1’, finish=false}
at com.starrocks.connector.flink.manager.TransactionTableRegion.commit(TransactionTableRegion.java:254) ~[flink_starrocks_20231104024049.jar:?]
at com.starrocks.connector.flink.manager.StarRocksSinkManagerV2.lambda$init$0(StarRocksSinkManagerV2.java:142) ~[flink_starrocks_20231104024049.jar:?]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_362]
2023-11-16 18:21:20,927 ERROR com.starrocks.connector.flink.manager.StarRocksSinkManagerV2 [] - Stream load failed
com.starrocks.data.load.stream.exception.StreamLoadFailException: Failed to prepare transaction, please check taskmanager log for details, Transaction{database=‘dwd’, table=‘dwd_io_cntr_transfer_real’, label=‘881afb56-4c43-4408-9c08-ff9915ca40d1’, finish=false}
at com.starrocks.connector.flink.manager.TransactionTableRegion.commit(TransactionTableRegion.java:254) ~[flink_starrocks_20231104024049.jar:?]
at com.starrocks.connector.flink.manager.StarRocksSinkManagerV2.lambda$init$0(StarRocksSinkManagerV2.java:142) ~[flink_starrocks_20231104024049.jar:?]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_362]
2023-11-16 18:21:20,927 INFO org.apache.kafka.clients.Metadata [] - [Consumer clientId=consumer-dwd_io_info_deail_real_kafak_2SR-103, groupId=dwd_io_info_deail_real_kafak_2SR] Cluster ID: kp_SyONSRdm-uPF880QMxQ
2023-11-16 18:21:20,927 ERROR com.starrocks.connector.flink.manager.StarRocksSinkManagerV2 [] - catch exception, wait rollback
com.starrocks.data.load.stream.exception.StreamLoadFailException: Failed to prepare transaction, please check taskmanager log for details, Transaction{database=‘dwd’, table=‘dwd_io_cntr_transfer_real’, label=‘881afb56-4c43-4408-9c08-ff9915ca40d1’, finish=false}
at com.starrocks.connector.flink.manager.TransactionTableRegion.commit(TransactionTableRegion.java:254) ~[flink_starrocks_20231104024049.jar:?]
at com.starrocks.connector.flink.manager.StarRocksSinkManagerV2.lambda$init$0(StarRocksSinkManagerV2.java:142) ~[flink_starrocks_20231104024049.jar:?]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_362]
