【详述】datax 导入数据到一定量的时候报错
【导入/导出方式】导入使用datax
【StarRocks版本】例如:1.19.3
【集群规模】例如:3fe(1 follower+2observer)+3be(fe与be混部)
【机器信息】8核 16cpu 32G内存
【附件】
Caused by: java.io.IOException: java.io.IOException: Failed to flush data to doris.
{“Status”:“Fail”,“BeginTxnTimeMs”:0,“Message”:“cancelled”,“NumberUnselectedRows”:0,“CommitAndPublishTimeMs”:0,“Label”:“f8bcffc9-86e7-43f2-ad2b-ad6005e65a7d”,“LoadBytes”:9433120,“StreamLoadPutTimeMs”:0,“NumberTotalRows”:0,“WriteDataTimeMs”:42,“TxnId”:54219,“LoadTimeMs”:120,“ReadDataTimeMs”:13,“NumberLoadedRows”:0,“NumberFilteredRows”:0}
at com.dorisdb.connector.datax.plugin.writer.doriswriter.manager.DorisWriterManager.asyncFlush(DorisWriterManager.java:127)
at com.dorisdb.connector.datax.plugin.writer.doriswriter.manager.DorisWriterManager.access$000(DorisWriterManager.java:15)
at com.dorisdb.connector.datax.plugin.writer.doriswriter.manager.DorisWriterManager$1.run(DorisWriterManager.java:94)
… 1 more
BE日志:
2022-02-24 12:38:47,396 INFO (replayer|72) [Catalog.replayJournal():2440] replayed journal id is 2230113, replay to journal id is 2230114
2022-02-24 12:38:47,396 INFO (replayer|72) [DatabaseTransactionMgr.replayUpsertTransactionState():1424] replay a visible transaction TransactionState. transaction id: 54202, label: 4388dc51-e244-4e1a-bcef-08865a10ccd9, db id: 11080, table id list: 12107, callback id: -1, coordinator: BE: xx.xx.xx.xx, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1645706325598, commit time: 1645706327378, finish time: 1645706327392, reason: attactment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@50d37b7c
2022-02-24 12:38:48,307 INFO (nioEventLoopGroup-3-21|107) [RestBaseAction.handleRequest():55] receive http request. url=/api/bootstrap?cluster_id=443812431&token=5ead8de4-4c68-4d2f-ad24-c1906a117227
数据表大小(导入的是v2表):