【详述】问题详细描述
目标表定义：
CREATE TABLE ods_t_record (
id int(11) NOT NULL COMMENT “id”,
type int(11) NOT NULL COMMENT “操作记录类型”,
lock_id int(11) NOT NULL COMMENT “锁ID”,
operate_date bigint(20) NOT NULL COMMENT “操作锁的时间”,
number varchar(50) NULL COMMENT “密码 或 卡号 或 指纹号”,
new_passcode varchar(50) NULL COMMENT “修改后的密码”,
key_id int(11) NULL COMMENT “电子钥匙ID”,
app_unlock_user_id int(11) NULL COMMENT “电子钥匙开锁的用户ID”,
app_unlock_user_name varchar(50) NULL COMMENT “电子钥匙开锁的用户姓名”,
app_unlock_unique_id int(11) NULL COMMENT “电子钥匙开锁唯一标识”,
clear_passcode_date bigint(20) NULL COMMENT “只有type-8 键盘上删除所有密码会有该日期”,
create_date varchar(64) NULL COMMENT “创建时间”
) ENGINE=OLAP
PRIMARY KEY(id)
COMMENT “锁记录表”
DISTRIBUTED BY HASH(id) BUCKETS 4
PROPERTIES (
“replication_num” = “2”,
“in_memory” = “false”,
“storage_format” = “DEFAULT”
);
DataX导入异常日志如下：
Caused by: java.io.IOException: Failed to flush data to StarRocks.
{“Status”:“Fail”,“BeginTxnTimeMs”:0,“Message”:"/opt/StarRocks-2.2.2/be/data/data/8/11177/326739431/0200000000041e92f6450026da6de98e64c810e541f90bbc_0.dat.tmp: No such file or directory",“NumberUnselectedRows”:0,“CommitAndPublishTimeMs”:0,“Label”:“91412cf6-99cb-411a-949f-71a54812c944”,“LoadBytes”:87243,“StreamLoadPutTimeMs”:1,“NumberTotalRows”:0,“WriteDataTimeMs”:23,“TxnId”:144486,“LoadTimeMs”:26,“ReadDataTimeMs”:0,“NumberLoadedRows”:0,“NumberFilteredRows”:0}
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor.doStreamLoad(StarRocksStreamLoadVisitor.java:73)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.asyncFlush(StarRocksWriterManager.java:161)
… 3 more
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.StarRocksWriter$Task.post(StarRocksWriter.java:134)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:65)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Writing records to StarRocks failed.
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.close(StarRocksWriterManager.java:116)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.StarRocksWriter$Task.post(StarRocksWriter.java:132)
… 2 more
Caused by: java.lang.RuntimeException: Writing records to StarRocks failed.
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.checkFlushException(StarRocksWriterManager.java:187)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.waitAsyncFlushingDone(StarRocksWriterManager.java:148)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.flush(StarRocksWriterManager.java:101)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.close(StarRocksWriterManager.java:114)
… 3 more
Caused by: java.io.IOException: java.io.IOException: Failed to flush data to StarRocks.
{“Status”:“Fail”,“BeginTxnTimeMs”:0,“Message”:"/opt/StarRocks-2.2.2/be/data/data/8/11177/326739431/0200000000041e92f6450026da6de98e64c810e541f90bbc_0.dat.tmp: No such file or directory",“NumberUnselectedRows”:0,“CommitAndPublishTimeMs”:0,“Label”:“91412cf6-99cb-411a-949f-71a54812c944”,“LoadBytes”:87243,“StreamLoadPutTimeMs”:1,“NumberTotalRows”:0,“WriteDataTimeMs”:23,“TxnId”:144486,“LoadTimeMs”:26,“ReadDataTimeMs”:0,“NumberLoadedRows”:0,“NumberFilteredRows”:0}
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.asyncFlush(StarRocksWriterManager.java:168)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.access$000(StarRocksWriterManager.java:21)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager$1.run(StarRocksWriterManager.java:132)
… 1 more
Caused by: java.io.IOException: Failed to flush data to StarRocks.
{“Status”:“Fail”,“BeginTxnTimeMs”:0,“Message”:"/opt/StarRocks-2.2.2/be/data/data/8/11177/326739431/0200000000041e92f6450026da6de98e64c810e541f90bbc_0.dat.tmp: No such file or directory",“NumberUnselectedRows”:0,“CommitAndPublishTimeMs”:0,“Label”:“91412cf6-99cb-411a-949f-71a54812c944”,“LoadBytes”:87243,“StreamLoadPutTimeMs”:1,“NumberTotalRows”:0,“WriteDataTimeMs”:23,“TxnId”:144486,“LoadTimeMs”:26,“ReadDataTimeMs”:0,“NumberLoadedRows”:0,“NumberFilteredRows”:0}
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor.doStreamLoad(StarRocksStreamLoadVisitor.java:73)
at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.asyncFlush(StarRocksWriterManager.java:161)
… 3 more
【导入/导出方式】
DataX
【背景】做过哪些操作？
目标表在starrocks中数据量不大，只有几千条数据，之前是可以导入成功过的，突然异常了。
【业务影响】
【StarRocks版本】例如：2.2.2
【集群规模】例如：3fe+3be（fe与be混部）
【机器信息】CPU虚拟核/内存/网卡，fe 8G， be 16G
【附件】