【Stream Load】报错Label Already Exists

💬 StarRocks 用户问答 数据导入与导出
#1

为了更快的定位您的问题，请提供以下信息，谢谢
【详述】通过Stream Load 功能流式向StarRocks导入数据，但是偶尔出现Label Already Exists的返回结果，但该label可以确认没用过，因为代码随机生成的label格式为{库名称}{表名称}{毫秒值时间戳}_{随机UUID}， 通过日志查看改label对应的为之前已经导入，但是由于publish 耗时较长，接近1min，导致后续二次重放改label，报错Label Already Exists; 从日志查看明显为commit时间过长，所以需要确认以下情况，

    1. 该场景下数据是否已经写入成功
    1. commit为何偶然性耗时超长
      【背景】kafak流式stream Load
      【业务影响】
      【是否存算分离】否
      【StarRocks版本】3.3.3
      【集群规模】例如：5fe（1 leader + 4 follower）+8be（fe与be混部）
      【机器信息】CPU虚拟核/内存/网卡，例如：48C/64G/万兆
      【表模型】主键模型
      【导入或者导出方式】Java Api stream Load
      【联系方式】为了在解决问题过程中能及时联系到您获取一些日志信息，请补充下您的联系方式，例如：社区群4-小李或者邮箱，谢谢
      【附件】
  • fe.log/be.INFO/相应截图
  • 完整的报错异常栈
  • stream load返回结果
{
    "TxnId": -1,
    "Label": "bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f",
    "Status": "Label Already Exists",
    "ExistingJobStatus": "FINISHED",
    "Message": "Label [bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f] has already been used.",
    "NumberTotalRows": 0,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 0,
    "LoadTimeMs": 0,
    "BeginTxnTimeMs": 0,
    "StreamLoadPlanTimeMs": 0,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 0, 
    "CommitAndPublishTimeMs": 0
}
  • 相关日志label截图
Jan 12, 2025 @ 04:43:05.469	doris16.data	140513046779648 stream_load_executor.cpp:193] begin transaction failed, errmsg=Label [bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f] has already been used.id=00410a95f6f5f7d5-46e07ed24f1b1d95, job_id=-1, txn_id: -1, label=bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db=bi
Jan 12, 2025 @ 04:43:03.693	doris16.data	140513046779648 stream_load.cpp:243] new income streaming load request.id=00410a95f6f5f7d5-46e07ed24f1b1d95, job_id=-1, txn_id: -1, label=bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db=bi, db=bi, tbl=t_onlinereport_mid_pri
Jan 12, 2025 @ 04:43:03.693	doris16.data	140513046779648 stream_load_executor.cpp:193] begin transaction failed, errmsg=Label [bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f] has already been used.id=00410a95f6f5f7d5-46e07ed24f1b1d95, job_id=-1, txn_id: -1, label=bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db=bi
Jan 12, 2025 @ 04:43:02.678	doris16.data	(thrift-server-pool-1095522|1158281) [DatabaseTransactionMgr.beginTransaction():185] begin transaction: txn_id: 22272877 with label bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f from coordinator BE: xx.xx.xx.ip1, listner id: -1
Jan 12, 2025 @ 04:43:02.678	doris16.data	(thrift-server-pool-1095522|1158281) [FrontendServiceImpl.loadTxnBegin():1187] receive txn begin request, db: bi, tbl: t_onlinereport_mid_pri, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, backend: xx.xx.xx.ip1
Jan 12, 2025 @ 04:42:50.830	doris01.data	(replayer|136) [DatabaseTransactionMgr.replayUpsertTransactionState():1775] replay a visible transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: 1736628170337, write cost: 208ms, publish total cost: 50546ms, total cost: 50754ms, reason: , newFinish attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@7441037f
Jan 12, 2025 @ 04:42:50.794	doris15.data	(replayer|142) [DatabaseTransactionMgr.replayUpsertTransactionState():1775] replay a visible transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: 1736628170337, write cost: 208ms, publish total cost: 50546ms, total cost: 50754ms, reason: , newFinish attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@218bb2b1
Jan 12, 2025 @ 04:42:50.663	doris02.data	(replayer|140) [DatabaseTransactionMgr.replayUpsertTransactionState():1775] replay a visible transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: 1736628170337, write cost: 208ms, publish total cost: 50546ms, total cost: 50754ms, reason: , newFinish attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@3800dd9b
Jan 12, 2025 @ 04:42:50.599	doris03.data	(replayer|142) [DatabaseTransactionMgr.replayUpsertTransactionState():1775] replay a visible transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: 1736628170337, write cost: 208ms, publish total cost: 50546ms, total cost: 50754ms, reason: , newFinish attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@7fc9e302
Jan 12, 2025 @ 04:42:50.591	doris16.data	(PUBLISH_VERSION|35) [DatabaseTransactionMgr.finishTransactionNew():1896] finish transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: 1736628119786, allow commit time: -1, commit time: 1736628119791, finish time: 1736628170337, write cost: 208ms, wait for publish cost: 8ms, publish rpc cost: 50537ms, finish txn cost: 1ms, publish total cost: 50546ms, total cost: 50754ms, reason: , newFinish attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@1f63b96d tabletCommitInfos size: 18 successfully
Jan 12, 2025 @ 04:42:00.634	doris15.data	(replayer|142) [DatabaseTransactionMgr.replayUpsertTransactionState():1772] replay a committed transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: -1, write cost: 208ms, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@e0ef981
Jan 12, 2025 @ 04:42:00.613	doris01.data	(replayer|136) [DatabaseTransactionMgr.replayUpsertTransactionState():1772] replay a committed transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: -1, write cost: 208ms, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@6664aba
Jan 12, 2025 @ 04:42:00.414	doris02.data	(replayer|140) [DatabaseTransactionMgr.replayUpsertTransactionState():1772] replay a committed transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: -1, write cost: 208ms, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@4c7e6089
Jan 12, 2025 @ 04:42:00.380	doris03.data	(replayer|142) [DatabaseTransactionMgr.replayUpsertTransactionState():1772] replay a committed transaction TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: -1, allow commit time: -1, commit time: 1736628119791, finish time: -1, write cost: 208ms, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@6d2fbae3
Jan 12, 2025 @ 04:42:00.277	doris16.data	(thrift-server-pool-1095249|1157977) [DatabaseTransactionMgr.commitPreparedTransaction():467] transaction:[TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: 1736628119786, allow commit time: -1, commit time: 1736628119791, finish time: -1, write cost: 208ms, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@1f63b96d tabletCommitInfos size: 18] successfully committed
Jan 12, 2025 @ 04:42:00.277	doris16.data	(thrift-server-pool-1095249|1157977) [DatabaseTransactionMgr.prepareTransaction():367] transaction:[TransactionState. txn_id: 22272480, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db id: 83043, table id list: 439012, callback id: -1, coordinator: BE: xx.xx.xx.ip2, transaction status: PREPARED, error replicas num: 0, replica ids: , prepare time: 1736628119583, write end time: 1736628119786, allow commit time: -1, commit time: -1, finish time: -1, reason:  attachment: com.starrocks.load.loadv2.ManualLoadTxnCommitAttachment@1f63b96d tabletCommitInfos size: 18] successfully prepare
Jan 12, 2025 @ 04:42:00.272	doris16.data	(thrift-server-pool-1095306|1158034) [DatabaseTransactionMgr.beginTransaction():185] begin transaction: txn_id: 22272480 with label bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f from coordinator BE: xx.xx.xx.ip2, listner id: -1
Jan 12, 2025 @ 04:42:00.272	doris16.data	(thrift-server-pool-1095306|1158034) [FrontendServiceImpl.loadTxnBegin():1187] receive txn begin request, db: bi, tbl: t_onlinereport_mid_pri, label: bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, backend: xx.xx.xx.ip2
Jan 12, 2025 @ 04:41:59.923	doris15.data	140437878204160 stream_load.cpp:243] new income streaming load request.id=bb406c9d230366f5-d5c1d8ae1d6a3b99, job_id=-1, txn_id: -1, label=bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, db=bi, db=bi, tbl=t_onlinereport_mid_pri
Jan 12, 2025 @ 04:41:59.923	doris15.data	140437878204160 stream_load_executor.cpp:77] begin to execute job. label=bi_t_onlinereport_mid_pri_1815200507_065dc2d3851f4531a7dbc2a4b7697c6f, txn_id: 22272480, query_id=bb406c9d-2303-66f5-d5c1-d8ae1d6a3b99
  • 集群监控

京ICP备2022026421号-1