【Broker Load】type:LOAD_RUN_FAIL; msg:OrcScannerAdapter::read_next failed. reason = std::bad_cast

💬 StarRocks 用户问答 常见 FAQ
#1
  • 执行Broker Load出现Err，详细日志：

2023-06-13 17:30:57.147 [job-0] INFO JobContainer - DataX Reader.Job [sqlserverreader] do post work.
2023-06-13 17:30:57.147 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-06-13 17:30:57.148 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /opt/datax/hook
2023-06-13 17:30:57.149 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%

     [total gc info] => 
             NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime     
             PS MarkSweep         | 7                  | 7                  | 7                  | 0.607s             | 0.607s             | 0.607s             
             PS Scavenge          | 269                | 269                | 269                | 3.139s             | 3.139s             | 3.139s

2023-06-13 17:30:57.149 [job-0] INFO JobContainer - PerfTrace not enable!
2023-06-13 17:30:57.150 [job-0] INFO StandAloneJobContainerCommunicator - Total 10398267 records, 2251817095 bytes | Speed 9.34MB/s, 45209 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 701.166s | All Task WaitReaderTime 33.982s | Percentage 100.00%
2023-06-13 17:30:57.151 [job-0] INFO JobContainer -
任务启动时刻 : 2023-06-13 17:27:03
任务结束时刻 : 2023-06-13 17:30:57
任务总计耗时 : 234s
任务平均流量 : 9.34MB/s
记录写入速度 : 45209rec/s
读出记录总数 : 10398267
读写失败总数 : 0

2023-06-13 17:30:57.705 INFO engine/jobsync.go:325 ingest data to hdfs:[/warehouse/tablespace/external/hive/cn_raw_stg.db/o_stg_aloha_order__order_detail] success,cost:3.92(m)
2023-06-13 17:30:57.706 DEBUG engine/task_sr.go:234
LOAD LABEL ods.aloha_order__order_detail_d_ci43cpb87oaslh8jntfg_20230613T173057
(
DATA INFILE(“hdfs://cnprod1ha/warehouse/tablespace/external/hive/cn_raw_stg.db/o_stg_aloha_order__order_detail/*”)
INTO TABLE aloha_order__order_detail_d
format as “orc”
(order_detail_id,order_id,user_id,store_id,item_id,sku,product_name,upc,product_marque,department,category,unit_cost,price_with_tax,smart_price,price_without_tax,tax_rate,order_quantity,delivery_quantity,wm_discount,external_discount,discount_amount,receivable_unit_price,item_type,report_code,pos_desc,detail_line,weight,scalable,scale_package_code,product_total_amount,remark,item_extend,create_by,update_by,create_time,update_time,sda,vendor_nbr,file_line,exchange_item_id,exchange_upc,etl_load_time,exchange_type,platform_service_charge,ts)

)
WITH BROKER prdbroker
(
“hadoop.security.authentication” = “kerberos”,
“kerberos_principal” = “",
“kerberos_keytab” = "”,
“username” = “44444444”,
“dfs.nameservices” = “xxxxx”,
“dfs.ha.namenodes.xxxxx” = “nn1,nn2”,
“dfs.namenode.rpc-address.xxxx.nn1” = “xxxxx.cn.wal-mart.com:8020”,
“dfs.namenode.rpc-address.xxxxx.nn2” = “xxxxx.cn.wal-mart.com:8020”,
“dfs.client.failover.proxy.provider.cnprod1ha” = “org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider”
)
properties(“timezone” = “UTC”)
2023-06-13 17:30:58.351 INFO engine/task_sr.go:247 [adhoc][#1/3] submit broker load with timeout=300 to starRocks success,cluster=adhoc,broker=prdbroker
2023-06-13 17:31:01.489 INFO engine/engine.go:135 #[1/0/1] waitTasks:[pro_aloha_order.order_detail] doneTasks:[],cost:4.00(m)

2023-06-13 17:31:08.356 DEBUG engine/comm.go:266 {
“JobId”: “347774789”,
“Label”: “aloha_order__order_detail_d_ci43cpb87oaslh8jntfg_20230613T173057”,
“State”: “CANCELLED”,
“Progress”: “ETL:N/A; LOAD:N/A”,
“Type”: “BROKER”,
“EtlInfo”: “”,
“TaskInfo”: “resource:N/A; timeout(s):14400; max_filter_ratio:0.0”,
“ErrorMsg”: “type:LOAD_RUN_FAIL; msg:OrcScannerAdapter::read_next failed. reason = std::bad_cast”,
“CreateTime”: “2023-06-13 09:30:57”,
“EtlStartTime”: “2023-06-13 09:31:00”,
“EtlFinishTime”: “2023-06-13 09:31:00”,
“LoadStartTime”: “2023-06-13 09:31:00”,
“LoadFinishTime”: “2023-06-13 09:31:00”,
“URL”: “”,
“JobDetails”: “{“Unfinished backends”:{“2f45c218-f534-4ad9-ae54-72c8dde858f5”:[10004,262283433,10003,206433247,206433251]},“ScannedRows”:1,“TaskNumber”:1,“All backends”:{“2f45c218-f534-4ad9-ae54-72c8dde858f5”:[10004,262283433,10003,206433247,10006,206433251]},“FileNumber”:32,“FileSize”:3770808791}”
}
2023-06-13 17:31:08.356 WARN engine/comm.go:233 #1/3 show load of starRocks err:type:LOAD_RUN_FAIL; msg:OrcScannerAdapter::read_next failed. reason = std::bad_cast,will retry after 30s
2023-06-13 17:31:48.364 DEBUG engine/comm.go:266 {
“JobId”: “347774789”,
“Label”: “aloha_order__order_detail_d_ci43cpb87oaslh8jntfg_20230613T173057”,
“State”: “CANCELLED”,
“Progress”: “ETL:N/A; LOAD:N/A”,
“Type”: “BROKER”,
“EtlInfo”: “”,
“TaskInfo”: “resource:N/A; timeout(s):14400; max_filter_ratio:0.0”,
“ErrorMsg”: “type:LOAD_RUN_FAIL; msg:OrcScannerAdapter::read_next failed. reason = std::bad_cast”,
“CreateTime”: “2023-06-13 09:30:57”,
“EtlStartTime”: “2023-06-13 09:31:00”,
“EtlFinishTime”: “2023-06-13 09:31:00”,
“LoadStartTime”: “2023-06-13 09:31:00”,
“LoadFinishTime”: “2023-06-13 09:31:00”,
“URL”: “”,
“JobDetails”: “{“Unfinished backends”:{“2f45c218-f534-4ad9-ae54-72c8dde858f5”:[10004,262283433,10003,206433247,206433251]},“ScannedRows”:1,“TaskNumber”:1,“All backends”:{“2f45c218-f534-4ad9-ae54-72c8dde858f5”:[10004,262283433,10003,206433247,10006,206433251]},“FileNumber”:32,“FileSize”:3770808791}”
}
2023-06-13 17:31:48.365 WARN engine/comm.go:233 #2/3 show load of starRocks err:type:LOAD_RUN_FAIL; msg:OrcScannerAdapter::read_next failed. reason = std::bad_cast,will retry after 30s

  • 解决办法：

检查hdfs路径下是否存在orc文件，异常文件夹，删除整个路径下的文件， 重新导入

京ICP备2022026421号-1