背景:Broker Load作业出现异常canceled
type:ETL_QUALITY_UNSATISFIED; msg:quality not good enough to cancel
这个报错指的是, 数据质量有问题,引发数据导入过程中出现Error,原因比较多,比如字段长度不足,分区不存在等。
详细:show load from xxx where label=xxxx
JobId: 11781086
Label: ads_user_sams_ptlc_bi_label_info_dev_p2023070608
State: CANCELLED
Progress: ETL:N/A; LOAD:N/A
Type: BROKER
EtlInfo: unselected.rows=0; dpp.abnorm.ALL=13626444; dpp.norm.ALL=0
TaskInfo: resource:N/A; timeout(s):14400; max_filter_ratio:0.0
ErrorMsg: type:ETL_QUALITY_UNSATISFIED; msg:quality not good enough to cancel
CreateTime: 2023-07-06 05:31:22
EtlStartTime: 2023-07-06 05:31:26
EtlFinishTime: 2023-07-06 05:31:26
LoadStartTime: 2023-07-06 05:31:26
LoadFinishTime: 2023-07-06 05:31:28
URL: http://10.233.51.136:8040/api/_load_error_log?file=error_log_8d400c57399e45b5_9ed7b351311540cb
JobDetails: {“Unfinished backends”:{“8d400c57-399e-45b5-9ed7-b351311540ca”:[]},“ScannedRows”:13626444,“TaskNumber”:1,“All backends”:{“8d400c57-399e-45b5-9ed7-b351311540ca”:[521069]},“FileNumber”:1,“FileSize”:41310685}
只需要拿到报错信息中的URL进行访问,我们就能看到原因
URL: http://10.233.51.136:8040/api/_load_error_log?file=error_log_8d400c57399e45b5_9ed7b351311540cb
[svccndlopsns@oser406436 ~]$ /bin/curl -s http://10.233.51.136:8040/api/_load_error_log?file=error_log_8d400c57399e45b5_9ed7b351311540cb
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810015694’, ‘2021-07-05’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810031162’, ‘2021-07-05’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810046940’, ‘2021-07-06’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810062344’, ‘2021-07-07’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810078490’, ‘2021-07-07’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810093001’, ‘2021-07-08’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810108845’, ‘2021-07-08’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810121452’, ‘2021-07-09’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810136811’, ‘2021-07-09’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810152833’, ‘2021-07-10’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810171126’, ‘2021-07-10’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810189896’, ‘2021-07-10’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810206464’, ‘2021-07-11’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810224764’, ‘2021-07-11’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810241745’, ‘2021-07-12’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810257903’, ‘2021-07-12’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810273791’, ‘2021-07-12’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810294763’, ‘2021-07-13’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810310486’, ‘2021-07-13’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810325845’, ‘2021-07-14’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810341583’, ‘2021-07-14’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810357037’, ‘2021-07-15’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810372899’, ‘2021-07-15’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810391489’, ‘2021-07-16’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810415343’, ‘2021-07-16’, 2023-07-06 18:13:06, 20143]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘181810433887’, ‘2021-07-17’,
我们可以看到,该作业导致CANCELLED的原因是,分区不存在。解决办法是新增分区。