【详述】问题详细描述
routine load 实时任务碰到ddl语句任务挂掉
ddl语句data是null值,routine任务就直接挂掉了,在重启后,发现这个时间段的数据有丢失
【背景】做过哪些操作?
【业务影响】
【StarRocks版本】2.0.7
【集群规模3fe+6be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡 48C/256G/万兆
routine load 建任务语句如下
CREATE ROUTINE LOAD ods.routine_ods_receipt_tb_pay_receipt_txn ON ods_receipt_tb_pay_receipt_txn_ri
COLUMNS (
sharding_time=date_format(create_time,’%Y-%m-%d %H:%i:%S’),
trade_master_no,order_id,merchant_id,create_time
)
PROPERTIES
(
“desired_concurrent_number”=“10”,
“max_batch_interval” = “60”,
“max_batch_rows” = “300000”,
“max_batch_size” = “209715200”,
“strict_mode” = “false”,
“format” = “json”,
“strip_outer_array”=“true”,
“json_root”="$.data"
)
FROM KAFKA
(
“kafka_broker_list”= “kafka:9092”,
“kafka_topic” = “doirs_topic_ods_receipt_tb_pay_receipt_txn”,
“property.group.id” = “doirs_group_ods_receipt_tb_pay_receipt_txn”,
“property.kafka_default_offsets” = “OFFSET_BEGINNING”
);