上游mysql表数据有400多万条,写入主键模型SR表。我已将缓冲区大小增加到90M,日志显示json格式最大支持100M,请问sink.buffer-flush相关参数如何配置,才能减缓插入的速度
SR primary-key 表:
CREATE TABLE IF NOT EXISTS test.dws_yly
(
BBillNo
varchar(50) not null,
Lot
varchar(30) not null,
GoodsGuid
varchar(32) not null,
BillDate
datetime,
DepartureDate
date,
ReceiveTime
datetime,
CargoOwnerGUID
varchar(32),
OldCode
varchar(100),
CargoOwnerName
varchar(100),
SaleCorpName
varchar(100),
GoodsCode
varchar(20),
GoodsName
varchar(300),
Qty
decimal(18,6)
)
PRIMARY KEY(BBillNo
,Lot
,GoodsGuid
)
DISTRIBUTED BY HASH(BBillNo
) BUCKETS 10
PROPERTIES(“replication_num” = “1”);
连接器配置:
报错日志:
调小刷新间隔5000ms:
仍然报错: