【详述】问题描述:
datax导入数据，目标端为主键模型的表，结构如下，导入了会儿报
delta_writer.cpp:80] Update memory limit exceed tablet:53099 8130750385 > 8117488189 top_tablets:53111(1851M)53107(1851M)53103(1811M)53099(1810M)55179(28M)
W0214 17:22:22.683980 29039 internal_service.cpp:205] tablet writer add chunk failed, message=Update memory limit exceed tablet:53099 8130750385 > 8117488189 top_tablets:53111(1851M)53107(1851M)53103(1811M)53099(1810M)55179(28M), id=1a29fb2c-5043-4c5f-987e-afa34bf14154, index_id=53098, sender_id=0
I0214 17:22:22.789904 29028 load_channel_mgr.cpp:301] Cancelled load channel load id=1a29fb2c50434c5f-987eafa34bf14154
表结构如下：
CREATE TABLE u_detail (
detail_id bigint(20) NOT NULL ,
user_id varchar(32) NOT NULL ,
service_range varchar(1024) NOT NULL ,
city varchar(255) NOT NULL ,
relname char(32) NOT NULL ,
tags varchar(64) NOT NULL ,
birthday date NOT NULL DEFAULT “1980-01-01” ,
hometown varchar(128) NOT NULL ,
livetown varchar(128) NOT NULL ,
signature varchar(255) NOT NULL ,
tel char(16) NOT NULL ,
qq char(10) NOT NULL ,
weixin varchar(64) NOT NULL ,
education varchar(64) NOT NULL ,
job int(11) NOT NULL ,
corparation varchar(255) NOT NULL ,
job_position int(11) NOT NULL ,
income int(11) NOT NULL ,
certificate_type int(11) NOT NULL ,
certificate_number varchar(32) NOT NULL ,
post_address varchar(128) NOT NULL ,
zipcode varchar(16) NOT NULL ,
homepage varchar(64) NOT NULL ,
about_me varchar(128) NOT NULL ,
hobby varchar(128) NOT NULL ,
sightml varchar(256) NOT NULL ,
lng decimal64(9, 6) NOT NULL ,
lat decimal64(9, 6) NOT NULL ,
fax varchar(16) NOT NULL ,
admin_city varchar(64) NOT NULL ,
cart_cookie varchar(2000) NOT NULL DEFAULT “[]” ,
privacy_setting tinyint(4) NOT NULL DEFAULT “0” ,
app_from varchar(255) NOT NULL DEFAULT “” ,
tag_switch varchar(255) NOT NULL DEFAULT ‘’
) ENGINE=OLAP
PRIMARY KEY(u_detail_id)
COMMENT “xxxx”
DISTRIBUTED BY HASH(u_detail_id) BUCKETS 15
PROPERTIES (
“replication_num” = “3”,
“in_memory” = “false”,
“storage_format” = “DEFAULT”
);
【StarRocks版本】例如：2.0.0GA
【集群规模】例如：3fe（1 follower+2observer）+5be（fe与be混部）
【机器信息】CPU虚拟核/内存/网卡，例如：8C/16G/万兆
相关设置信息：
-
并行度：4
-
cbo是否开启：show variables like ‘%cbo%’;
±------------------------------------±------+
| Variable_name | Value |
±------------------------------------±------+
| cbo_enable_dp_join_reorder | true |
| cbo_enable_greedy_join_reorder | true |
| cbo_enable_low_cardinality_optimize | true |
| cbo_enable_replicated_join | true |
| cbo_max_reorder_node_use_dp | 10 |
| cbo_max_reorder_node_use_exhaustive | 4 |
| cbo_use_correlated_join_estimate | true |
±------------------------------------±------+ -
be节点cpu和内存使用率截图
