【详述】broker load WHERE predicate 不生效
【背景】无
【业务影响】
【StarRocks版本】例如：1.19.5
【集群规模】3fe（1 follower+2observer）+5be（fe与be混部）
【机器信息】
【附件】
创建hive表：
CREATE TABLE IF NOT EXISTS car_status (
did int COMMENT “”,
event_time int COMMENT “”,
load_weight int COMMENT “”,
speed float COMMENT “”,
voltage float COMMENT “”
)
STORED AS PARQUET;
insert into car_status values(null,15374686,0,13.0,475.0)
创建starRocks表：
CREATE TABLE IF NOT EXISTS car_status (
did int(11) NOT NULL COMMENT “”,
event_time bigint(20) NOT NULL COMMENT “”,
load_weight int(10) NULL COMMENT “”,
speed float NULL COMMENT “”,
voltage float NULL COMMENT “”
)
primary KEY(did, event_time)
DISTRIBUTED BY HASH(did) BUCKETS 10;
load数据：
LOAD LABEL car_statusffffddda ( DATA INFILE(“hdfs://nameservice1/user/hive/warehouse/dc_test.db/car_status_ex/*”) INTO TABLE car_status COLUMNS TERMINATED BY “\x01” FORMAT AS “parquet”(did,event_time,load_weight,speed,voltage) set(did=did,event_time=event_time,load_weight=load_weight,speed=speed,voltage=voltage) where did is not null ) WITH BROKER ‘hdfs_broker’ ( “dfs.client.failover.proxy.provider.nameservice1”= “org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider”, “dfs.ha.namenoodes.nameservice1”= “namenode245,namenode269”, “dfs.namenode.rpc-address.nameservice1.namenode245”= “192.168.0.3:8020”, “dfs.namenode.rpc-address.nameservice1.namenode269”== “192.168.0.5:8020”, “dfs.nameservices”= “nameservice1” ) PROPERTIES ( “timeout” = “3600”, “max_filter_ratio”=“0.0” );
ErrorMsg: type:LOAD_RUN_FAIL; msg:all partitions have no load data
where did!=‘null’ -----> ETL_RUN_FAIL; msg:‘null’ is not a number
where did!=null -----> LOAD_RUN_FAIL; msg:all partitions have no load data
不要where ------> 导入了null的那条数据 ，转换成了0
数据格式parquet,