为了更快的定位您的问题，请提供以下信息，谢谢
【详述】flink写入聚合模型HLL类型时报错,关于为null的字段已经过滤掉了，还是报错
【背景】写入聚合表，字段类型为HLL
【业务影响】
【StarRocks版本】2.5
【集群规模】3fe
【机器信息】略
【表模型】明细模型
【导入或者导出方式】Flink
【联系方式】764208769@qq.com
【附件】
=================版本
flink 1.17
db: 2.5
=================报错信息
Caused by: java.lang.RuntimeException: com.starrocks.data.load.stream.exception.StreamLoadFailException: Stream load failed because of error, db:, table: xxxx, label: 67fca88f-24b0-4f5b-bb43-463f41ed01d4,
responseBody: {
“TxnId”: 108814040,
“Label”: “67fca88f-24b0-4f5b-bb43-463f41ed01d4”,
“Status”: “Fail”,
“Message”: “unknown reference column, column=member_id, reference=mid”,
“NumberTotalRows”: 0,
“NumberLoadedRows”: 0,
“NumberFilteredRows”: 0,
“NumberUnselectedRows”: 0,
“LoadBytes”: 0,
“LoadTimeMs”: 0,
“BeginTxnTimeMs”: 0,
“StreamLoadPutTimeMs”: 0,
“ReadDataTimeMs”: 0,
“WriteDataTimeMs”: 0,
“CommitAndPublishTimeMs”: 0
}
errorLog: null
at com.starrocks.connector.flink.manager.StarRocksSinkManagerV2.AssertNotException(StarRocksSinkManagerV2.java:367)
at com.starrocks.connector.flink.manager.StarRocksSinkManagerV2.flush(StarRocksSinkManagerV2.java:299)
at com.starrocks.connector.flink.table.sink.StarRocksDynamicSinkFunctionV2.snapshotState(StarRocksDynamicSinkFunctionV2.java:197)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:88)
at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:222)
… 25 more
[CIRCULAR REFERENCE:com.starrocks.data.load.stream.exception.StreamLoadFailException: Stream load failed because of error, db: xxx, table: xxx, label: 67fca88f-24b0-4f5b-bb43-463f41ed01d4,
=================表字段
session_id hll HLL_UNION NOT NULL ,
union_id hll HLL_UNION NOT NULL ,
member_id hll HLL_UNION NOT NULL ,
device_id hll HLL_UNION NOT NULL,
=================代码配置信息
.withProperty(“sink.properties.columns”,
“省略无关字段session_id=hll_hash(sid),union_id=hll_hash(uid),member_id=hll_hash(mid),device_id=hll_hash(did)”)
=================模型信息
尝试过的模型v1
@JSONField(name = “session_id”)
private String sessionId;
@JSONField(name = “union_id”)
private String unionId;
@JSONField(name = “member_id”)
private String memberId;
@JSONField(name = “device_id”)
private String deviceId;
尝试过的模型v2
@JSONField(name = "session_id")
private Long sessionId;
@JSONField(name = "union_id")
private Long unionId;
@JSONField(name = "member_id")
private Long memberId;
@JSONField(name = "device_id")
private Long deviceId;
@JSONField(name = "sid")
private String sid;
@JSONField(name = "uid")
private String uid;
@JSONField(name = "mid")
private String mid;
@JSONField(name = "did")
private String did;