【是否存算分离】否
【StarRocks版本】3.5.19
【集群规模】例如：docker 安装 starrocks/allin1-ubuntu:3.5.19
【联系方式】微信号：wufanwind
【背景】创建 分区异步物化视图，定时查询paimon的agg模型的分区表，进行分区增量更新物化视图结果。但是只有首次全量分区更新可以成功，后面的增量分区更新都失败。
【详述】paimon 版本 ： 1.4.1
1、paimon的分区表建表语句：
CREATE TABLE etl.agg_good_cnt_day (
stat_dt STRING NOT NULL,
stat_hour STRING NOT NULL,
amount DECIMAL ( 18, 2 ),
cnt BIGINT,
PRIMARY KEY ( stat_dt, stat_hour ) NOT ENFORCED
) PARTITIONED BY ( stat_dt ) WITH (
‘fields.amount.aggregate-function’ = ‘sum’,
‘merge-engine’ = ‘aggregation’,
‘fields.cnt.aggregate-function’ = ‘sum’,
‘changelog-producer’ = ‘lookup’
)
2、sr的物化视图语句
CREATE MATERIALIZED VIEW bbb
REFRESH ASYNC EVERY(INTERVAL 1 minute)
PARTITION BY str2date(stat_dt, ‘%Y-%m-%d’)
PROPERTIES (
“replication_num” = “1”
)
AS
select stat_dt,sum(amount) from paimon_catalog.etl.agg_good_cnt_day
group by stat_dt;
3、查询物化视图运行记录，是FAILED状态
select * from information_schema.task_runs where task_name=‘mv-36721’ order by CREATE_TIME desc;
load_tracking_logs报错内容:
Error: The row is out of partition ranges. Please add a new partition… Row: [‘2006-02-28’, 390.00]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘2008-08-17’, 206.00]
Error: The row is out of partition ranges. Please add a new partition… Row: [‘2026-07-17’, 111.00]
… 等等
EXTRA_MESSAGE字段内容如下：
{
“forceRefresh”: false,
“mvPartitionsToRefresh”: [“p20040922_20040923”],
“refBasePartitionsToRefreshMap”: {“agg_good_cnt_day”: [“p20040922”]},
“basePartitionsToRefreshMap”: {},
“processStartTime”: 1784284675457,
“executeOption”: {
“priority”: 0,
“taskRunProperties”: {“warehouse”: “default_warehouse”, “mvId”: “42696”},
“isMergeRedundant”: true,
“isManual”: false,
“isSync”: false,
“isReplay”: false
},
“planBuilderMessage”: {“agg_good_cnt_day”: “(stat_dt >= ‘2004-09-22’) AND (stat_dt < ‘2004-09-23’)”}
}
4、排查 bbb分区 ，是存在这个分区的
SHOW PARTITIONS FROM bbb;
PartitionName PartitionKey
p20040922_20040923 stat_dt
【个人猜想问题原因】
sr的异步物化视图任务，正确识别到了分区’2004-09-22’有数据变化，所以开始’2004-09-22’这个分区的数据刷新，但是分区条件谓词下推和paimon 这个版本的这种模型表不兼容，导致把所有分区的数据都查询出来，无法和对应分区匹配。