为了更快的定位您的问题,请提供以下信息,谢谢
【详述】在聚合模型的基础上构建一个的同步物化视图不刷新。
【背景】创建的聚合表,聚合表上创建sum 和 bitmap_union的物化视图,查询基表和物化视图表,发现命中物化的sum结果更新了,但是bitmap结果没有更新。不清楚是不是3.2.6不支持聚合模型bitmap。
【业务影响】数据不刷新
【是否存算分离】否
【StarRocks版本】例如:3.2.6
【集群规模】例如:3fe(1 follower+2observer)+6be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【表模型】例如:聚合模型
【导入或者导出方式】例如:Flink 1.12
【联系方式】
– 创建表
CREATE TABLE rullup_test
(
dt
date NOT NULL COMMENT “”,
hour
tinyint(4) NOT NULL COMMENT “”,
app
varchar(255) NOT NULL COMMENT “”,
country
varchar(300) NULL COMMENT “”,
os
varchar(300) NULL COMMENT “”,
cuid_uv
bitmap BITMAP_UNION NULL COMMENT “”,
uid_uv
bitmap BITMAP_UNION NULL COMMENT “”,
pv_col
bigint(20) SUM NULL DEFAULT “0” COMMENT “”
) ENGINE=OLAP
AGGREGATE KEY(dt
, hour
, app
, country
, os
)
COMMENT “OLAP”
PARTITION BY RANGE(dt
)
(PARTITION p20240801 VALUES [(“2024-08-01”), (“2024-08-01”)),
PARTITION p20240802 VALUES [(“2024-08-02”), (“2024-08-02”)))
DISTRIBUTED BY HASH(hour
) BUCKETS 1
PROPERTIES (
“replication_num” = “3”,
“dynamic_partition.enable” = “true”,
“dynamic_partition.time_unit” = “DAY”,
“dynamic_partition.time_zone” = “Asia/Shanghai”,
“dynamic_partition.start” = “-60”,
“dynamic_partition.end” = “3”,
“dynamic_partition.prefix” = “p”,
“dynamic_partition.buckets” = “1”,
“dynamic_partition.history_partition_num” = “0”,
“in_memory” = “false”,
“enable_persistent_index” = “false”,
“replicated_storage” = “true”,
“compression” = “LZ4”
);
– 创建物化视图
CREATE MATERIALIZED VIEW rullup_test_rullup AS
SELECT dt,hour,bitmap_union(cuid_uv),sum(pv_col)
FROM rullup_test
GROUP BY dt,hour;
– flink 1.12 写入数据
– 结果
命中物化视图sql – SELECT dt,hour,bitmap_union_count(cuid_uv),sum(pv_col) FROM rull_test where dt = ‘20240802’ GROUP BY dt,hour order by hour;
±-----------±-----±----------------------------±------------+
| dt | hour | bitmap_union_count(cuid_uv) | sum(pv_col) |
±-----------±-----±----------------------------±------------+
| 2024-08-02 | 15 | 0 | 17145839 |
| 2024-08-02 | 16 | 0 | 87 |
±-----------±-----±----------------------------±------------+
不命中物化视图sql – SELECT count(),dt,hour,bitmap_union_count(cuid_uv),sum(pv_col) FROM rull_test where dt = ‘20240802’ GROUP BY dt,hour order by hour;
±---------±-----------±-----±----------------------------±------------+
| count() | dt | hour | bitmap_union_count(cuid_uv) | sum(pv_col) |
±---------±-----------±-----±----------------------------±------------+
| 328 | 2024-08-02 | 15 | 166378 | 17145839 |
| 8 | 2024-08-02 | 16 | 12 | 87 |
±---------±-----------±-----±----------------------------±------------+
发现bitmap_union_count(cuid_uv) 计算结果明显不同,但是sum(pv_col) 列值是一样的。