基于物化视图的物化视图,异步定时刷新时,是不是数据每次都是全量覆盖?

基于物化视图的物化视图,异步定时刷新时,是不是数据每次都是全量覆盖?SR版本2.5.1,测试环境中几张测试表,基表都没有数据更新时,一级物化视图(基于基表)也不会更新,但是二级物化视图(基于一级物化视图与基表)似乎每次都是全量更新,因为这时CPU明显很高(删掉这个二级物化视图,cpu就降下来了)

从fe的日志中,也明显看出一级物化视图与二级物化视图的处理方式不同:

方便发下建表和物化视图的创建语句么?

CREATE TABLE sales_records(
record_id INT,
seller_id INT,
store_id INT,
sale_date DATE,
sale_amt BIGINT
)
DISTRIBUTED BY HASH(record_id)
PROPERTIES (
“replication_num” = “1”
);

INSERT INTO sales_records
VALUES
(001,01,1,“2022-03-13”,8573),
(002,02,2,“2022-03-14”,6948),
(003,01,1,“2022-03-14”,4319),
(004,03,3,“2022-03-15”,8734),
(005,03,3,“2022-03-16”,4212),
(006,02,2,“2022-03-17”,9515);

CREATE MATERIALIZED VIEW mv_store_sales
DISTRIBUTED BY HASH( store_id ) BUCKETS 1
REFRESH ASYNC START(‘2023-02-14 16:52:00’) EVERY (interval 2 SECOND)
AS
select store_id, sum(sale_amt) as sum_sale_amt
from sales_records
group by store_id

CREATE MATERIALIZED VIEW mv_sales_records_all
DISTRIBUTED BY HASH( record_id ) BUCKETS 1
REFRESH ASYNC START(‘2023-02-14 16:52:00’) EVERY (interval 3 SECOND)
AS
select a.record_id, a.seller_id, a.store_id, a.sale_date, a.sale_amt, a.sale_amt / b.sum_sale_amt as per
from sales_records a
join mv_store_sales b on a.store_id = b.store_id

以上是用简单的数据,模拟出同样的效果:
2023-02-14 16:56:49,513 INFO (pool-19-thread-67|14845) [PartitionBasedMaterializedViewRefreshProcessor.processTaskRun():147] no partitions to refresh for materialized view mv_store_sales
2023-02-14 16:56:49,597 INFO (thrift-server-pool-1178|14922) [ThriftServerEventProcessor.deleteContext():112] delete thrift context. client: TNetworkAddress(hostname:172.100.91.207, port:42326)
2023-02-14 16:56:50,523 INFO (pool-19-thread-67|14845) [InsertPlanner.plan():124] property[], ANY, []
2023-02-14 16:56:50,541 INFO (pool-19-thread-67|14845) [InsertPlanner.plan():124] property[], ANY, []
2023-02-14 16:56:50,541 INFO (pool-19-thread-67|14845) [DatabaseTransactionMgr.beginTransaction():308] begin transaction: txn_id: 6385 with label insert_850d8586-ac45-11ed-9c94-04d4c4f073ef from coordinator FE: 172.100.91.207, listner id: -1
2023-02-14 16:56:50,547 INFO (pool-19-thread-67|14845) [QeProcessorImpl.registerQuery():81] register query id = 850d8586-ac45-11ed-9c94-04d4c4f073ef, job: 81615
2023-02-14 16:56:50,547 INFO (pool-19-thread-67|14845) [Coordinator.prepareResultSink():759] dispatch load job: 850d8586-ac45-11ed-9c94-04d4c4f073ef to [TNetworkAddress(hostname:172.100.91.207, port:9060)]
2023-02-14 16:56:50,551 INFO (thrift-server-pool-1181|14925) [ThriftServerEventProcessor.createContext():97] create thrift context. client: TNetworkAddress(hostname:172.100.91.207, port:42332)
2023-02-14 16:56:50,584 INFO (PUBLISH_VERSION|21) [PublishVersionDaemon.publishVersionForOlapTable():125] send publish tasks for txn_id: 6385
2023-02-14 16:56:50,584 INFO (pool-19-thread-67|14845) [DatabaseTransactionMgr.commitTransaction():479] transaction:[TransactionState. txn_id: 6385, label: insert_850d8586-ac45-11ed-9c94-04d4c4f073ef, db id: 11001, table id list: 81109, callback id: -1, coordinator: FE: 172.100.91.207, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1676365010542, commit time: 1676365010579, finish time: -1, write cost: 37ms, reason: attachment: com.starrocks.transaction.InsertTxnCommitAttachment@69f6e54a] successfully committed
2023-02-14 16:56:50,600 INFO (PUBLISH_VERSION|21) [DatabaseTransactionMgr.finishTransaction():1036] finish transaction TransactionState. txn_id: 6385, label: insert_850d8586-ac45-11ed-9c94-04d4c4f073ef, db id: 11001, table id list: 81109, callback id: -1, coordinator: FE: 172.100.91.207, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1676365010542, commit time: 1676365010579, finish time: 1676365010595, write cost: 37ms, wait for publish cost: 5ms, publish rpc cost: 6ms, finish txn cost: 5ms, publish total cost: 16ms, total cost: 53ms, reason: attachment: com.starrocks.transaction.InsertTxnCommitAttachment@69f6e54a successfully
2023-02-14 16:56:50,606 INFO (pool-19-thread-67|14845) [OlapTable.renamePartition():735] rename partition mv_sales_records_all in table mv_sales_records_all
2023-02-14 16:56:50,612 INFO (pool-19-thread-67|14845) [InsertOverwriteJobRunner.handle():122] insert overwrite job:81611 succeed. createPartitionElapse:7 ms, insertElapse:65 ms
2023-02-14 16:56:50,612 INFO (pool-19-thread-67|14845) [QeProcessorImpl.unregisterQuery():91] deregister query id 850d8586-ac45-11ed-9c94-04d4c4f073ef
2023-02-14 16:56:51,378 INFO (colocate group clone checker|75) [ColocateTableBalancer.matchGroup():436] finished to match colocate group. cost: 0 ms, in lock time: 0 ms

show tablet from mv_sales_records_all;
通过以上语句查出的tabletId,一直在变化,也应该能说明该物化视图的数据一直在更新。

有结论了吗,,,,,?

异步定时刷新的方式,当前是insert overwrite覆盖的

是全量?

当前最小粒度是分区级全量