为了更快的定位您的问题，请提供以下信息，谢谢
【详述】tablet已经删除，但是对应talet的数据文件仍然存在。有效数据文件只有1.4T左右
【StarRocks版本】3.2.6
1 数据文件只占磁盘总空间的10%:
2 经排查有大量已经删除的talet的数据文件仍然存在
3 tablet已经删除
2 经排查有大量已经删除的talet的数据文件仍然存在
3 tablet已经删除
FE日志tablet:47036351日志
2024-08-26 11:31:46.761+08:00 INFO (starrocks-taskrun-pool-1292|48089977) [LocalMetastore.buildCreateReplicaTasks():2034] build create replica tasks for index index id: 20824439; index state: NORMAL; row count: 0; tablets size: 60; visibleTxnId: 0; tablets: [tablet: id=47036315, tablet: id=47036318, tablet: id=47036321, tablet: id=47036324, tablet: id=47036327, tablet: id=47036330, tablet: id=47036333, tablet: id=47036336, tablet: id=47036339, tablet: id=47036342, tablet: id=47036345, tablet: id=47036348, tablet: id=47036351, tablet: id=47036354, tablet: id=47036357, tablet: id=47036360, tablet: id=47036363, tablet: id=47036366, tablet: id=47036369, tablet: id=47036372, tablet: id=47036375, tablet: id=47036378, tablet: id=47036381, tablet: id=47036384, tablet: id=47036387, tablet: id=47036390, tablet: id=47036393, tablet: id=47036396, tablet: id=47036399, tablet: id=47036402, tablet: id=47036405, tablet: id=47036408, tablet: id=47036411, tablet: id=47036414, tablet: id=47036417, tablet: id=47036420, tablet: id=47036423, tablet: id=47036426, tablet: id=47036429, tablet: id=47036432, tablet: id=47036435, tablet: id=47036438, tablet: id=47036441, tablet: id=47036444, tablet: id=47036447, tablet: id=47036450, tablet: id=47036453, tablet: id=47036456, tablet: id=47036459, tablet: id=47036462, tablet: id=47036465, tablet: id=47036468, tablet: id=47036471, tablet: id=47036474, tablet: id=47036477, tablet: id=47036480, tablet: id=47036483, tablet: id=47036486, tablet: id=47036489, tablet: id=47036492, ]; db 294008 table 20824438 partition partition_id: 47036314; name: p202408_47036313; partition_state.name: NORMAL; base_index: index id: 20824439; index state: NORMAL; row count: 0; tablets size: 60; visibleTxnId: 0; tablets: [tablet: id=47036315, tablet: id=47036318, tablet: id=47036321, tablet: id=47036324, tablet: id=47036327, tablet: id=47036330, tablet: id=47036333, tablet: id=47036336, tablet: id=47036339, tablet: id=47036342, tablet: id=47036345, tablet: id=47036348, tablet: id=47036351, tablet: id=47036354, tablet: id=47036357, tablet: id=47036360, tablet: id=47036363, tablet: id=47036366, tablet: id=47036369, tablet: id=47036372, tablet: id=47036375, tablet: id=47036378, tablet: id=47036381, tablet: id=47036384, tablet: id=47036387, tablet: id=47036390, tablet: id=47036393, tablet: id=47036396, tablet: id=47036399, tablet: id=47036402, tablet: id=47036405, tablet: id=47036408, tablet: id=47036411, tablet: id=47036414, tablet: id=47036417, tablet: id=47036420, tablet: id=47036423, tablet: id=47036426, tablet: id=47036429, tablet: id=47036432, tablet: id=47036435, tablet: id=47036438, tablet: id=47036441, tablet: id=47036444, tablet: id=47036447, tablet: id=47036450, tablet: id=47036453, tablet: id=47036456, tablet: id=47036459, tablet: id=47036462, tablet: id=47036465, tablet: id=47036468, tablet: id=47036471, tablet: id=47036474, tablet: id=47036477, tablet: id=47036480, tablet: id=47036483, tablet: id=47036486, tablet: id=47036489, tablet: id=47036492, ]; ; rollup count: 0; committedVersion: 1; committedVersionHash: 0; distribution_info.type: RANDOM; distribution_info: com.starrocks.catalog.RandomDistributionInfo@73bc77b4
2024-08-26 11:31:46.764+08:00 INFO (thrift-server-pool-30964700|48567547) [LeaderImpl.finishCreateReplica():407] finish create replica. tablet id: 47036351, be: 10007, report version: 17207618031836, tablet type: TABLET_TYPE_DISK
2024-08-26 11:31:46.764+08:00 INFO (thrift-server-pool-30964708|48567577) [LeaderImpl.finishCreateReplica():407] finish create replica. tablet id: 47036351, be: 10009, report version: 17207619741636, tablet type: TABLET_TYPE_DISK
2024-08-26 11:31:56.858+08:00 INFO (tablet scheduler|35) [DiskAndTabletLoadReBalancer.selectAlternativeTabletsForCluster():111] select tablets to balance backend tablet distribution: total 35, medium HDD, tablets[show up to 100]: [47036477, 47036351, 47036447, 10097, 47036302, 47036447, 47036489, 10081, 47036221, 47036026, 47036173, 47036236, 47036299, 47036266, 47036348, 47036462, 47036333, 47036429, 47036345, 47036284, 47036429, 47036492, 47036399, 47036140, 47036191, 47036360, 47036453, 47036327, 47036390, 47036236, 47036061, 47036351, 47036318, 47036414, 47036094]
2024-08-26 11:31:57.859+08:00 INFO (tablet scheduler|35) [TabletScheduler.scheduleTablet():808] change TabletSchedCtx status from null to HEALTHY, partition visible version: 2, visibleTxnId: 27905022, tablet: 47036351, all version infos: 10007:2/-1/2/0:NORMAL:ALIVE,10009:2/-1/2/0:NORMAL:ALIVE,
2024-08-26 11:31:57.859+08:00 INFO (tablet scheduler|35) [TabletScheduler.schedulePendingTablets():640] add task to agent task queue: tablet id: 47036351, schema hash: 638971200, storageMedium: HDD, visible version(hash): 2-0, src backend: 10.106.40.74, src path hash: -7304654999848414668, dest backend: 10.106.40.74, dest path hash: 3928197426938143646, is local: true
2024-08-26 11:31:57.948+08:00 INFO (thrift-server-pool-30964711|48567624) [TabletSchedCtx.finishCloneTask():1049] clone finished: tablet id: 47036351, status: HEALTHY, state: FINISHED, type: BALANCE. from backend: 10009, src path hash: -7304654999848414668. to backend: 10009, dest path hash: 3928197426938143646
2024-08-26 11:31:57.948+08:00 INFO (thrift-server-pool-30964711|48567624) [TabletScheduler.removeTabletCtx():1588] remove the tablet 47036351. because: finished
2024-08-26 11:32:17.087+08:00 INFO (tablet scheduler|35) [DiskAndTabletLoadReBalancer.selectAlternativeTabletsForCluster():111] select tablets to balance backend tablet distribution: total 8, medium HDD, tablets[show up to 100]: [10097, 10081, 47036447, 47036333, 47036345, 47036429, 47036236, 47036351]
2024-08-26 11:32:18.087+08:00 INFO (tablet scheduler|35) [TabletScheduler.scheduleTablet():808] change TabletSchedCtx status from null to HEALTHY, partition visible version: 2, visibleTxnId: 27905022, tablet: 47036351, all version infos: 10007:2/-1/2/0:NORMAL:ALIVE,10009:2/-1/2/0:NORMAL:ALIVE,
2024-08-26 11:32:18.088+08:00 INFO (tablet scheduler|35) [TabletScheduler.schedulePendingTablets():640] add task to agent task queue: tablet id: 47036351, schema hash: 638971200, storageMedium: HDD, visible version(hash): 2-0, src backend: 10.106.40.130, src path hash: -702940173499575259, dest backend: 10.106.40.130, dest path hash: 5260005199039007643, is local: true
2024-08-26 11:32:18.167+08:00 INFO (thrift-server-pool-30964835|48567859) [TabletSchedCtx.finishCloneTask():1049] clone finished: tablet id: 47036351, status: HEALTHY, state: FINISHED, type: BALANCE. from backend: 10007, src path hash: -702940173499575259. to backend: 10007, dest path hash: 5260005199039007643
2024-08-26 11:32:18.167+08:00 INFO (thrift-server-pool-30964835|48567859) [TabletScheduler.removeTabletCtx():1588] remove the tablet 47036351. because: finished
2024-08-26 11:35:07.393+08:00 INFO (ReportHandler|190) [ReportHandler.addDropReplicaTask():964] delete tablet[47036351] from backend[10007] because not found in meta
2024-08-26 11:35:39.961+08:00 INFO (ReportHandler|190) [ReportHandler.addDropReplicaTask():964] delete tablet[47036351] from backend[10009] because not found in meta
FE日志tablet:47036351日志
BE10007关于talet 47036351日志:
I0826 11:31:46.757215 574274 agent_server.cpp:426] Submit task success. type=CREATE, signature=47036351, task_count_in_queue=4
I0826 11:31:46.758189 892817 tablet_manager.cpp:169] Creating tablet 47036351
I0826 11:31:46.758482 892817 tablet_manager.cpp:238] Created tablet 47036351
I0826 11:31:46.791477 892921 local_tablets_channel.cpp:678] LocalTabletsChannel txn_id: 27905022 load_id: b8a66134-635b-11ef-9e56-525400d84ba9 open 17 delta writer: [47036486:2][47036477:1][47036465:2][47036456:1][47036318:2][47036435:1][47036330:1][47036339:2][47036351:1][47036360:2][47036372:1][47036381:2][47036393:1][47036402:2][47036414:1][47036423:2][47036444:2] 0 failed_tablets: _num_remaining_senders: 56
I0826 11:31:54.093851 892865 local_tablets_channel.cpp:548] LocalTabletsChannel txn_id: 27905022 load_id: b8a66134-635b-11ef-9e56-525400d84ba9 commit 8 tablets: 47036477,47036456,47036435,47036330,47036351,47036372,47036393,47036414
I0826 11:31:54.322743 892996 txn_manager.cpp:298] Commit txn successfully. tablet: 47036351, txn_id: 27905022, rowsetid: 02000000fd96ce86154f6526d973eab32cfd0dc763d4ad84 #segment:5 #delfile:0 #uptfiles:0
I0826 11:31:55.046574 267629 txn_manager.cpp:339] add txn info history. txn_id: 27905022, partition_id: 47036314, tablet_id: 47036351, schema_hash: 638971200, rowset_id: 02000000fd96ce86154f6526d973eab32cfd0dc763d4ad84, version: 2
I0826 11:31:55.046586 267629 publish_version.cpp:167] Publish txn success tablet:47036351 version:2 tablet_max_version:2 partition:47036314 txn_id: 27905022 rowset:02000000fd96ce86154f6526d973eab32cfd0dc763d4ad84
I0826 11:31:55.788466 892883 size_tiered_compaction_policy.cpp:353] pick tablet 47036351 for size-tiered compaction rowset version=0-2 score=29 level_size=1 total_size=26248922 segment_num=6 force_base_compaction=0 reached_max_versions=0
I0826 11:31:55.798620 892886 compaction_manager.cpp:87] submit task to compaction pool, task_id:18890256, tablet_id:47036351, compaction_type:base, compaction_score:29 for round:19125973, candidates_size:15
I0826 11:31:55.821859 892885 compaction_task.cpp:39] start compaction. task_id:18890256, tablet:47036351, algorithm:VERTICAL_COMPACTION, compaction_type:base, compaction_score:29, output_version:[0-2], input rowsets size:2
I0826 11:31:58.786844 892885 compaction_task.cpp:137] compaction finish. status:OK, task info:[CompactionTaskInfo] task_id:18890256, tablet_id:47036351, compaction score:29, algorithm:VERTICAL_COMPACTION, state:COMPACTION_SUCCESS, compaction_type:base, output_version:[0-2], start_time:2024-08-26 11:31:55.821, end_time:2024-08-26 11:31:58.786, elapsed_time:2964985 us, input_rowsets_size:26248921, input_segments_num:5, input_rowsets_num:2, input_rows_num:1493099, output_num_rows:1493099, merged_rows:0, filtered_rows:0, output_segments_num:1, output_rowset_size:26115992, column_group_size:7, total_output_num_rows:10451693, total_merged_rows:0, total_del_filtered_rows:0, is_shortcut_compaction:0, is_manual_compaction:0, progress:100
I0826 11:32:18.083606 574274 agent_server.cpp:444] Submit task success. type=CLONE, signature=47036351, task_count_in_queue=1
I0826 11:32:18.083935 268541 engine_storage_migration_task.cpp:86] begin to process storage migrate. tablet_id=47036351, schema_hash=638971200, tablet=47036351.638971200.004d981d95d01d40-ff38934b7dde178c, dest_store=/data1/emr/starrocks/be/storage
I0826 11:32:18.140412 268541 tablet_manager.cpp:905] Loading tablet 47036351 from /data1/emr/starrocks/be/storage/data/388/47036351/638971200. force=0 restore=0
I0826 11:32:18.161094 268541 tablet_manager.cpp:1469] Start to drop tablet 47036351
I0826 11:32:18.161242 268541 tablet_manager.cpp:1513] Succeed to drop tablet 47036351
I0826 11:32:18.161252 268541 tablet_manager.cpp:137] Added duplicated tablet. tablet_id=47036351 old_tablet_path=/data/emr/starrocks/be/storage/data/995/47036351/638971200 new_tablet_path=/data1/emr/starrocks/be/storage/data/388/47036351/638971200
I0826 11:32:18.161396 268541 agent_task.cpp:358] local tablet migration succeeded. status: OK, signature: 47036351
I0826 11:32:18.161403 268541 tablet_manager.cpp:949] Reporting tablet info. tablet_id=47036351
I0826 11:35:07.390457 574274 agent_server.cpp:430] Submit task success. type=DROP, signature=47036351, task_count_in_queue=21
I0826 11:35:07.392004 273802 tablet_manager.cpp:394] Start to drop tablet 47036351
I0826 11:35:07.392125 273802 tablet_manager.cpp:439] Succeed to drop tablet 47036351
I0826 11:35:07.392313 273802 agent_task.cpp:155] Remove task success. type=DROP, signature=47036351, task_count_in_queue=20
I0826 11:35:12.419454 892877 tablet_manager.cpp:1079] Removed /data1/emr/starrocks/be/storage/data/388/47036351
W0826 11:35:12.515421 892877 storage_engine.cpp:1104] tablet not found, remove rowset meta, rowset_id=02000000fd96ce86154f6526d973eab32cfd0dc763d4ad84 tablet: 47036351
BE10009关于tablet日志：
I0826 11:31:46.756857 226414 agent_server.cpp:426] Submit task success. type=CREATE, signature=47036351, task_count_in_queue=4
I0826 11:31:46.757944 913473 tablet_manager.cpp:169] Creating tablet 47036351
I0826 11:31:46.758422 913473 tablet_manager.cpp:238] Created tablet 47036351
I0826 11:31:46.790818 913529 local_tablets_channel.cpp:678] LocalTabletsChannel txn_id: 27905022 load_id: b8a66134-635b-11ef-9e56-525400d84ba9 open 17 delta writer: [47036489:1][47036477:2][47036468:1][47036456:2][47036321:1][47036330:2][47036447:1][47036342:1][47036351:2][47036363:1][47036372:2][47036384:1][47036393:2][47036405:1][47036414:2][47036426:1][47036435:2] 0 failed_tablets: _num_remaining_senders: 56
I0826 11:31:54.322238 912902 txn_manager.cpp:298] Commit txn successfully. tablet: 47036351, txn_id: 27905022, rowsetid: 02000000fe4a8199d34b3134ec002a4f069552f7b56beea0 #segment:5 #delfile:0 #uptfiles:0
I0826 11:31:55.047042 525781 txn_manager.cpp:339] add txn info history. txn_id: 27905022, partition_id: 47036314, tablet_id: 47036351, schema_hash: 638971200, rowset_id: 02000000fe4a8199d34b3134ec002a4f069552f7b56beea0, version: 2
I0826 11:31:55.047056 525781 publish_version.cpp:167] Publish txn success tablet:47036351 version:2 tablet_max_version:2 partition:47036314 txn_id: 27905022 rowset:02000000fe4a8199d34b3134ec002a4f069552f7b56beea0
I0826 11:31:56.039783 913539 size_tiered_compaction_policy.cpp:353] pick tablet 47036351 for size-tiered compaction rowset version=0-2 score=29 level_size=1 total_size=26248922 segment_num=6 force_base_compaction=0 reached_max_versions=0
I0826 11:31:56.040397 913542 compaction_manager.cpp:87] submit task to compaction pool, task_id:18968649, tablet_id:47036351, compaction_type:base, compaction_score:29 for round:19203428, candidates_size:15
I0826 11:31:56.041133 525817 compaction_task.cpp:39] start compaction. task_id:18968649, tablet:47036351, algorithm:VERTICAL_COMPACTION, compaction_type:base, compaction_score:29, output_version:[0-2], input rowsets size:2
I0826 11:31:57.857728 226414 agent_server.cpp:444] Submit task success. type=CLONE, signature=47036351, task_count_in_queue=4
I0826 11:31:57.858302 525883 engine_storage_migration_task.cpp:86] begin to process storage migrate. tablet_id=47036351, schema_hash=638971200, tablet=47036351.638971200.94412019f8d3189b-0fea90a7f80cefa2, dest_store=/data1/emr/starrocks/be/storage
I0826 11:31:57.940940 525883 tablet_manager.cpp:905] Loading tablet 47036351 from /data1/emr/starrocks/be/storage/data/207/47036351/638971200. force=0 restore=0
I0826 11:31:57.942031 525883 tablet_manager.cpp:1469] Start to drop tablet 47036351
I0826 11:31:57.942239 525883 tablet_manager.cpp:1513] Succeed to drop tablet 47036351
I0826 11:31:57.942258 525883 tablet_manager.cpp:137] Added duplicated tablet. tablet_id=47036351 old_tablet_path=/data/emr/starrocks/be/storage/data/832/47036351/638971200 new_tablet_path=/data1/emr/starrocks/be/storage/data/207/47036351/638971200
I0826 11:31:57.942493 525883 agent_task.cpp:358] local tablet migration succeeded. status: OK, signature: 47036351
I0826 11:31:57.942507 525883 tablet_manager.cpp:949] Reporting tablet info. tablet_id=47036351
I0826 11:31:57.942981 525817 vertical_compaction_task.cpp:243] vertical compaction task_id:18968649, tablet:47036351 is stopped.
W0826 11:31:57.961738 525817 compaction_task.cpp:184] compaction task:18968649, tablet:47036351 failed.
/data/emr/service/jenkins/workspace/starrocks-pipeline/build-315/starrocks/be/src/storage/vertical_compaction_task.cpp:38 _vertical_compaction_data(&statistics), task info:[CompactionTaskInfo] task_id:18968649, tablet_id:47036351, compaction score:29, algorithm:VERTICAL_COMPACTION, state:COMPACTION_FAILED, compaction_type:base, output_version:[0-2], start_time:2024-08-26 11:31:56.041, end_time:2024-08-26 11:31:57.961, elapsed_time:1920653 us, input_rowsets_size:26248921, input_segments_num:5, input_rowsets_num:2, input_rows_num:1493099, output_num_rows:0, merged_rows:0, filtered_rows:0, output_segments_num:0, output_rowset_size:0, column_group_size:7, total_output_num_rows:7632539, total_merged_rows:0, total_del_filtered_rows:0, is_shortcut_compaction:0, is_manual_compaction:0, progress:73
I0826 11:34:12.306675 913533 tablet_manager.cpp:1079] Moved /data/emr/starrocks/be/storage/data/832/47036351
I0826 11:35:39.957653 226414 agent_server.cpp:430] Submit task success. type=DROP, signature=47036351, task_count_in_queue=22
I0826 11:35:39.959777 913472 tablet_manager.cpp:394] Start to drop tablet 47036351
I0826 11:35:39.959789 913472 tablet_manager.cpp:439] Succeed to drop tablet 47036351
I0826 11:35:39.959932 913472 agent_task.cpp:155] Remove task success. type=DROP, signature=47036351, task_count_in_queue=27
I0826 11:37:23.005239 913533 tablet_manager.cpp:1079] Removed /data1/emr/starrocks/be/storage/data/207/47036351
