为了更快的定位您的问题,请提供以下信息,谢谢
【详述】问题详细描述
be.WARN
W1118 21:13:13.805485 764 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=dd8e7d4f-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225707, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
E1118 21:13:15.282914 917 scan_operator.cpp:422] scan fragment de8846ac-a5ae-11ef-99e1-0242be89fd3e driver 1 Scan tasks error: Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
W1118 21:13:15.283385 755 pipeline_driver.cpp:313] pull_chunk returns not ok status Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:15.283452 755 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, instance_id=de8846ac-a5ae-11ef-99e1-0242be89fd3e, status=Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:15.283591 752 tablet_sink_sender.cpp:248] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.283664 752 tablet_sink_sender.cpp:248] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.283704 752 tablet_sink_sender.cpp:248] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.283762 752 tablet_sink_sender.cpp:248] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.284475 752 tablet_sink_sender.cpp:248] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.289635 758 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.289907 753 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.289954 760 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.289918 764 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:15.297482 759 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[10004], load_info=load_id=de8846ac-a5ae-11ef-99e1-0242be89fd3a, txn_id: 1225708, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W1118 21:13:22.253155 902 query_context.cpp:646] Retrying ReportExecStatus: No more data to read.
E1118 21:13:28.862882 907 scan_operator.cpp:422] scan fragment e6dde9a8-a5ae-11ef-99e1-0242be89fd3f driver 6 Scan tasks error: Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
W1118 21:13:28.866178 760 pipeline_driver.cpp:313] pull_chunk returns not ok status Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:28.866222 760 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=e6dde9a8-a5ae-11ef-99e1-0242be89fd3a, instance_id=e6dde9a8-a5ae-11ef-99e1-0242be89fd3f, status=Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
E1118 21:13:30.034988 910 scan_operator.cpp:422] scan fragment e79049bd-a5ae-11ef-99e1-0242be89fd3f driver 7 Scan tasks error: Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
W1118 21:13:30.071928 757 pipeline_driver.cpp:313] pull_chunk returns not ok status Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:30.071982 757 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a, instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3f, status=Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:52.257467 902 query_context.cpp:646] Retrying ReportExecStatus: No more data to read.
W1118 21:14:22.260623 902 query_context.cpp:646] Retrying ReportExecStatus: No more data to read.
W1118 21:14:52.264259 902 query_context.cpp:646] Retrying ReportExecStatus: No more data to read.
be.INFO
I1118 21:13:29.776834 730 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3b, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=0, is_pipeline=1, chunk_size=4096
I1118 21:13:29.776878 730 fragment_executor.cpp:182] Prepare(): query_id=e772af9c-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3b is_stream_pipeline=0 backend_num=0
I1118 21:13:29.780560 731 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3c, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=1, is_pipeline=1, chunk_size=4096
I1118 21:13:29.780598 731 fragment_executor.cpp:182] Prepare(): query_id=e772af9c-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3c is_stream_pipeline=0 backend_num=1
I1118 21:13:29.912829 750 pipeline_driver_executor.cpp:341] [Driver] Succeed to report exec state: fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3c, is_done=1
I1118 21:13:29.914564 750 pipeline_driver_executor.cpp:341] [Driver] Succeed to report exec state: fragment_instance_id=e772af9c-a5ae-11ef-99e1-0242be89fd3b, is_done=1
I1118 21:13:29.994297 727 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3b, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=0, is_pipeline=1, chunk_size=4096
I1118 21:13:29.994333 727 fragment_executor.cpp:182] Prepare(): query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3b is_stream_pipeline=0 backend_num=0
I1118 21:13:29.996068 733 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3c, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=1, is_pipeline=1, chunk_size=4096
I1118 21:13:29.996097 733 fragment_executor.cpp:182] Prepare(): query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3c is_stream_pipeline=0 backend_num=1
I1118 21:13:30.000003 735 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3d, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=2, is_pipeline=1, chunk_size=4096
I1118 21:13:30.000033 735 fragment_executor.cpp:182] Prepare(): query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3d is_stream_pipeline=0 backend_num=2
I1118 21:13:30.004904 732 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3e, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=3, is_pipeline=1, chunk_size=4096
I1118 21:13:30.004945 732 fragment_executor.cpp:182] Prepare(): query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3e is_stream_pipeline=0 backend_num=3
I1118 21:13:30.005192 723 internal_service.cpp:424] exec plan fragment, fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3f, coord=TNetworkAddress(hostname=youmei-node00, port=9020), backend=4, is_pipeline=1, chunk_size=4096
I1118 21:13:30.005213 723 fragment_executor.cpp:182] Prepare(): query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a fragment_instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3f is_stream_pipeline=0 backend_num=4
E1118 21:13:30.034988 910 scan_operator.cpp:422] scan fragment e79049bd-a5ae-11ef-99e1-0242be89fd3f driver 7 Scan tasks error: Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
W1118 21:13:30.071928 757 pipeline_driver.cpp:313] pull_chunk returns not ok status Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
W1118 21:13:30.071982 757 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=e79049bd-a5ae-11ef-99e1-0242be89fd3a, instance_id=e79049bd-a5ae-11ef-99e1-0242be89fd3f, status=Not found: no delete vector found tablet:757407 segment:0 version:2
/build/starrocks/be/src/storage/update_manager.cpp:187 get_del_vec_in_meta(meta, tsid, version, pdelvec->get(), &latest_version)
/build/starrocks/be/src/storage/rowset/segment_iterator.cpp:965 _init()
/build/starrocks/be/src/storage/tablet_reader.cpp:247 _collect_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/olap_chunk_source.cpp:397 _prj_iter->get_next(chunk)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:248 _get_scan_status()
I1118 21:13:30.072465 755 sink_buffer.cpp:215] fragment_instance_id e79049bd-a5ae-11ef-99e1-0242be89fd3d -> e79049bd-a5ae-11ef-99e1-0242be89fd3d, _num_uncancelled_sinkers 7, _is_finishing false, _num_remaining_eos 8
I1118 21:13:30.073184 757 sink_buffer.cpp:215] fragment_instance_id e79049bd-a5ae-11ef-99e1-0242be89fd3f -> e79049bd-a5ae-11ef-99e1-0242be89fd3f, _num_uncancelled_sinkers 7, _is_finishing false, _num_remaining_eos 8
I1118 21:13:30.073521 752 sink_buffer.cpp:215] fragment_instance_id e79049bd-a5ae-11ef-99e1-0242be89fd3d -> e79049bd-a5ae-11ef-99e1-0242be89fd3d, _num_uncancelled_sinkers 6, _is_finishing false, _num_remaining_eos 8
I1118 21:13:30.073980 753 sink_buffer.cpp:215] fragment_instance_id e79049bd-a5ae-11ef-99e1-0242be89fd3e -> e79049bd-a5ae-11ef-99e1-0242be89fd3e, _num_uncancelled_sinkers 7, _is_finishing false, _num_remaining_eos 8
I1118 21:13:30.074101 758 sink_buffer.cpp:215] fragment_instance_id e79049bd-a5ae-11ef-99e1-0242be89fd3f -> e79049bd-a5ae-11ef-99e1-0242be89fd3f, _num_uncancelled_sinkers 6, _is_finishing false, _num_remaining_eos 8
【背景】做过哪些操作?
【业务影响】
【是否存算分离】
【StarRocks版本】例如:3.2.7
【集群规模】例如:3fe(3 follower+ 0observer)+3be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
【联系方式】为了在解决问题过程中能及时联系到您获取一些日志信息,请补充下您的联系方式,例如:社区群4-小李或者邮箱,谢谢
【附件】
- fe.log/beINFO/相应截图
- 慢查询:
- Profile信息,获取Profile,通过Profile分析查询瓶颈
- 并行度:show variables like ‘%parallel_fragment_exec_instance_num%’;
- pipeline是否开启:show variables like ‘%pipeline%’;
- be节点cpu和内存使用率截图
- 查询报错:
- query_dump,怎么获取query_dump文件
- be crash
- be.out
- coredump,如何获取coredump
- 外表查询报错
- be.out和fe.warn.log