【详述】在执行多表关联操作的时候偶尔发生查询失败反馈错误代码如下:
transmit chunk rpc failed
【背景】目前解决方案是关闭pipeline SET GLOBAL enable_pipeline_engine = false
【业务影响】
【StarRocks版本】例如:2.2.5
【集群规模】3fe(1 follower+2observer)+3be(fe与be混部)
【机器信息】CPU虚拟核/内存/网卡,例如:32C/128G/千兆
【附件】
log 如下
2022-09-15 09:59:22,642 INFO (replayer|96) [Catalog.replayJournal():2522] replayed journal id is 3489575, replay to journal id is 3489576
2022-09-15 09:59:24,946 INFO (nioEventLoopGroup-3-11|118) [RestBaseAction.handleRequest():55] receive http request. url=/metrics
2022-09-15 09:59:26,724 WARN (thrift-server-pool-1|178) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode INTERNAL_ERROR transmit chunk rpc failed:02705ea6-349a-11ed-b973-5e65fcc4101e, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41031
2022-09-15 09:59:26,724 WARN (thrift-server-pool-1|178) [Coordinator.updateStatus():868] one instance report fail throw updateStatus(), need cancel. job id: -1, query id: 02705ea6-349a-11ed-b973-5e65fcc41006, instance id: 02705ea6-349a-11ed-b973-5e65fcc41031
2022-09-15 09:59:26,725 INFO (thrift-server-pool-1|178) [Coordinator.cancelInternal():963] unfinished instance: 02705ea6-349a-11ed-b973-5e65fcc41025
2022-09-15 09:59:26,924 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc4102b
2022-09-15 09:59:26,924 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc4102d
2022-09-15 09:59:26,925 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41026
2022-09-15 09:59:26,926 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b97c3-5e65fcc4102c
2022-09-15 09:59:26,926 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41023
2022-09-15 09:59:26,927 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41025
2022-09-15 09:59:26,927 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc4102a
2022-09-15 09:59:26,927 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41028
2022-09-15 09:59:26,927 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41027
2022-09-15 09:59:26,927 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41024
2022-09-15 09:59:26,927 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41022
2022-09-15 09:59:26,928 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41029
2022-09-15 09:59:26,928 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode CANCELLED InternalError, query_id=02705ea6-349a-11ed-b973-5e65fcc41006 instance_id=02705ea6-349a-11ed-b973-5e65fcc41021
2022-09-15 09:59:26,928 WARN (thrift-server-pool-5|196) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode INTERNAL_ERROR transmit chunk rpc failed:02a5c687-349a-11ed-b973-5e65fcc4100e, query_id=02a5c687-349a-11ed-b973-5e65fcc41006 instance_id=02a5c687-349a-11ed-b973-5e65fcc41048
2022-09-15 09:59:26,928 WARN (thrift-server-pool-5|196) [Coordinator.updateStatus():868] one instance report fail throw updateStatus(), need cancel. job id: -1, query id: 02a5c687-349a-11ed-b973-5e65fcc41006, instance id: 02a5c687-349a-11ed-b973-5e65fcc41048
2022-09-15 09:59:26,929 WARN (starrocks-mysql-nio-pool-3934|4763) [Coordinator.getNext():887] get next fail, need cancel. status errorCode CANCELLED InternalError, query id: 02a5c687-349a-11ed-b973-5e65fcc41006
2022-09-15 09:59:26,929 INFO (thrift-server-pool-5|196) [Coordinator.cancelInternal():963] unfinished instance: 02a5c687-349a-11ed-b973-5e65fcc41028
2022-09-15 09:59:26,930 WARN (thrift-server-pool-0|176) [Coordinator.updateFragmentExecStatus():1661] one instance report fail errorCode INTERNAL_ERROR transmit chunk rpc failed:02a5c687-349a-11ed-b973-5e65fcc4100f, query_id=02a5c687-349a-11ed-b973-5e65fcc41006 instance_id=02a5c687-349a-11ed-b973-5e65fcc4104e
可以设置 set global enable_exchange_pass_through=true 观察。
还有是目前2.2版本还不建议使用pipelineengine呢。
辛苦发送下建表语句,SQL,explain costs + sql 的结果。
pipelineengine 2.2.5默认是开启的,enable_exchange_pass_through 这个参数的作用是什么,我在官网上找不到相关介绍。
你好。涉及到表比较多。是多表join 查询。目前关闭pipeline后,报错情况明显下降了。我们主表数据在2.4亿,语句是关联4个表,查询条件较复杂。数据您那边不好复现