insert into 导致BE出现大量ERROR

💬 StarRocks 用户问答 日常运维
#1

【StarRocks版本】例如：3.1.9

测试下来，每次做insert into 导入数据的时候，BE总会有下面的很多报错。 不过好像没什么影响。 请问这些错误是啥意思，能否在后续版本过滤掉

E0321 15:29:28.371799 4232 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 27 Scan tasks error: Cancelled: canceled state
E0321 15:29:29.710595 4569 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 38 Scan tasks error: Cancelled: Cancelled because of runtime state is cancelled
/build/starrocks/be/src/connector/hive_connector.cpp:709 _scanner->get_next(state, chunk)
E0321 15:29:30.294092 4904 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 17 Scan tasks error: Cancelled: Cancelled because of runtime state is cancelled
/build/starrocks/be/src/connector/hive_connector.cpp:709 _scanner->get_next(state, chunk)
E0321 15:29:30.409484 4378 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 30 Scan tasks error: Cancelled: Cancelled because of runtime state is cancelled
/build/starrocks/be/src/connector/hive_connector.cpp:709 _scanner->get_next(state, chunk)
E0321 15:29:31.709573 4280 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 13 Scan tasks error: Cancelled: canceled state
E0321 15:29:31.776949 4834 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 39 Scan tasks error: Cancelled: canceled state
E0321 15:29:31.923074 4872 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 38 Scan tasks error: Cancelled: canceled state
E0321 15:29:31.950430 4551 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 8 Scan tasks error: Cancelled: canceled state
E0321 15:29:31.963696 4285 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 16 Scan tasks error: Cancelled: canceled state
E0321 15:29:32.054293 4863 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 41 Scan tasks error: Cancelled: canceled state
E0321 15:29:35.383184 4754 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 23 Scan tasks error: Cancelled: canceled state
E0321 15:29:36.057873 4796 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 39 Scan tasks error: Cancelled: Cancelled because of runtime state is cancelled
/build/starrocks/be/src/connector/hive_connector.cpp:709 _scanner->get_next(state, chunk)
E0321 15:29:49.519001 4398 scan_operator.cpp:412] scan fragment bbb3b5bc-e754-11ee-88c0-30b9300c8f7f driver 39 Scan tasks error: Cancelled: canceled state

#2

任务都是成功的是吗 3.1.9版本？ 我尝试验证下 看看能不能提个优化

#3

是的，都是手动执行的INSERT数据源为catelog的hive表，并且都成功了

#4

顺便瞅瞅个，生产环境be.WARNINGS大量出现这种日志。不知道是啥意思。

W0321 21:11:03.497184 399109 pipeline_driver_executor.cpp:166] [Driver] Process error, query_id=784a7a1e-e784-11ee-b2d9-0242e753711b, instance_id=784a7a1e-e784-11ee-b2d9-0242e7537121, status=Unknown code(48): : no global dict
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
W0321 21:11:03.497246 399120 pipeline_driver.cpp:308] pull_chunk returns not ok status Unknown code(48): : global dict greater than DICT_DECODE_MAX_SIZE
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
W0321 21:11:03.497335 399120 pipeline_driver_executor.cpp:166] [Driver] Process error, query_id=784a7a1f-e784-11ee-b2d9-0242e753711b, instance_id=784a7a1f-e784-11ee-b2d9-0242e7537122, status=Unknown code(48): : global dict greater than DICT_DECODE_MAX_SIZE
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
E0321 21:11:03.633271 399832 scan_operator.cpp:412] scan fragment 785f3aa1-e784-11ee-b2d9-0242e7537121 driver 0 Scan tasks error: Unknown code(48): : no global dict
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
W0321 21:11:03.633407 399138 pipeline_driver.cpp:308] pull_chunk returns not ok status Unknown code(48): : no global dict
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
W0321 21:11:03.633481 399138 pipeline_driver_executor.cpp:166] [Driver] Process error, query_id=785f3aa1-e784-11ee-b2d9-0242e753711b, instance_id=785f3aa1-e784-11ee-b2d9-0242e7537121, status=Unknown code(48): : no global dict
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
E0321 21:11:03.633960 399838 scan_operator.cpp:412] scan fragment 785f3aa0-e784-11ee-b2d9-0242e7537122 driver 0 Scan tasks error: Unknown code(48): : global dict greater than DICT_DECODE_MAX_SIZE
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
W0321 21:11:03.634083 399085 pipeline_driver.cpp:308] pull_chunk returns not ok status Unknown code(48): : global dict greater than DICT_DECODE_MAX_SIZE
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
W0321 21:11:03.634117 399085 pipeline_driver_executor.cpp:166] [Driver] Process error, query_id=785f3aa0-e784-11ee-b2d9-0242e753711b, instance_id=785f3aa0-e784-11ee-b2d9-0242e7537122, status=Unknown code(48): : global dict greater than DICT_DECODE_MAX_SIZE
/build/starrocks/be/src/storage/meta_reader.cpp:179 _collect(_params->fields[i], _params->cids[i], (*dsts)[i], _params->field_type[i])
/build/starrocks/be/src/storage/meta_reader.cpp:79 _collect_context.seg_collecters[_collect_context.cursor_idx]->collect(&columns)
/build/starrocks/be/src/storage/olap_meta_reader.cpp:158 _read((*result).get(), chunk_capacity - chunk_start)
/build/starrocks/be/src/exec/pipeline/scan/scan_operator.cpp:237 _get_scan_status()
W0321 21:11:06.041303 399101 mem_hook.cpp:266] large memory alloc: 1206529456 bytes, stack:
@ 0x2c41bb3 malloc
@ 0x88bf775 operator new()
@ 0x2ab836e std::vector<>::_M_range_insert<>()
@ 0x2abc1b4 starrocks::BinaryColumnBase<>::append()
@ 0x32a25da starrocks::NullableColumn::append()
@ 0x3387a73 starrocks::JoinHashTable::append_chunk()
@ 0x388d17c starrocks::HashJoinBuilder::append_chunk()
@ 0x3886f6c starrocks::HashJoiner::append_chunk_to_ht()
@ 0x36b1d69 starrocks::pipeline::HashJoinBuildOperator::push_chunk()
@ 0x36b9218 starrocks::pipeline::SpillableHashJoinBuildOperator::push_chunk()
@ 0x3668358 starrocks::pipeline::PipelineDriver::process()
@ 0x365891e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x2cf8f5a starrocks::ThreadPool::dispatch_thread()
@ 0x2cf39ea starrocks::thread::supervise_thread()
@ 0x7f8b61191ea5 start_thread
@ 0x7f8b605929fd __clone
@ (nil) (unknown)
W0321 21:11:07.953943 399073 mem_hook.cpp:266] large memory alloc: 2412981644 bytes, stack:
@ 0x2c41bb3 malloc
@ 0x88bf775 operator new()
@ 0x2ab836e std::vector<>::_M_range_insert<>()
@ 0x2abc1b4 starrocks::BinaryColumnBase<>::append()
@ 0x32a25da starrocks::NullableColumn::append()
@ 0x3387a73 starrocks::JoinHashTable::append_chunk()
@ 0x388d17c starrocks::HashJoinBuilder::append_chunk()
@ 0x3886f6c starrocks::HashJoiner::append_chunk_to_ht()
@ 0x36b1d69 starrocks::pipeline::HashJoinBuildOperator::push_chunk()
@ 0x36b9218 starrocks::pipeline::SpillableHashJoinBuildOperator::push_chunk()
@ 0x3668358 starrocks::pipeline::PipelineDriver::process()
@ 0x365891e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x2cf8f5a starrocks::ThreadPool::dispatch_thread()
@ 0x2cf39ea starrocks::thread::supervise_thread()
@ 0x7f8b61191ea5 start_thread
@ 0x7f8b605929fd __clone
@ (nil) (unknown)

#5

老哥，请问您这个问题解决了吗？

京ICP备2022026421号-1