array_agg 函数导致内部错误 com.starrocks.common.UserException: Internal error: std::get: wrong index for variant

💬 StarRocks 用户问答
#1

【详述】使用 array_agg 函数时, 提示错误: com.starrocks.common.UserException: Internal error: std::get: wrong index for variant
【背景】做过哪些操作？
【业务影响】 是
【是否存算分离】 是
【StarRocks版本】3.2.8
【集群规模】例如：3fe（1 follower+2observer）+20be
【机器信息】be 16C/64G/万兆
【联系方式】StarRocks 3.0-存算分离用户群: 可以自然点嘛
【附件】

SELECT '20240731'                   AS __d,
       IFNULL(base.device_id, '')            AS ifa,
       array_agg(ifa_bundle.bundle) AS media_bundle_array,
       IFNULL(base.make, '')                 AS make,
       IFNULL(base.model, '')                AS model,
       IFNULL(base.osv, '')                  AS os_version,
       IFNULL(base.lang_code, '')            AS lang_code,
       IFNULL(base.region, '')               AS region,
       base.latest_day                       as latest_day,
       base.country                          as country
from (SELECT device_id,
              make,
              model,
              osv,
              lang_code,
              region,
              latest_day,
              country
       FROM dws.rtb_ml_di_summary
       where __d = '20240731')
       as base
       LEFT JOIN (SELECT device_id, bundle
                     FROM dws.rtb_did_bundle) AS ifa_bundle
                     ON base.device_id = ifa_bundle.device_id
GROUP BY IFNULL(base.device_id, ''),
       IFNULL(base.make, ''),
       IFNULL(base.model, ''),
       IFNULL(base.osv, ''),
       IFNULL(base.lang_code, ''),
       IFNULL(base.region, ''),
       base.latest_day,
       base.country;
  • 查询报错：
process failed.
com.starrocks.common.UserException: Internal error: std::get: wrong index for variant
        at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2219) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.executeInsert(InsertOverwriteJobRunner.java:322) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.doLoad(InsertOverwriteJobRunner.java:161) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:141) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.transferTo(InsertOverwriteJobRunner.java:201) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.prepare(InsertOverwriteJobRunner.java:223) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.handle(InsertOverwriteJobRunner.java:138) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobRunner.run(InsertOverwriteJobRunner.java:126) ~[starrocks-fe.jar:?]
        at com.starrocks.load.InsertOverwriteJobMgr.executeJob(InsertOverwriteJobMgr.java:87) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.StmtExecutor.handleInsertOverwrite(StmtExecutor.java:1803) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:1893) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.StmtExecutor.handleDMLStmtWithProfile(StmtExecutor.java:1812) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:671) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:415) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:610) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:917) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: com.starrocks.common.DdlException: Internal error: std::get: wrong index for variant
        at com.starrocks.common.ErrorReport.reportDdlException(ErrorReport.java:95) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.StmtExecutor.handleDMLStmt(StmtExecutor.java:2023) ~[starrocks-fe.jar:?]
        ... 19 more

如果将
array_agg(ifa_bundle.bundle) AS media_bundle_array,
改为
array_agg(distinct ifa_bundle.bundle) AS media_bundle_array,
可以正常运行,

怀疑array_agg函数, 不带distinct时有 bug

#2

发个be.WARNING 里面应该有dump stack

#3 
W0801 09:57:19.039206   447 tablet_sink_sender.cpp:296] close channel failed. channel_name=NodeChannel[788306], load_info=load_id=b3c36381-4feb-11ef-b
ff9-aafbb7306a22, txn_id: 4088488, parallel=1, compress_type=2, error_msg=Cancelled by pipeline engine
W0801 09:57:19.117835   452 stack_util.cpp:350] 2024-08-01 09:57:19.117776, query_id=b3c36381-4feb-11ef-bff9-aafbb7306a22, fragment_instance_id=b3c363
81-4feb-11ef-bff9-aafbb7306a30 throws exception: std::bad_variant_access, trace:
     @          0x3cf946c  std::__throw_bad_variant_access()
    @          0x3cf9491  std::__throw_bad_variant_access()
    @          0x3d091ca  starrocks::BinaryColumnBase<>::append_datum()
    @          0x3f21f8d  starrocks::NullableColumn::append_datum()
    @          0x50f11cf  starrocks::ArrayAggAggregateFunctionV2::convert_to_serialize_format()
    @          0x60fff73  starrocks::Aggregator::output_chunk_by_streaming()
    @          0x623a937  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::_try_to_spill_by_auto()
    @          0x623ace6  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::push_chunk()
    @          0x5f1a2d2  starrocks::pipeline::PipelineDriver::process()
    @          0x679ab5e  starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x6b311ac  starrocks::ThreadPool::dispatch_thread()
    @          0x6b2a6ba  starrocks::Thread::supervise_thread()
    @     0x7ff12b1d9ac3  (unknown)
    @     0x7ff12b26b850  (unknown)
    @              (nil)  (unknown)

W0801 09:57:19.117956   452 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=b3c36381-4feb-11ef-bff9-aafbb7306a22, instance_id=b3c36381-4feb-11ef-bff9-aafbb7306a30, status=Internal error: Internal error: std::get: wrong index for variant
W0801 09:57:19.119218   442 stack_util.cpp:350] 2024-08-01 09:57:19.119185, query_id=b3c36381-4feb-11ef-bff9-aafbb7306a22, fragment_instance_id=b3c36381-4feb-11ef-bff9-aafbb7306a30 throws exception: std::bad_variant_access, trace:
     @          0x3cf946c  std::__throw_bad_variant_access()
    @          0x3cf9491  std::__throw_bad_variant_access()
    @          0x3d091ca  starrocks::BinaryColumnBase<>::append_datum()
    @          0x3f21f8d  starrocks::NullableColumn::append_datum()
    @          0x50f11cf  starrocks::ArrayAggAggregateFunctionV2::convert_to_serialize_format()
    @          0x60fff73  starrocks::Aggregator::output_chunk_by_streaming()
    @          0x623a937  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::_try_to_spill_by_auto()
    @          0x623ace6  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::push_chunk()
    @          0x5f1a2d2  starrocks::pipeline::PipelineDriver::process()
    @          0x679ab5e  starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x6b311ac  starrocks::ThreadPool::dispatch_thread()
    @          0x6b2a6ba  starrocks::Thread::supervise_thread()
    @     0x7ff12b1d9ac3  (unknown)
    @     0x7ff12b26b850  (unknown)
    @              (nil)  (unknown)

W0801 09:57:19.119366   442 pipeline_driver_executor.cpp:170] [Driver] Process error, query_id=b3c36381-4feb-11ef-bff9-aafbb7306a22, instance_id=b3c36381-4feb-11ef-bff9-aafbb7306a30, status=Internal error: Internal error: std::get: wrong index for variant
#4

set global enable_agg_spill_preaggregation=false; 已知的一个bug since 3.2.8 可以先关闭

#5

3.2.10 修复的

#6

好的 谢谢

京ICP备2022026421号-1