常见 Crash / BUG / 优化查询

trueeyu · 2024年07月15日 06:13

使用迁移工具迁移数据后，读Decimal的表 Crash

*** Aborted at 1720971684 (unix time) try “date -d @1720971684” if you are using GNU date ***
PC: @ 0x2c39b4d starrocks::DecimalV3Column<>::put_mysql_row_buffer()
*** SIGSEGV (@0x8) received by PID 6303 (TID 0x2b3d6a763700) from PID 8; stack trace: ***
@ 0x67283e2 google::(anonymous namespace)::FailureSignalHandler()
@ 0x2b3d38f77cab os::Linux::chained_handler()
@ 0x2b3d38f7c59c JVM_handle_linux_signal
@ 0x2b3d38f6f8f8 signalHandler()
@ 0x2b3d3965b5d0 (unknown)
@ 0x2c39b4d starrocks::DecimalV3Column<>::put_mysql_row_buffer()
@ 0x5860704 starrocks::MysqlResultWriter::process_chunk()
@ 0x37a1e0d starrocks::pipeline::ResultSinkOperator::push_chunk()
@ 0x3842949 starrocks::pipeline::PipelineDriver::process()
@ 0x383378e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x2e493aa starrocks::ThreadPool::dispatch_thread()
@ 0x2e43e0a starrocks:
:supervise_thread()
@ 0x2b3d39653dd5 start_thread
@ 0x2b3d3a28d02d __clone
@ 0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/42333
Jira
问题版本:
- 3.1.0 ～ 3.1.10
- 3.2.0 ～ 3.2.6
- 3.3.0
修复版本:
- 3.1.11+
- 3.2.7+
- 3.3.1+
问题原因:
临进解决办法:
- 删除有问题的表，并升级到 Fix 版本，ADMIN SET FRONTEND CONFIG(“enable_legacy_compatibility_for_replication”=“true”); 后，重新使用迁移工具迁移（https://docs.starrocks.io/zh/docs/administration/data_migration_tool/)

trueeyu · 2024年07月15日 06:21

Spill process_chunk crash

*** Aborted at 1720077467 (unix time) try "date -d @1720077467" if you are using GNU date ***
PC: @          0x3853414 _ZNSt17_Function_handlerIFN9starrocks8StatusOrISt10shared_ptrINS0_5ChunkEEEEvEZNS0_24SpillProcessTasksBuilder7finallyIZNS0_8pipeline34SpillablePartitionSortSinkOperator13set_finishingEPNS0_12RuntimeStateEEUlSC_T_E0_EEvSD_EUlvE_E9_M_invokeERKSt9_Any_data
*** SIGSEGV (@0x90) received by PID 418760 (TID 0x2b5963a97700) from PID 144; stack trace: ***
    @          0x67c6d22 google::(anonymous namespace)::FailureSignalHandler()
    @     0x2b58c5bca2cb os::Linux::chained_handler()
    @     0x2b58c5bced5c JVM_handle_linux_signal
    @     0x2b58c5bc1c68 signalHandler()
    @     0x2b58c62b45e0 (unknown)
    @          0x3853414 _ZNSt17_Function_handlerIFN9starrocks8StatusOrISt10shared_ptrINS0_5ChunkEEEEvEZNS0_24SpillProcessTasksBuilder7finallyIZNS0_8pipeline34SpillablePartitionSortSinkOperator13set_finishingEPNS0_12RuntimeStateEEUlSC_T_E0_EEvSD_EUlvE_E9_M_invokeERKSt9_Any_data
    @          0x38e4328 starrocks::pipeline::SpillProcessOperator::pull_chunk()
    @          0x3868146 starrocks::pipeline::PipelineDriver::process()
    @          0x385a70e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x2e7d84c starrocks::ThreadPool::dispatch_thread()
    @          0x2e774ca starrocks::Thread::supervise_thread()
    @     0x2b58c62ace25 start_thread
    @     0x2b58c6ee134d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48130
Jira
问题版本:
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.13
- 3.2.0 ~ 3.2.9
- 3.3.0
修复版本:
- 3.0.10+
- 3.1.14+
- 3.2.10+
- 3.3.1+
问题原因:
临进解决办法:

trueeyu · 2024年07月15日 06:40

Window function + decimal crash

*** Aborted at 1716369559 (unix time) try "date -d @1716369559" if you are using GNU date ***
PC: @     0x7fa0b9fb5aa0 __memset_sse2
*** SIGSEGV (@0x7f9f42bf1000) received by PID 71755 (TID 0x7fa017016700) from PID 1119817728; stack trace: ***
    @          0x64fe742 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7fa0bac2b630 (unknown)
    @     0x7fa0b9fb5aa0 __memset_sse2
    @          0x32b5f5c _ZN9starrocks13DecimalV3Cast9to_stringInEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNSt9enable_ifIX29is_underlying_type_of_decimalIT_EES9_E4typeEii
    @          0x32b60ab starrocks::DecimalV3Column<>::put_mysql_row_buffer()
    @          0x5652f0c starrocks::MysqlResultWriter::process_chunk()
    @          0x35c7185 starrocks::pipeline::ResultSinkOperator::push_chunk()
    @          0x3668358 starrocks::pipeline::PipelineDriver::process()
    @          0x365891e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x2cf8f5a starrocks::ThreadPool::dispatch_thread()
    @          0x2cf39ea starrocks::Thread::supervise_thread()
    @     0x7fa0bac23ea5 start_thread
    @     0x7fa0ba024b0d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/46199
Jira
问题版本:
- 3.1.0 ~ 3.1.11
- 3.2.0 ~ 3.2.7
修复版本:
- 3.1.12+
- 3.2.8+
问题原因:
临时解决办法:
- set global cbo_push_down_distinct_below_window=false

trueeyu · 2024年07月17日 11:52

str_to_date crash

*** Aborted at 1718702913 (unix time) try "date -d @1718702913" if you are using GNU date ***
PC: @          0x2dde60d starrocks::TimestampValue::from_uncommon_format_str()
*** SIGSEGV (@0x0) received by PID 149374 (TID 0x153e743bc700) from PID 0; stack trace: ***
    @          0x674aa82 google::(anonymous namespace)::FailureSignalHandler()
    @     0x153f44b61249 os::Linux::chained_handler()
    @     0x153f44b6750a JVM_handle_linux_signal
    @     0x153f44b58fbc signalHandler()
    @     0x153f43d29cf0 (unknown)
    @          0x2dde60d starrocks::TimestampValue::from_uncommon_format_str()
    @          0x2ddf134 starrocks::TimestampValue::from_uncommon_format_str()
    @          0x5c65509 starrocks::TimeFunctions::str_to_date_uncommon()
    @          0x5c661fe starrocks::TimeFunctions::str_to_date()
    @          0x4e2ede4 starrocks::VectorizedFunctionCallExpr::evaluate_checked()
    @          0x45bcbe3 starrocks::ExprContext::evaluate()
    @          0x45bcf2f starrocks::ExprContext::evaluate()
    @          0x37ca917 starrocks::pipeline::ProjectOperator::push_chunk()
    @          0x386f3f4 starrocks::pipeline::PipelineDriver::process()
    @          0x3860f3e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x2e6bd0a starrocks::ThreadPool::dispatch_thread()
    @          0x2e6676a starrocks::Thread::supervise_thread()
    @     0x153f43d1f1ca start_thread
    @     0x153f42fe5e73 __GI___clone
    @                0x0 (unknown)

Github Issue:
- https://github.com/StarRocks/starrocks/issues/46198
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/46146
Jira
问题版本:
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.12
- 3.2.0 ~ 3.2.7
修复版本:
- 3.0.10+
- 3.1.13+
- 3.2.8+
问题原因:
临时解决办法:

trueeyu · 2024年07月18日 06:04

storage/tmp 目录下 crm 文件堆积

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/47283
Jira
问题版本:
- 3.1.0 ~ 3.1.13
- 3.2.0 ~ 3.2.8
- 3.3.0
修复版本:
- 3.1.14+
- 3.2.9+
- 3.3.1+
问题原因:
临时解决办法:

trueeyu · 2024年07月18日 06:06

Parser stack overflow

java.lang.StackOverflowError: null
        at org.antlr.v4.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:149) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at org.antlr.v4.runtime.BufferedTokenStream.nextTokenOnChannel(BufferedTokenStream.java:313) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at org.antlr.v4.runtime.CommonTokenStream.adjustSeekIndex(CommonTokenStream.java:70) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at org.antlr.v4.runtime.BufferedTokenStream.consume(BufferedTokenStream.java:137) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:537) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:393) ~[antlr4-runtime-4.9.2.jar:4.9.2]
        at com.starrocks.sql.parser.StarRocksParser.primaryExpression(StarRocksParser.java:40812) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.valueExpression(StarRocksParser.java:40052) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.predicate(StarRocksParser.java:39607) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.booleanExpression(StarRocksParser.java:39481) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.expression(StarRocksParser.java:39195) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.primaryExpression(StarRocksParser.java:40928) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.valueExpression(StarRocksParser.java:40052) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.predicate(StarRocksParser.java:39607) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.booleanExpression(StarRocksParser.java:39481) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.expression(StarRocksParser.java:39195) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.primaryExpression(StarRocksParser.java:40928) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.valueExpression(StarRocksParser.java:40052) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.predicate(StarRocksParser.java:39607) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.booleanExpression(StarRocksParser.java:39481) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.expression(StarRocksParser.java:39195) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.primaryExpression(StarRocksParser.java:40928) ~[starrocks-fe.jar:?]
        at com.starrocks.sql.parser.StarRocksParser.valueExpression(StarRocksParser.java:40052) ~[starrocks-fe.jar:?]
        at

Github Issue:
Github Fix PR:
Jira
问题版本:
修复版本:
问题原因:
临时解决办法:
- 调下JVM的参数: -Xss 调大

trueeyu · 2024年07月19日 10:56

IO error Input/output error

Dmesg -T 或是 /var/log/messages 一般有如下日志

blk_update_request: critical medium error

Github Issue:
Github Fix PR:
Jira
问题版本:
修复版本:
问题原因:
- 磁盘故障
解决办法:

trueeyu · 2024年08月8日 07:36

升级后，主键模型索引 Fail to do LZ4FRAME decompress

get_applied_rowsets failed, tablet updates is in error state: tablet:96156414 apply tablet: 96156414 failed, status: Invalid argument: _apply_rowset_commit error: apply rowset update state failed: Invalid argument: Fail to do LZ4FRAME decompress, res=ERROR_frameType_unknown

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48659
Jira
问题版本:
- 3.3.1
修复版本:
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月1日 10:19

BloomFilter 索引导致查询结果不对

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48632
Jira
问题版本:
- 3.3.0 ~ 3.3.1
修复版本:
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月1日 10:21

DeltaLake.getSnapshot 时间过长

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/46526
Jira
问题版本:
- 3.2.0 ~ latest
- 3.3.0 ~ 3.3.1
修复版本:
- 3.2 未解决
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月1日 10:27

Global Runtime Filter 导致 Join 结果不对

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/44111
- https://github.com/StarRocks/starrocks/pull/48496
Jira
问题版本:
- 2.5.0 ~ 2.5.22
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.13
- 3.2.0 ~ 3.2.9
- 3.3.0 ~ 3.3.1
修复版本:
- 2.5.23+
- 3.0.10+
- 3.1.14+
- 3.2.10+
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月1日 10:50

FE ATNConfig 占用大量内存

num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:     320005004    15360240192  org.antlr.v4.runtime.atn.ATNConfig
   2:       3167245    12616079640  [Lorg.antlr.v4.runtime.dfa.DFAState;
   3:       7630136     3961312952  [Ljava.lang.Object; (java.base@11.0.20.1)
   4:      59908057     2875586736  java.util.HashMap$Node (java.base@11.0.20.1)
   5:      41277716     1651108640  org.antlr.v4.runtime.atn.SingletonPredictionContext
   6:       5130700     1267901504  [B (java.base@11.0.20.1)
   7:       1939016      927555344  [Ljava.util.HashMap$Node; (java.base@11.0.20.1)
   8:      13615342      871381888  java.util.LinkedHashMap$Entry (java.base@11.0.20.1)
   9:       5157746      825239360  com.starrocks.catalog.Replica
  10:      26627877      639069048  java.lang.Long (java.base@11.0.20.1)
  11:       8359196      358183080  [Lorg.antlr.v4.runtime.atn.PredictionContext;

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48776
Jira
问题版本:
- 3.1 ~ latest
- 3.2 ~ latest
- 3.3.0 ~ 3.3.1
修复版本:
- 3.1 未修复
- 3.2 未修复
- 3.3.2+
问题原因:
临时解决办法:
- 升级到 Fix 版本，并且设置 fe.conf enable_parser_context_cache=false (会影响parser 性能），出问题的时候再打开，与特定SQL相关，根本原因还未找到

trueeyu · 2024年08月1日 10:58

3.3 做了 SchemaChange 后，回滚到 3.2, 导入 crash

tracker:replication consumption: 0
*** Aborted at 1721981314 (unix time) try "date -d @1721981314" if you are using GNU date ***
PC: @          0x5264ae5 starrocks::MemTable::insert()
*** SIGSEGV (@0x0) received by PID 1 (TID 0x7f1dc4bcd700) from PID 0; stack trace: ***
    @          0x67f9e42 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f2332c6b630 (unknown)
    @          0x5264ae5 starrocks::MemTable::insert()
    @          0x52d5c36 starrocks::lake::DeltaWriterImpl::write()
    @          0x52d6325 starrocks::lake::DeltaWriter::write()
    @          0x52c4f19 starrocks::lake::AsyncDeltaWriterImpl::execute()
    @          0x695cc7c bthread::ExecutionQueueBase::_execute()
    @          0x695d9f8 bthread::ExecutionQueueBase::_execute_tasks()
    @          0x2e875dc starrocks::ThreadPool::dispatch_thread()
    @          0x2e8125a starrocks::Thread::supervise_thread()
    @     0x7f2332c63ea5 start_thread
    @     0x7f2332064b0d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/49075
Jira
问题版本:
- 3.2.0 ~ 3.2.9
修复版本:
- 3.2.10+
问题原因:
临时解决办法:

trueeyu · 2024年08月1日 11:11

开启 Spill 后 array_agg 报错

 com.starrocks.common.UserException: Internal error: std::get: wrong index for variant

W0801 09:57:19.117835   452 stack_util.cpp:350] 2024-08-01 09:57:19.117776, query_id=b3c36381-4feb-11ef-bff9-aafbb7306a22, fragment_instance_id=b3c363
81-4feb-11ef-bff9-aafbb7306a30 throws exception: std::bad_variant_access, trace:
     @          0x3cf946c  std::__throw_bad_variant_access()
    @          0x3cf9491  std::__throw_bad_variant_access()
    @          0x3d091ca  starrocks::BinaryColumnBase<>::append_datum()
    @          0x3f21f8d  starrocks::NullableColumn::append_datum()
    @          0x50f11cf  starrocks::ArrayAggAggregateFunctionV2::convert_to_serialize_format()
    @          0x60fff73  starrocks::Aggregator::output_chunk_by_streaming()
    @          0x623a937  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::_try_to_spill_by_auto()
    @          0x623ace6  starrocks::pipeline::SpillableAggregateBlockingSinkOperator::push_chunk()
    @          0x5f1a2d2  starrocks::pipeline::PipelineDriver::process()
    @          0x679ab5e  starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x6b311ac  starrocks::ThreadPool::dispatch_thread()
    @          0x6b2a6ba  starrocks::Thread::supervise_thread()
    @     0x7ff12b1d9ac3  (unknown)
    @     0x7ff12b26b850  (unknown)
    @              (nil)  (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48700
Jira
问题版本:
- 3.1.0 ~ 3.1.14
- 3.2.0 ~ 3.2.9
- 3.3.0 ~ 3.3.1
修复版本:
- 3.1.15+
- 3.2.10+
- 3.3.2+
问题原因:
临时解决办法:
- set global enable_agg_spill_preaggregation=false

trueeyu · 2024年08月8日 06:23

Grouping Sets 导致BE报错或 crash

deserialize chunk data failed. column slot id: 86, column row count: 18446744073709551615, expected row count: 200. There is probably a bug here.

java.lang.RuntimeException: Runtime error: encode size does not equal when decoding, encode size = 4294967296, but decode get size = 1662, raw size = 2444.

*** Aborted at 1718758341 (unix time) try “date -d @1718758341” if you are using GNU date ***
PC: @ 0x8557ace svb_decode_avx_simple
*** SIGSEGV (@0x7fc28176c000) received by PID 27 (TID 0x7fc2f326a640) from PID 18446744071586627584; stack trace: ***
@ 0x7cfae6a google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fc358cfe520 (unknown)
@ 0x8557ace svb_decode_avx_simple
@ 0x8557d21 streamvbyte_decode
@ 0x55619ff starrocks::serde::(anonymous namespace)::decode_integers<>()
@ 0x5563a5b starrocks::ColumnVisitorMutableAdapter<>::visit()
@ 0x3d1001d starrocks::ColumnFactory<>::accept_mutable()
@ 0x55669ea starrocks::serde::ColumnArraySerde::deserialize()
@ 0x5566ccf starrocks::ColumnVisitorMutableAdapter<>::visit()
@ 0x3d638dd starrocks::ColumnFactory<>::accept_mutable()
@ 0x55669ea starrocks::serde::ColumnArraySerde::deserialize()
@ 0x5566c83 starrocks::ColumnVisitorMutableAdapter<>::visit()
@ 0x3d6561d starrocks::ColumnFactory<>::accept_mutable()
@ 0x55669ea starrocks::serde::ColumnArraySerde::deserialize()
@ 0x65620ba starrocks::serde::ProtobufChunkDeserializer::deserialize()
@ 0x662074b starrocks::DataStreamRecvr::SenderQueue::_deserialize_chunk()
@ 0x66249c7 starrocks::DataStreamRecvr::PipelineSenderQueue::get_chunk()
@ 0x66179d6 starrocks::DataStreamRecvr::get_chunk_for_pipeline()
@ 0x60a9493 starrocks::pipeline::ExchangeSourceOperator::pull_chunk()
@ 0x5c7e4d0 starrocks::pipeline::PipelineDriver::process()
@ 0x6508830 starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
@ 0x68ac92c starrocks::ThreadPool::dispatch_thread()
@ 0x68a5d6a starrocks::thread::supervise_thread()
@ 0x7fc358d50ac3 (unknown)
@ 0x7fc358de2850 (unknown)
@ 0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/48765
Jira
问题版本:
- 3.1.0 ～ 3.1.14
- 3.2.0 ～ 3.2.9
- 3.3.0 ～ 3.3.1
修复版本:
- 3.1.15+
- 3.2.10+
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月8日 07:34

SchemaChange 期间，查询会导致 BE crash

*** Aborted at 1721960790 (unix time) try "date -d @1721960790" if you are using GNU date ***
PC: @          0x36f80c0 starrocks::FixedLengthColumnBase<>::put_mysql_row_buffer()
*** SIGSEGV (@0x0) received by PID 2097 (TID 0x7f51f17a9700) from PID 0; stack trace: ***
    @          0xa15ccd2 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f5288b0a630 (unknown)
    @          0x36f80c0 starrocks::FixedLengthColumnBase<>::put_mysql_row_buffer()
    @          0x402e593 starrocks::StructColumn::put_mysql_row_buffer()
    @          0x6bcf7fc starrocks::MysqlResultWriter::process_chunk()
    @          0x440111d starrocks::pipeline::ResultSinkOperator::push_chunk()
    @          0x449e552 starrocks::pipeline::PipelineDriver::process()
    @          0x448e1bd starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x39c247c starrocks::ThreadPool::dispatch_thread()
    @          0x39bc14a starrocks::Thread::supervise_thread()
    @     0x7f5288b02ea5 start_thread
    @     0x7f5287cedb0d __clone
    @                0x0 (unknown)

*** Aborted at 1721942589 (unix time) try "date -d @1721942589" if you are using GNU date ***
PC: @     0x7f167d532d10 __memmove_ssse3_back
*** SIGSEGV (@0x0) received by PID 2765 (TID 0x7f160564e700) from PID 0; stack trace: ***
    @          0xa15ccd2 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f167f16154f os::Linux::chained_handler()
    @     0x7f167f1673b8 JVM_handle_linux_signal
    @     0x7f167f158db8 signalHandler()
    @     0x7f167e2f1630 (unknown)
    @     0x7f167d532d10 __memmove_ssse3_back
    @          0x36fcff8 starrocks::FixedLengthColumnBase<>::append()
    @          0x400e456 starrocks::NullableColumn::append()
    @          0x402d9e6 starrocks::StructColumn::append()
    @          0x400e46e starrocks::NullableColumn::append()
    @          0x3fecd42 starrocks::Chunk::append()
    @          0x62b6311 starrocks::ChunkPipelineAccumulator::push()
    @          0x45185fc starrocks::pipeline::ChunkAccumulateOperator::push_chunk()
    @          0x449e552 starrocks::pipeline::PipelineDriver::process()
    @          0x448e1bd starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x39c247c starrocks::ThreadPool::dispatch_thread()
    @          0x39bc14a starrocks::Thread::supervise_thread()
    @     0x7f167e2e9ea5 start_thread
    @     0x7f167d4d4b0d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/49355
Jira
问题版本:
- 3.2.0 ～ 3.2.9
- 3.3.0 ～ 3.3.1
修复版本:
- 3.2.10+
- 3.3.2+
问题原因:
临时解决办法:

trueeyu · 2024年08月8日 07:55

Change user null pointer

2024-06-11 11:19:24.834+08:00 WARN (starrocks-mysql-nio-pool-1739|29976) [ReadListener.lambda$handleEvent$0():81] Exception happened in one session(com.starrocks.mysql.nio.NConnectContext@159c7d69).
java.nio.BufferUnderflowException: null
        at java.nio.Buffer.nextGetIndex(Buffer.java:643) ~[?:?]
        at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:165) ~[?:?]
        at com.starrocks.mysql.MysqlProto.readByte(MysqlProto.java:312) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.MysqlProto.readInt1(MysqlProto.java:316) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.MysqlProto.readVInt(MysqlProto.java:342) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.MysqlProto.readLenEncodedString(MysqlProto.java:375) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.MysqlChangeUserPacket.readFrom(MysqlChangeUserPacket.java:88) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.MysqlProto.changeUser(MysqlProto.java:263) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.handleChangeUser(ConnectProcessor.java:139) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:507) ~[starrocks-fe.jar:?]
        at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:799) ~[starrocks-fe.jar:?]
        at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:829) ~[?:?]

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/47796
Jira
问题版本:
- 3.1.0 ~ 3.1.13
- 3.2.0 ~ 3.2.8
- 3.3.0
修复版本:
- 3.1.14+
- 3.2.9+
- 3.3.1+
问题原因:
临时解决办法:

trueeyu · 2024年08月8日 08:02

Parquet json 类型导入报错

Vectorized engine does not support the operator, cast from VARBINARY to ARRAY failed, maybe use switch function backend

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/49385
Jira
问题版本:
- 3.2.0 ～ 3.2.9
- 3.3.0 ～ 3.3.2
修复版本:
- 3.2.10+
- 3.3.3+
问题原因:
临时解决办法:

trueeyu · 2024年08月8日 10:31

获取统计信息，生成大量 Union ALL，导致BE内存占用过高

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/44671
Jira
问题版本:
- 3.1.0 ~ 3.1.11
- 3.2.0 ~ 3.2.4
修复版本:
- 3.1.12+
- 3.2.5+
问题原因:
临时解决办法:

trueeyu · 2024年08月15日 09:08

物化视图 crash

F0814 15:02:13.101240 32573 column_aggregate_func.cpp:379] Check failed: false unreachable path, unknown key column type: 22

*** Aborted at 1722929720 (unix time) try "date -d @1722929720" if you are using GNU date ***
PC: @     0x2aeb1af3d207 __GI_raise
*** SIGABRT (@0xa132) received by PID 41266 (TID 0x2afe8f136700) from PID 41266; stack trace: ***
    @          0x6538582 google::(anonymous namespace)::FailureSignalHandler()
    @     0x2aeb1a3d35d0 (unknown)
    @     0x2aeb1af3d207 __GI_raise
    @     0x2aeb1af3e8f8 __GI_abort
    @          0x2a6413e starrocks::failure_function()
    @          0x652bf5d google::LogMessage::Fail()
    @          0x652e3cf google::LogMessage::SendToLog()
    @          0x652baae google::LogMessage::Flush()
    @          0x652e9d9 google::LogMessageFatal::~LogMessageFatal()
    @          0x55edab2 starrocks::ColumnAggregatorFactory::create_key_column_aggregator()
    @          0x501a5a9 starrocks::ChunkAggregator::ChunkAggregator()
    @          0x5482b6f starrocks::AggregateIterator::init_encoded_schema()
    @          0x5005c71 starrocks::TabletReader::_init_collector()
    @          0x5006fc9 starrocks::TabletReader::open()
    @          0x4fea4a7 starrocks::SchemaChangeHandler::_do_process_alter_tablet_normal()
    @          0x4fec53b starrocks::SchemaChangeHandler::_do_process_alter_tablet()
    @          0x4fed22a starrocks::SchemaChangeHandler::process_alter_tablet()
    @          0x4f7eafa starrocks::EngineAlterTabletTask::execute()
    @          0x4e2d0ae starrocks::StorageEngine::execute_task()
    @          0x3280b30 starrocks::run_alter_tablet_task()
    @          0x2d0b99c starrocks::ThreadPool::dispatch_thread()
    @          0x2d0564a starrocks::Thread::supervise_thread()
    @     0x2aeb1a3cbdd5 start_thread

*** Aborted at 1723618933 (unix time) try “date -d @1723618933” if you are using GNU date ***
PC: @ 0x7fedbeafa387 __GI_raise
*** SIGABRT (@0x7e5a) received by PID 32346 (TID 0x7fed94ff0700) from PID 32346; stack trace: ***
@ 0x63bc462 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fedbf7c9630 (unknown)
@ 0x7fedbeafa387 __GI_raise
@ 0x7fedbeafba78 __GI_abort
@ 0x2a274de starrocks::failure_function()
@ 0x63afe3d google::LogMessage::Fail()
@ 0x63b22af google::LogMessage::SendToLog()
@ 0x63af98e google::LogMessage::Flush()
@ 0x63b28b9 google::LogMessageFatal::~LogMessageFatal()
@ 0x4cdf972 starrocks::create_value_aggregator()
@ 0x4ce01c1 starrocks::ColumnAggregatorFactory::create_value_column_aggregator()
@ 0x4763261 starrocks::ChunkAggregator::ChunkAggregator()
@ 0x4764765 starrocks::MemTable::_init_aggregator_if_needed()
@ 0x47649e5 starrocks::MemTable::MemTable()
@ 0x4cf0b3b starrocks::DeltaWriter::_reset_mem_table()
@ 0x4cf23e9 starrocks::DeltaWriter::write()
@ 0x59681e7 starrocks::AsyncDeltaWriter::_execute()
@ 0x6517eec bthread::ExecutionQueueBase::_execute()
@ 0x6518c68 bthread::ExecutionQueueBase::_execute_tasks()
@ 0x4fa7de2 starrocks::ThreadPool::dispatch_thread()
@ 0x4fa287a starrocks:
:supervise_thread()
@ 0x7fedbf7c1ea5 start_thread
@ 0x7fedbebc2b0d __clone
@ 0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/49528
Jira
问题版本:
- 3.1.0 ~ 3.1.14
- 3.2.0 ~ 3.2.9
- 3.3.0 ~ 3.3.2
修复版本:
- 3.1.15+
- 3.2.10+
- 3.3.3+
问题原因:
临时解决办法:

常见 Crash / BUG / 优化 查询

使用迁移工具迁移数据后，读Decimal的表 Crash

Spill process_chunk crash

Window function + decimal crash

str_to_date crash

storage/tmp 目录下 crm 文件堆积

Parser stack overflow

IO error Input/output error

升级后，主键模型索引 Fail to do LZ4FRAME decompress

BloomFilter 索引导致查询结果不对

DeltaLake.getSnapshot 时间过长

Global Runtime Filter 导致 Join 结果不对

FE ATNConfig 占用大量内存

3.3 做了 SchemaChange 后，回滚到 3.2, 导入 crash

开启 Spill 后 array_agg 报错

Grouping Sets 导致BE报错或 crash

SchemaChange 期间，查询会导致 BE crash

Change user null pointer

Parquet json 类型导入报错

获取统计信息，生成大量 Union ALL，导致BE内存占用过高

物化视图 crash

常见 Crash / BUG / 优化查询