常见 Crash / BUG / 优化查询

trueeyu · 2024年09月27日 02:24

Rocksdb Tombstone 占用大量 CPU

--16.31%--rocksdb::MemTable::NewRangeTombstoneIterator
          |          
           --15.92%--rocksdb::FragmentedRangeTombstoneList::FragmentedRangeTombstoneList
                     |          
                      --13.88%--rocksdb::FragmentedRangeTombstoneList::FragmentTombstones
                                |          
                                |--6.38%--rocksdb::FragmentedRangeTombstoneList::FragmentTombstones(std::unique_ptr<rocksdb::InternalIteratorBase<rocksdb::Slice>, std::default_delete<rocksdb::InternalIteratorBase<rocksdb::Slice> > >, rocksdb::InternalKeyComparator const&, bool, std::vector<unsigned long, std::allocator<unsigned long> > const&)::{lambda(rocksdb::Slice const&)#1}::operator()
                                |          |          
                                |          |--2.28%--std::_Rb_tree<unsigned long, unsigned long, std::_Identity<unsigned long>, std::less<unsigned long>, std::allocator<unsigned long> >::_M_get_insert_hint_unique_pos
                                |          |  
                                |           --0.68%--__memcmp_sse4_1
                                |          
                                |--2.36%--operator new
                                |          |          
                                |           --1.67%--my_malloc
                                |                     |          
                                |                      --0.51%--je_malloc_default
                                |          
                                |--1.77%--std::_Rb_tree<rocksdb::ParsedInternalKey, rocksdb::ParsedInternalKey, std::_Identity<rocksdb::ParsedInternalKey>, rocksdb::ParsedInternalKeyComparator, std::allocator<rocksdb::ParsedInternalKey> >::_M_emplace_unique<rocksdb::Slice&, unsigned long&, rocksdb::ValueType>
                                |          |          
                                |           --1.34%--rocksdb::InternalKeyComparator::Compare
                                |                     |          
                                |                      --0.55%--__memcmp_sse4_1
                                |          
                                 --0.70%--std::_Rb_tree_insert_and_rebalance

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/49798
- https://github.com/StarRocks/starrocks/pull/50037
Jira
问题版本:
- 3.1.0 ~ 3.1.14
- 3.2.0 ~ 3.2.10
- 3.3.0 ~ 3.3.2
修复版本:
- 3.1.15+
- 3.2.11+
- 3.3.3+
问题原因:
临时解决办法:

trueeyu · 2024年09月29日 01:58

Not found dict for cid:

SQL 涉及 Array 类型

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/51148
Jira
问题版本:
- 3.3.0 ～ 3.3.3
修复版本:
- 3.3.4+
问题原因:
临时解决办法:
- set global low_cardinality_optimize_v2=false;

trueeyu · 2024年10月9日 11:33

Thrift Server 线程泄漏，占用大量线程不释放

一般使用 FlinkConnector 或 SparkConnector 从 StarRocks导出数据会导致这个现象，并占用大量内存不释放。

    0x7fafd5348bcf  __poll
         0x98d537e  apache::thrift::transport::TSocket::read()
         0x98e1041  apache::thrift::transport::TBufferedTransport::readSlow()
         0x502c970  apache::thrift::transport::readAll<>()
         0x50366d6  apache::thrift::protocol::TVirtualProtocol<>::readMessageBegin_virt()
         0x8483b20  apache::thrift::TDispatchProcessor::process()
         0x98e7379  apache::thrift::server::TConnectedClient::run()
         0x98e8278  apache::thrift::server::TThreadedServer::TConnectedClientRunner::run()
         0x98eab0f  apache::thrift::concurrency::Thread::threadMain()
         0x98c19d9  std::thread::_State_impl<>::_M_run()
         0xe820e54  execute_native_thread_routine
    0x7fafd52c4ac3  (unknown)
    0x7fafd5355a04  clone
             (nil)  (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/50420
Jira
问题版本:
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.10
- 3.3.0 ~ 3.3.3
修复版本:
- 3.1.16+
- 3.2.11+
- 3.3.4+
问题原因:
临时解决办法:

trueeyu · 2024年10月18日 01:57

Join reorder failed

java.util.NoSuchElementException
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1000)
at com.google.common.collect.Ordering.min(Ordering.java:568)
at com.google.common.collect.Ordering.min(Ordering.java:593)
at com.starrocks.sql.optimizer.rule.join.JoinReorderDP.getBestExpr(JoinReorderDP.java:101)
at com.starrocks.sql.optimizer.rule.join.JoinReorderDP.enumerate(JoinReorderDP.java:48)
at com.starrocks.sql.optimizer.rule.join.JoinOrder.reorder(JoinOrder.java:231)
at com.starrocks.sql.optimizer.rule.join.ReorderJoinRule.enumerate(ReorderJoinRule.java:95)
at com.starrocks.sql.optimizer.rule.join.ReorderJoinRule.transform(ReorderJoinRule.java:234)
at com.starrocks.sql.optimizer.Optimizer.memoOptimize(Optimizer.java:815)
at com.starrocks.sql.optimizer.Optimizer.optimizeByCost(Optimizer.java:265)
at com.starrocks.sql.optimizer.Optimizer.optimize(Optimizer.java:189)
at com.starrocks.sql.StatementPlanner.createQueryPlanWithReTry(StatementPlanner.java:318)
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:133)
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:92)
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:548)
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:353)
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:548)
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:882)
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/51766
Jira
问题版本:
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.11
- 3.3.0 ~ 3.3.4
修复版本:
- 3.1.16+
- 3.2.12+
- 3.3.5+
问题原因:
临时解决办法:

trueeyu · 2024年10月18日 02:00

系统表查询 crash

*** Check failure stack trace: ***
    @     0x1479ab07d4d1 JVM_handle_linux_signal
    @     0x1479ab0701fc signalHandler()
terminate called recursively
terminate called recursively
    @     0x1479aa500cf0 (unknown)
    @          0x3cba1d7 starrocks::SchemaHelper::_call_rpc()
    @          0x3cba5b0 starrocks::SchemaHelper::list_fe_locks()
    @          0x3cbddba starrocks::SysFeLocks::start()
    @          0x3cf6188 _ZZSt9call_onceIZN9starrocks8pipeline17SchemaChunkSource5startEPNS0_12RuntimeStateEEUlvE_JEEvRSt9once_flagOT_DpOT0_ENUlvE0_4_FUNEv
    @     0x1479aa4fde67 __pthread_once_slow
    @          0x3cf6123 starrocks::pipeline::SchemaChunkSource::start()
    @          0x39be383 _ZZN9starrocks8pipeline12ScanOperator18_trigger_next_scanEPNS_12RuntimeStateEiENKUlvE_clEv
    @          0x3acd011 starrocks::workgroup::ScanExecutor::worker_thread()
    @          0x305409c starrocks::ThreadPool::dispatch_thread()
    @          0x304d5aa starrocks::Thread::supervise_thread()

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/45927
Jira
问题版本:
- 3.2.0 ~ 3.2.11
修复版本:
- 3.2.12+
问题原因:
临时解决办法:

trueeyu · 2024年10月18日 02:04

BE has overloaded with pipeline drivers

BE has overloaded with pipeline drivers

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/45314
Jira
问题版本:
- 2.5.0 ~ 2.5.20
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.11
- 3.2.0 ~ 3.2.6
修复版本:
- 2.5.21+
- 3.0.10+
- 3.1.12+
- 3.2.7+
问题原因:
临时解决办法:

trueeyu · 2024年10月18日 02:09

Partition TOPN crash

*** Aborted at 1728705795 (unix time) try "date -d @1728705795" if you are using GNU date ***
PC: @          0x349bee4 _ZN14pdqsort_detail12pdqsort_loopIN9__gnu_cxx17__normal_iteratorIPN9starrocks20VerticalColumnSorter16CompactChunkItemIdEESt6vectorIS6_SaIS6_EEEEZNS3_L19sort_and_tie_helperIZNS4_8do_visitIdEENS3_6StatusERKNS3_21FixedLengthColumnBaseIT_EEEUlRKS6_SL_E_SA_EESE_RKSt6atomicIbEPKNS3_6ColumnEbRT0_RS8_IhSaIhEESG_St4pairIiiEbmPmEUlSG_SU_E0_Lb0EEEvSG_SG_SU_ib
*** SIGSEGV (@0x7fb84bc0f008) received by PID 16730 (TID 0x7fb90eea8700) from PID 1270935560; stack trace: ***
    @          0x62e7a62 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7fb990b4a7fb os::Linux::chained_handler()
    @     0x7fb990b4f35c JVM_handle_linux_signal
    @     0x7fb990b41e78 signalHandler()
    @     0x7fb98f6aa630 (unknown)
    @          0x349bee4 _ZN14pdqsort_detail12pdqsort_loopIN9__gnu_cxx17__normal_iteratorIPN9starrocks20VerticalColumnSorter16CompactChunkItemIdEESt6vectorIS6_SaIS6_EEEEZNS3_L19sort_and_tie_helperIZNS4_8do_visitIdEENS3_6StatusERKNS3_21FixedLengthColumnBaseIT_EEEUlRKS6_SL_E_SA_EESE_RKSt6atomicIbEPKNS3_6ColumnEbRT0_RS8_IhSaIhEESG_St4pairIiiEbmPmEUlSG_SU_E0_Lb0EEEvSG_SG_SU_ib
    @          0x349c01e _ZN14pdqsort_detail12pdqsort_loopIN9__gnu_cxx17__normal_iteratorIPN9starrocks20VerticalColumnSorter16CompactChunkItemIdEESt6vectorIS6_SaIS6_EEEEZNS3_L19sort_and_tie_helperIZNS4_8do_visitIdEENS3_6StatusERKNS3_21FixedLengthColumnBaseIT_EEEUlRKS6_SL_E_SA_EESE_RKSt6atomicIbEPKNS3_6ColumnEbRT0_RS8_IhSaIhEESG_St4pairIiiEbmPmEUlSG_SU_E0_Lb0EEEvSG_SG_SU_ib
    @          0x3513e9e starrocks::VerticalColumnSorter::do_visit<>()
    @          0x3515736 starrocks::ColumnVisitorAdapter<>::visit()
    @          0x29300ff starrocks::ColumnFactory<>::accept()
    @          0x3476c5f starrocks::sort_vertical_columns()
    @          0x34ecc51 starrocks::VerticalColumnSorter::do_visit()
    @          0x34ecea6 starrocks::ColumnVisitorAdapter<>::visit()
    @          0x31f64dc starrocks::ColumnFactory<>::accept()
    @          0x3476c5f starrocks::sort_vertical_columns()
    @          0x347769f starrocks::sort_vertical_chunks()
    @          0x340135a starrocks::ChunksSorterTopn::_partial_sort_col_wise()
    @          0x34016a7 starrocks::ChunksSorterTopn::_filter_and_sort_data()
    @          0x34031b5 starrocks::ChunksSorterTopn::_sort_chunks()
    @          0x34037b0 starrocks::ChunksSorterTopn::do_done()
    @          0x33bd5c5 starrocks::ChunksSorter::done()
    @          0x35dd520 starrocks::pipeline::LocalPartitionTopnContext::transfer_all_chunks_from_partitioner_to_sorters()
    @          0x35ddac7 starrocks::pipeline::LocalPartitionTopnContext::push_one_chunk_to_partitioner()
    @          0x35bdd99 starrocks::pipeline::LocalPartitionTopnSinkOperator::push_chunk()
    @          0x3601cb3 starrocks::pipeline::PipelineDriver::process()
    @          0x35f387e starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x2c144ac starrocks::ThreadPool::dispatch_thread()
    @          0x2c0e12a starrocks::Thread::supervise_thread()
    @     0x7fb98f6a2ea5 start_thread
    @     0x7fb98f3cbb0d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/30759
Jira
问题版本:
- 2.5.0 ~ 2.5.16
- 3.0.0 ~ 3.0.8
- 3.1.0 ~ 3.1.6
- 3.2.0 ~ 3.2.11
修复版本:
- 2.5.17+
- 3.0.9+
- 3.1.7+
- 3.2.12+
问题原因:
临时解决办法:

trueeyu · 2024年10月24日 11:59

RuntimeFilter 下推导致 crash

*** Aborted at 1725338148 (unix time) try "date -d @1725338148" if you are using GNU date ***
PC: @          0x2dc0ce0 starrocks::memequal()
*** SIGSEGV (@0x7) received by PID 167210 (TID 0x7f57b5d00700) from PID 7; stack trace: ***
    @          0x5c32442 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f58f08c7630 (unknown)
    @          0x2dc0ce0 starrocks::memequal()
    @          0x3a6b20a starrocks::vectorized::UnpackConstColumnBinaryFunction<>::evaluate<>()
    @          0x3a6b61c starrocks::vectorized::VectorizedBinaryPredicate<>::evaluate_checked()
    @          0x4045b19 starrocks::vectorized::DictMappingExpr::evaluate_checked()
    @          0x3b7df24 starrocks::vectorized::VectorizedCaseExpr<>::evaluate_no_case()
    @          0x3b84e0d starrocks::vectorized::VectorizedCaseExpr<>::evaluate_checked()
    @          0x394a3b3 starrocks::ExprContext::evaluate()
    @          0x4133af0 starrocks::vectorized::RuntimeFilterProbeCollector::update_selectivity()
    @          0x4134caf starrocks::vectorized::RuntimeFilterProbeCollector::evaluate()
    @          0x2dd4506 starrocks::pipeline::Operator::eval_runtime_bloom_filters()
    @          0x2ddab55 starrocks::pipeline::ScanOperator::pull_chunk()
    @          0x2db7ed0 starrocks::pipeline::PipelineDriver::process()
    @          0x5207fca starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0x4bea3a2 starrocks::ThreadPool::dispatch_thread()
    @          0x4be4e3a starrocks::Thread::supervise_thread()
    @     0x7f58f08bfea5 start_thread
    @     0x7f58efedab0d __clone
    @                0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/50690
Jira
问题版本:
- 2.5.0 ~ 2.5.22
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.10
- 3.3.0 ~ 3.3.3
修复版本:
- 2.5.23+
- 3.0.10+
- 3.1.16+
- 3.2.11+
- 3.3.4+
问题原因:
临时解决办法:
- Set global cbo_enable_low_cardinality_optimize=false
- Set global low_cardinality_optimize_v2=false

trueeyu · 2024年10月24日 12:07

生成列 nullpointer

java.util.ConcurrentModificationException
        at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1511)
        at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1544)
        at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1542)
        at com.starrocks.sql.analyzer.QueryAnalyzer$GeneratedColumnExprMappingCollector.reAnalyzeExpressionBasedOnCurrentScope(QueryAnalyzer.java:196)
        at com.starrocks.sql.analyzer.QueryAnalyzer$GeneratedColumnExprMappingCollector.visitSubquery(QueryAnalyzer.java:239)
        at com.starrocks.sql.analyzer.QueryAnalyzer$GeneratedColumnExprMappingCollector.visitSubquery(QueryAnalyzer.java:130)
        at com.starrocks.sql.ast.SubqueryRelation.accept(SubqueryRelation.java:66)
        at com.starrocks.sql.analyzer.QueryAnalyzer$GeneratedColumnExprMappingCollector.process(QueryAnalyzer.java:135)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSubquery(QueryAnalyzer.java:934)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSubquery(QueryAnalyzer.java:278)
        at com.starrocks.sql.ast.SubqueryRelation.accept(SubqueryRelation.java:66)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:283)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:365)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:278)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/51755
Jira
问题版本:
- 3.2.0 ~ 3.2.11
- 3.3.0 ~ 3.3.5
修复版本:
- 3.2.12+
- 3.3.6+
问题原因:
临时解决办法:

trueeyu · 2024年11月12日 03:13

设置 sql_mode=“ERROR_IF_OVERFLOW” 后，BE crash

*** SIGABRT (@0x3eb00138aa5) received by PID 1280677 (TID 0x7f1e2cdfd640) from PID 1280677; stack trace: ***
    @     0x7f1f29989ee8 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x99ee7)
    @          0x8e6d929 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0x7f1f29932520 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4251f)
    @     0x7f1f299869fc pthread_kill
    @     0x7f1f29932476 raise
    @     0x7f1f299187f3 abort
    @          0xc5f6023 __gnu_cxx::__verbose_terminate_handler() [clone .cold]
    @          0xc5f45fc __cxxabiv1::__terminate(void (*)())
    @          0xc5f4667 std::terminate()
    @          0xc5f47c9 __cxa_throw
    @          0x66ef486 starrocks::DecimalNonDecimalCast<(starrocks::OverflowMode)2, (starrocks::LogicalType)48, (starrocks::LogicalType)9, int, int>::decimal_from(std::shared_ptr<starrocks::Column> const&, int, int)
    @          0x66ef5f0 std::shared_ptr<starrocks::Column> starrocks::UnpackConstColumnUnaryFunction<starrocks::DecimalFrom<(starrocks::OverflowMode)2> >::evaluate<(starrocks::LogicalType)9, (starrocks::LogicalType)48, int const&, int const&>(std::shared_ptr<starrocks::Column> co5^A
    @          0x66ef7b1 std::shared_ptr<starrocks::Column> starrocks::DealNullableColumnUnaryFunction<starrocks::UnpackConstColumnUnaryFunction<starrocks::DecimalFrom<(starrocks::OverflowMode)2> > >::evaluate<(starrocks::LogicalType)9, (starrocks::LogicalType)48, int const&, int 5^A
    @          0x66f049c starrocks::VectorizedCastExpr<(starrocks::LogicalType)9, (starrocks::LogicalType)48, false>::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*)
    @          0x6834aa4 starrocks::VectorizedInConstPredicate<(starrocks::LogicalType)48>::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @          0x5d8a1e1 starrocks::ExprContext::open(starrocks::RuntimeState*)
    @          0x5d8c680 starrocks::Expr::open(std::vector<starrocks::ExprContext*, std::allocator<starrocks::ExprContext*> > const&, starrocks::RuntimeState*)
    @          0x4e4fc86 starrocks::pipeline::ProjectOperatorFactory::prepare(starrocks::RuntimeState*)
    @          0x5009ded starrocks::pipeline::NormalExecutionGroup::prepare_pipelines(starrocks::RuntimeState*)
    @          0x4f42996 starrocks::pipeline::FragmentContext::prepare_all_pipelines()
    @          0x4e2da6e starrocks::pipeline::FragmentExecutor::_prepare_pipeline_driver(starrocks::ExecEnv*, starrocks::pipeline::UnifiedExecPlanFragmentParams const&)
    @          0x4e31acd starrocks::pipeline::FragmentExecutor::prepare(starrocks::ExecEnv*, starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&)
    @          0x7a9ad86 starrocks::PInternalServiceImplBase<starrocks::PInternalService>::_exec_plan_fragment_by_pipeline(starrocks::TExecPlanFragmentParams const&, starrocks::TExecPlanFragmentParams const&)
    @          0x7aa0d95 starrocks::PInternalServiceImplBase<starrocks::PInternalService>::_exec_plan_fragment(brpc::Controller*, starrocks::PExecPlanFragmentRequest const*)
    @          0x7aab7cf starrocks::PInternalServiceImplBase<starrocks::PInternalService>::_exec_plan_fragment(google::protobuf::RpcController*, starrocks::PExecPlanFragmentRequest const*, starrocks::PExecPlanFragmentResult*, google::protobuf::Closure*)
    @          0x7943f0d starrocks::PriorityThreadPool::work_thread(int)
    @          0x8e0f93b thread_proxy

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52752
Jira
问题版本:
- 2.5.0 ~ latest
- 3.0.0 ~ latest
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.5
修复版本:
- 2.5 未修复
- 3.0 未修复
- 3.1.16+
- 3.2.13+
- 3.3.6+
问题原因:
临时解决办法:

trueeyu · 2024年11月12日 03:19

ORC where 条件中有 is null 查询结果不对

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52555
Jira
问题版本:
- 2.5.0 ~ 2.5.22
- 3.0.0 ~ 3.0.9
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.5
修复版本:
- 2.5.23+
- 3.0.10+
- 3.1.16+
- 3.2.13+
- 3.3.6+
问题原因:
临时解决办法:

trueeyu · 2024年11月12日 03:38

Stream Load 报错 text is empty

accept bad request: /bad-request, error: text is empty (possibly HTTP/0.9)

Github Issue:
- https://github.com/StarRocks/starrocks/issues/52516
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52582
Jira
问题版本:
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.5
修复版本:
- 3.1.16+
- 3.2.13+
- 3.3.6+
问题原因:
临时解决办法:

trueeyu · 2024年11月12日 11:29

274. get chunk with sources not supported

W0618 20:10:32.126293 15920 rowset_merger.cpp:573] reader get next error. tablet=9720982, err=Not supported: get chunk with sources not supported

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/47192
Jira
问题版本:
- 3.1.0 ~ 3.1.13
- 3.2.0 ~ 3.2.8
- 3.3.0
修复版本:
- 3.1.14+
- 3.2.9+
- 3.3.1+
问题原因:
临时解决办法:

trueeyu · 2024年11月15日 12:25

BE start 过程中 crash

*** Aborted at 1730782578 (unix time) try "date -d @1730782578" if you are using GNU date ***
PC: @          0x5e7b608 std::_Function_handler<void (), starrocks::init_metrics()::{lambda()#2}>::_M_invoke(std::_Any_data const&)
*** SIGSEGV (@0x10) received by PID 23969 (TID 0x7f269307c700) from PID 16; stack trace: ***
    @     0x7f269cb88997 __pthread_once_slow
    @          0x7a6fc00 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0x7f269cb8b730 (/usr/lib/x86_64-linux-gnu/libpthread-2.28.so+0x1272f)
    @          0x5e7b608 std::_Function_handler<void (), starrocks::init_metrics()::{lambda()#2}>::_M_invoke(std::_Any_data const&)
    @          0x3517746 starrocks::calculate_metrics(void*)
    @          0xc1a8360 execute_native_thread_routine
    @     0x7f269cb80fa3 start_thread
    @     0x7f269c90406f clone

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52765
Jira
问题版本:
- 3.1.0 ~ latest
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.6
修复版本:
- 3.1 未修复
- 3.2.13+
- 3.3.7+
问题原因:
临时解决办法:

trueeyu · 2024年11月15日 12:33

array_map crash

#0  0x000000000612b73c in starrocks::StringFunctions::split (context=0x144631f75c80, columns=...) at be/src/exprs/split.cpp:131
#1  0x00000000051e1129 in starrocks::VectorizedFunctionCallExpr::evaluate_checked (this=0x142147865c00, context=<optimized out>, ptr=0x144114bdb340) at be/src/exprs/function_call_expr.cpp:186
#2  0x0000000004a4361b in starrocks::ArrayElementExpr::evaluate_checked (this=0x142147865700, context=0x140bc3389180, chunk=0x144114bdb340) at be/src/exprs/array_element_expr.cpp:40
#3  0x00000000052a07d7 in starrocks::LambdaFunction::evaluate_checked (this=0x142147865200, context=0x140bc3389180, chunk=0x144114bdb340) at be/src/exprs/expr.h:100
#4  0x000000000497062b in starrocks::ExprContext::evaluate (this=this@entry=0x140bc3389180, e=<optimized out>, chunk=0x144114bdb340, filter=filter@entry=0x0) at be/src/exprs/expr_context.cpp:184
#5  0x00000000052aa476 in starrocks::ArrayMapExpr::evaluate_lambda_expr<false, false> (this=0x142147864d00, context=0x140bc3389180, chunk=0x14524ee8d210, input_elements=..., result_null_column=...) at be/src/common/status.h:211
#6  0x00000000052a3e5c in starrocks::ArrayMapExpr::evaluate_checked (this=0x142147864d00, context=0x140bc3389180, chunk=0x14524ee8d210) at be/src/exprs/array_map_expr.cpp:367
#7  0x000000000497062b in starrocks::ExprContext::evaluate (this=this@entry=0x140bc3389180, e=<optimized out>, chunk=chunk@entry=0x14524ee8d210, filter=filter@entry=0x0) at be/src/exprs/expr_context.cpp:184
#8  0x00000000052a9925 in starrocks::ArrayMapExpr::evaluate_lambda_expr<false, false> (this=0x142147863e00, context=0x140bc3389180, chunk=0x14524ee8d210, input_elements=..., result_null_column=...)
    at be/src/exprs/array_map_expr.cpp:72

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52909
Jira
问题版本:
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.6
修复版本:
- 3.3.7+
- 3.2.13+
问题原因:
临时解决办法:

trueeyu · 2024年11月19日 03:11

Metadata cache crash

*** Aborted at 1729915586 (unix time) try “date -d @1729915586” if you are using GNU date ***
PC: @ 0x551a664 starrocks::MetadataCache::_cache_value_deleter()
*** SIGSEGV (@0x0) received by PID 3337 (TID 0x7f87b7df4700) from PID 0; stack trace: ***
@ 0x6c90c02 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7f905ce6e095 os::Linux::chained_handler()
@ 0x7f905ce73091 JVM_handle_linux_signal
@ 0x7f905ce65e48 signalHandler()
@ 0x7f905c319630 (unknown)
@ 0x551a664 starrocks::MetadataCache::_cache_value_deleter()
@ 0x3032975 starrocks::LRUCache::insert()
@ 0x3032b42 starrocks::ShardedLRUCache::insert()
@ 0x551ad4e starrocks::MetadataCache::cache_rowset()
@ 0x5d133ff starrocks::Rowset::do_load()
@ 0x5d135ef starrocks::Rowset::load()
@ 0x5599114 starrocks::TabletReader::prepare()
@ 0x5d0814d starrocks::VerticalCompactionTask::_compact_column_group()
@ 0x5d08d43 starrocks::VerticalCompactionTask::_vertical_compaction_data()
@ 0x5d094b9 starrocks::VerticalCompactionTask::run_impl()
@ 0x5d02989 starrocks::CompactionTask::run()
@ 0x5605db3 _ZNSt17_Function_handlerIFvvEZN9starrocks17CompactionManager9_scheduleEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x3020b8c starrocks::ThreadPool::dispatch_thread()
@ 0x301a10a starrocks::supervise_thread()
@ 0x7f905c311ea5 start_thread
@ 0x7f905b712b0d __clone
@ 0x0 (unknown)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52968
Jira
问题版本:
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.6
修复版本:
- 3.2.13+
- 3.3.7+
问题原因:
临时解决办法:
- be.conf metadata_cache_memory_limit_percent=0

trueeyu · 2024年11月21日 02:35

Segment flush crash

*** Aborted at 1724803695 (unix time) try "date -d @1724803695" if you are using GNU date ***
PC: @          0x5e2d41e starrocks::SegmentFlushToken::submit(starrocks::DeltaWriter*, brpc::Controller*, starrocks::PTabletWriterAddSegmentRequest const*, starrocks::PTabletWriterAddSegmentResult*, google::protobuf::Closure*)
*** SIGSEGV (@0x38) received by PID 2659 (TID 0x7fb5c7745700) from PID 56; stack trace: ***
    @     0x7fb69de3620b __pthread_once_slow
    @          0x79bda20 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0x7fb69ecaf54f os::Linux::chained_handler(int, siginfo_t*, void*)
    @     0x7fb69ecb53b8 JVM_handle_linux_signal
    @     0x7fb69eca6db8 signalHandler(int, siginfo_t*, void*)
    @     0x7fb69de3f630 (/usr/lib64/libpthread-2.17.so+0xf62f)
    @          0x5e2d41e starrocks::SegmentFlushToken::submit(starrocks::DeltaWriter*, brpc::Controller*, starrocks::PTabletWriterAddSegmentRequest const*, starrocks::PTabletWriterAddSegmentResult*, google::protobuf::Closure*)
    @          0x603fe64 starrocks::AsyncDeltaWriter::write_segment(starrocks::AsyncDeltaWriterSegmentRequest const&)
    @          0x369ed9d starrocks::LocalTabletsChannel::add_segment(brpc::Controller*, starrocks::PTabletWriterAddSegmentRequest const*, starrocks::PTabletWriterAddSegmentResult*, google::protobuf::Closure*)
    @          0x3697901 starrocks::LoadChannel::add_segment(brpc::Controller*, starrocks::PTabletWriterAddSegmentRequest const*, starrocks::PTabletWriterAddSegmentResult*, google::protobuf::Closure*)
    @          0x369056a starrocks::LoadChannelMgr::add_segment(brpc::Controller*, starrocks::PTabletWriterAddSegmentRequest const*, starrocks::PTabletWriterAddSegmentResult*, google::protobuf::Closure*)
    @          0x7c02834 brpc::policy::ProcessRpcRequest(brpc::InputMessageBase*)
    @          0x7ba4db7 brpc::ProcessInputMessage(void*)
    @          0x7ba6135 brpc::InputMessenger::OnNewMessages(brpc::Socket*)
    @          0x7b6e7ae brpc::Socket::ProcessEvent(void*)
    @          0x7b3c4c2 bthread::TaskGroup::task_runner(long)

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/50444
Jira
问题版本:
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.10
- 3.3.0 ~ 3.3.2
修复版本:
- 3.1.16+
- 3.2.11+
- 3.3.3+
问题原因:
临时解决办法:

trueeyu · 2024年11月21日 06:31

表达式分区 swap partition 后查询结果不对

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52894
Jira
问题版本:
- 3.1.0 ~ latest
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.6
修复版本:
- 3.1 未修复
- 3.2.13+
- 3.3.7+
问题原因:
临时解决办法:

trueeyu · 2024年11月21日 06:43

使用 pulsar routine load 内存统计有问题

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/50899
Jira
问题版本:
- 3.1.0 ~ 3.1.15
- 3.2.0 ~ 3.2.11
- 3.3.0 ~ 3.3.3
修复版本:
- 3.1.16+
- 3.2.12+
- 3.3.4+
问题原因:
临时解决办法:

trueeyu · 2024年11月22日 06:17

Random 分区表 failed to get tablet

failed to get tablet. tablet_id=60274, with schema_hash=1996401921, reason=tablet does not exist backend [id=10002] [host=starrocks01]

Github Issue:
Github Fix PR:
- https://github.com/StarRocks/starrocks/pull/52258
Jira
问题版本:
- 3.2.0 ~ 3.2.12
- 3.3.0 ~ 3.3.5
修复版本:
- 3.2.13+
- 3.3.6+
问题原因:
临时解决办法:

常见 Crash / BUG / 优化 查询

Rocksdb Tombstone 占用大量 CPU

Not found dict for cid:

Thrift Server 线程泄漏，占用大量线程不释放

Join reorder failed

系统表查询 crash

BE has overloaded with pipeline drivers

Partition TOPN crash

RuntimeFilter 下推导致 crash

生成列 nullpointer

设置 sql_mode=“ERROR_IF_OVERFLOW” 后，BE crash

ORC where 条件中有 is null 查询结果不对

Stream Load 报错 text is empty

274. get chunk with sources not supported

BE start 过程中 crash

array_map crash

Metadata cache crash

Segment flush crash

表达式分区 swap partition 后查询结果不对

使用 pulsar routine load 内存统计有问题

Random 分区表 failed to get tablet

常见 Crash / BUG / 优化查询