两个表的左外联接无响应

为了更快的定位您的问题,请提供以下信息,谢谢
【详述】问题详细描述
两个表左外联接,系统无响应。
select *from t_org_info a left join t_org_info b on a.parent_id = b.parent_id
这个表是主键表,parent_id不是主键,也没有按parent_id分桶,parent_id就是一个普通的字段。建表语句中的参数配置如下:
ENGINE=OLAP

PRIMARY KEY(org_id)

DISTRIBUTED BY HASH(org_id)

PROPERTIES (

“replication_num” = “3”,

“in_memory” = “false”,

“enable_persistent_index” = “true”,

“replicated_storage” = “true”,

“compression” = “LZ4”

);

【背景】做过哪些操作?
系统从3.2.2升级到3.2.4。按照官方文档说明升的级。
【业务影响】
starrocks完全无响应,系统瘫痪。
【是否存算分离】

【StarRocks版本】例如:
3.2.4
【集群规模】例如:3fe(1 follower+2observer)+5be(fe与be混部)
1fe +3be
【机器信息】CPU虚拟核/内存/网卡,例如:48C/64G/万兆
Intel® Xeon® CPU E5-2620 v3 @ 2.40GHz/16G/
【联系方式】为了在解决问题过程中能及时联系到您获取一些日志信息,请补充下您的联系方式,例如:社区群4-小李或者邮箱,谢谢
Starrocks社区群18-Mathematics
【附件】

  • fe.log/beINFO/相应截图
  • 慢查询:
    • Profile信息
    • 并行度:show variables like ‘%parallel_fragment_exec_instance_num%’;
    • pipeline是否开启:show variables like ‘%pipeline%’;
    • be节点cpu和内存使用率截图
  • 查询报错:
  • be crash
    • be.out
  • 外表查询报错
    • be.out和fe.warn.log

show backends 和show frontends 结果看下,fe和be状态是正常的么

前后端信息.xlsx (11.9 KB)

麻烦您看下。

fe占用cpu资源会升高,不正常,其他的参数不知道怎么看了。be是正常的,我重启fe之后就会变好。

还有就是,我怀疑这个问题是异步物化视图引起的,我在数据库里建了很多异步物化视图,今天把异步物化视图删了,就好了。麻烦您帮忙查一下具体原因,因为我们还是需要用异步物化视图的。

另外我在单表查询时,爆了一个异常,也是跟异步物化视图相关的:
==> fe.log <==
2024-06-25 14:22:22,322 WARN (starrocks-mysql-nio-pool-1|204) [CachingMvPlanContextBuilder.loadMvPlanContext():78] load mv plan cache failed: v_123
com.starrocks.sql.analyzer.SemanticException: Getting analyzing error. Detail message: Column ‘a.b.c’ cannot be resolved.
at com.starrocks.sql.analyzer.Scope.resolveField(Scope.java:96) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitSlot(SqlToScalarOperatorTranslator.java:303) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitSlot(SqlToScalarOperatorTranslator.java:256) ~[starrocks-fe.jar:?]
at com.starrocks.analysis.SlotRef.accept(SlotRef.java:510) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visit(SqlToScalarOperatorTranslator.java:289) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator.translate(SqlToScalarOperatorTranslator.java:183) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator.translate(SqlToScalarOperatorTranslator.java:157) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.project(QueryTransformer.java:236) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.project(QueryTransformer.java:226) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.plan(QueryTransformer.java:129) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:258) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.processSetOperation(RelationTransformer.java:293) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitUnion(RelationTransformer.java:263) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitUnion(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.UnionRelation.accept(UnionRelation.java:31) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.transform(RelationTransformer.java:208) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:659) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SubqueryRelation.accept(SubqueryRelation.java:66) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.planFrom(QueryTransformer.java:165) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.plan(QueryTransformer.java:80) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:258) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.transform(RelationTransformer.java:208) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.SubqueryUtils.getLogicalPlan(SubqueryUtils.java:113) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitExistsPredicate(SqlToScalarOperatorTranslator.java:466) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitExistsPredicate(SqlToScalarOperatorTranslator.java:256) ~[starrocks-fe.jar:?]
at com.starrocks.analysis.ExistsPredicate.accept(ExistsPredicate.java:118) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visit(SqlToScalarOperatorTranslator.java:289) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitCompoundPredicate(SqlToScalarOperatorTranslator.java:425) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visitCompoundPredicate(SqlToScalarOperatorTranslator.java:256) ~[starrocks-fe.jar:?]
at com.starrocks.analysis.CompoundPredicate.accept(CompoundPredicate.java:154) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator$Visitor.visit(SqlToScalarOperatorTranslator.java:289) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.SqlToScalarOperatorTranslator.translate(SqlToScalarOperatorTranslator.java:183) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.filter(QueryTransformer.java:259) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.plan(QueryTransformer.java:91) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:258) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.transform(RelationTransformer.java:208) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:659) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SubqueryRelation.accept(SubqueryRelation.java:66) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitJoin(RelationTransformer.java:779) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitJoin(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.JoinRelation.accept(JoinRelation.java:134) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.planFrom(QueryTransformer.java:165) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.plan(QueryTransformer.java:80) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:258) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.transform(RelationTransformer.java:208) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:659) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSubquery(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SubqueryRelation.accept(SubqueryRelation.java:66) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitJoin(RelationTransformer.java:779) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitJoin(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.JoinRelation.accept(JoinRelation.java:134) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.planFrom(QueryTransformer.java:165) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.QueryTransformer.plan(QueryTransformer.java:80) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:258) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.visitSelect(RelationTransformer.java:141) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:67) ~[starrocks-fe.jar:?]
at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:63) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.transformer.RelationTransformer.transform(RelationTransformer.java:208) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.rule.transformation.materialization.MvUtils.getRuleOptimizedLogicalPlan(MvUtils.java:434) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MaterializedViewOptimizer.optimize(MaterializedViewOptimizer.java:56) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MaterializedViewOptimizer.optimize(MaterializedViewOptimizer.java:31) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MvPlanContextBuilder.getPlanContext(MvPlanContextBuilder.java:36) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.CachingMvPlanContextBuilder.loadMvPlanContext(CachingMvPlanContextBuilder.java:76) ~[starrocks-fe.jar:?]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406) ~[caffeine-2.9.3.jar:?]
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916) ~[?:?]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404) ~[caffeine-2.9.3.jar:?]
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387) ~[caffeine-2.9.3.jar:?]
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108) ~[caffeine-2.9.3.jar:?]
at com.github.benmanes.caffeine.cache.LocalManualCache.get(LocalManualCache.java:62) ~[caffeine-2.9.3.jar:?]
at com.starrocks.sql.optimizer.CachingMvPlanContextBuilder.getPlanContext(CachingMvPlanContextBuilder.java:61) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MvRewritePreprocessor.getMVWithContext(MvRewritePreprocessor.java:458) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MvRewritePreprocessor.getMvWithPlanContext(MvRewritePreprocessor.java:584) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.MvRewritePreprocessor.prepare(MvRewritePreprocessor.java:250) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.Optimizer.prepare(Optimizer.java:283) ~[starrocks-fe.jar:?]
at com.starrocks.sql.optimizer.Optimizer.optimize(Optimizer.java:142) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.createQueryPlan(StatementPlanner.java:187) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:130) ~[starrocks-fe.jar:?]
at com.starrocks.sql.StatementPlanner.plan(StatementPlanner.java:87) ~[starrocks-fe.jar:?]
at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:520) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:395) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:589) ~[starrocks-fe.jar:?]
at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:883) ~[starrocks-fe.jar:?]
at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:842) ~[?:?]

提供一个querydump

麻烦网上翻,我提供了,在excel里。

建议升级到最新的3.2版本。3.2.4版本上这个版本没有捕获住,可能会导致升级失败。

改写异常应该是另外一个问题,但这个理论上不影响正常的query,可以试下set enable_materialized_view_rewrite=false 关闭改写试下

没有看懂,没有捕获住是什么意思?

改了这个配置,改成false查询就好了,改成true就没有响应。