【详述】我这边用java连SR查一张视图嵌套视图的表(可以理解为执行了一条复杂sql,占用内存600mb左右),场景是会同时调用相同接口四五十次,也就相当于执行该复杂sql几十次,然后报内存超限
java.sql.SQLSyntaxErrorException: Memory of process exceed limit. Pipeline Backend: 10.9.2.*, fragment: 0e0d76d7-6e4b-11ee-babc-024206dd4917 Used: 26384823888, Limit: 26383056444. Mem usage has exceed the limit of BE
【采用手段】
1、开启队列,配置相关参数配置,但是没生效,没达到队列预期。
2、修改相关并发限制,有些失效,有些会出现SQLSyntaxErrorException: Need be queued but exceed query queue capacity
【集群相关配置】
1、队列配置(不管默认还是配置一下参数都无效果)
query_queue_concurrency_limit = 20
query_queue_mem_used_pct_limit = 0.6
query_queue_cpu_used_permille_limit = 700
query_queue_max_queued_queries=20
2、BE的metric相关配置
starrocks_be_bitmap_index_mem_bytes 0
starrocks_be_bloom_filter_index_mem_bytes 0
starrocks_be_chunk_allocator_mem_bytes 104202064
starrocks_be_clone_mem_bytes 0
starrocks_be_column_metadata_mem_bytes 106895404
starrocks_be_column_pool_mem_bytes 0
starrocks_be_column_zonemap_index_mem_bytes 28529900
starrocks_be_compaction_mem_bytes 7337048
starrocks_be_consistency_mem_bytes 0
starrocks_be_load_mem_bytes 0
starrocks_be_metadata_mem_bytes 238091481
starrocks_be_ordinal_index_mem_bytes 33696992
starrocks_be_process_mem_bytes 3942574576
starrocks_be_query_mem_bytes 0
starrocks_be_rowset_metadata_mem_bytes 38038091
starrocks_be_schema_change_mem_bytes 0
starrocks_be_segment_metadata_mem_bytes 17984649
starrocks_be_segment_zonemap_mem_bytes 15114983
starrocks_be_short_key_index_mem_bytes 301203
starrocks_be_storage_page_cache_mem_bytes 0
starrocks_be_tablet_metadata_mem_bytes 75173337
starrocks_be_tablet_schema_mem_bytes 2538609
starrocks_be_update_mem_bytes 1324503030
1、版本: version 2.5.10 RELEASE (build 9feb716a34) Built on 2023-08-06 07:27:09 by StarRocks@localhost
2、FE:3台,8核 16GB
3、BE:3台,8核 32GB
4、测试场景:同一秒并发执行40个sql,其中一个sql预算占600mb, 600mb*40=24g,mem_limit是90,40个sql全部报错Memory of process exceed limit.
5、开启队列没有生效