Starrocks3.1.14无法查询es

为了更快的定位您的问题,请提供以下信息,谢谢
【详述】问题详细描述
【背景】做过哪些操作?
【业务影响】sr无法作为统一查询引擎查询es
【是否存算分离】是
【StarRocks版本】例如:3.1.14
【集群规模】例如:1fe + 1cn + 腾讯cos对象存储

希望sr catalog查询es,自建单节点,单fe,单cn,存算分离。

es版本。腾讯元的es6.4.3

CREATE EXTERNAL CATALOG es43xerr_catalog
comment “es43xx_catalog”
PROPERTIES (“enable_docvalue_scan” = “true”,
“es.nodes.wan.only” = “false”,
“hosts” = “10.10.xxxx:9200”,
“es.net.ssl” = “false”,
“type” = “es”,
“user” = “elastic”,
“password” = “xxxxxxxx”,
“enable_keyword_sniff” = “true”
)

set catalog es43xerr_catalog;
use default_db;
show tables;

MySQL [default_db]> show tables;
ERROR 1064 (HY000): es indexes are null, maybe this is error

报错,fe.log也没有具体的报错原因。

修改源码,增加打印。

private String execute(String path) throws StarRocksConnectorException {
    int retrySize = nodes.length;
    StarRocksConnectorException scratchExceptionForThrow = null;
    OkHttpClient client;
    if (sslEnabled) {
        client = getOrCreateSSLClient();
    } else {
        client = NETWORK_CLIENT;
    }
    for (int i = 0; i < retrySize; i++) {
        // maybe should add HTTP schema to the address
        // actually, at this time we can only process http protocol
        // NOTE. currentNode may have some spaces.
        // User may set a config like described below:
        // hosts: "http://192.168.0.1:8200, http://192.168.0.2:8200"
        // then currentNode will be "http://192.168.0.1:8200", " http://192.168.0.2:8200"
        LOG.warn("CUSTOM_LOG: currentNode = " + currentNode);

        currentNode = currentNode.trim();
        if (!(currentNode.startsWith("http://") || currentNode.startsWith("https://"))) {
            currentNode = "http://" + currentNode;
        }
        Request request = builder.get()
                .url(currentNode + "/" + path)      //path  /_cat/indices?h=index&format=json&s=index:asc
                .build();
        Response response = null;
        if (LOG.isTraceEnabled()) {
            LOG.trace("es rest client request URL: {}", currentNode + "/" + path);
        }
        try {
            response = client.newCall(request).execute();
            LOG.warn("CUSTOM_LOG: response = " + response);
            LOG.warn("CUSTOM_LOG: response.isSuccessful = " + response.isSuccessful());
            LOG.warn("CUSTOM_LOG: response.body = " + response.body().string());
            if (response.isSuccessful()) {
                return response.body().string();
            }
        } catch (IOException e) {
            LOG.warn("request node [{}] [{}] failures {}, try next nodes", currentNode, path, e);
            scratchExceptionForThrow = new StarRocksConnectorException(e.getMessage());
        } finally {
            if (response != null) {
                response.close();
            }
        }
        selectNextNode();
    }
    LOG.warn("try all nodes [{}],no other nodes left", nodes);
    if (scratchExceptionForThrow != null) {
        throw scratchExceptionForThrow;
    }
    LOG.warn("CUSTOM_LOG: return = null");
    return null;
}

====

fe异常日志:

2024-08-19 14:34:29.641+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():211] CUSTOM_LOG: currentNode = 10.10.0.43:9200
2024-08-19 14:34:29.724+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():226] CUSTOM_LOG: response = Response{protocol=http/1.1, code=401, message=Unauthorized, url=http://10.10.0.43:9200/_cat/indices?h=index&format=json&s=index:asc}
2024-08-19 14:34:29.725+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():227] CUSTOM_LOG: response.isSuccessful = false
2024-08-19 14:34:29.725+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():228] CUSTOM_LOG: response.body = {“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“unable to authenticate user [elastic] for REST request [/_cat/indices?h=index&format=json&s=index:asc]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}}],“type”:“security_exception”,“reason”:“unable to authenticate user [elastic] for REST request [/_cat/indices?h=index&format=json&s=index:asc]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}},“status”:401}

另外,验证了es的密码无论写对写错,都报这个异常。

但是通过curl命令封装的话

curl -u ‘elastic:xxxxx’ ‘http://10.1xxxx:9200/_cat/indices?h=index&format=json&s=index:asc

这个shell脚本还是有返回索引列表的。

https://github.com/StarRocks/starrocks/issues/49960

git issue