为了更快的定位您的问题,请提供以下信息,谢谢
【详述】问题详细描述
【背景】做过哪些操作?
【业务影响】sr无法作为统一查询引擎查询es
【是否存算分离】是
【StarRocks版本】例如:3.1.14
【集群规模】例如:1fe + 1cn + 腾讯cos对象存储
希望sr catalog查询es,自建单节点,单fe,单cn,存算分离。
es版本。腾讯元的es6.4.3
CREATE EXTERNAL CATALOG es43xerr_catalog
comment “es43xx_catalog”
PROPERTIES (“enable_docvalue_scan” = “true”,
“es.nodes.wan.only” = “false”,
“hosts” = “10.10.xxxx:9200”,
“es.net.ssl” = “false”,
“type” = “es”,
“user” = “elastic”,
“password” = “xxxxxxxx”,
“enable_keyword_sniff” = “true”
)
set catalog es43xerr_catalog;
use default_db;
show tables;
MySQL [default_db]> show tables;
ERROR 1064 (HY000): es indexes are null, maybe this is error
报错,fe.log也没有具体的报错原因。
修改源码,增加打印。
private String execute(String path) throws StarRocksConnectorException {
int retrySize = nodes.length;
StarRocksConnectorException scratchExceptionForThrow = null;
OkHttpClient client;
if (sslEnabled) {
client = getOrCreateSSLClient();
} else {
client = NETWORK_CLIENT;
}
for (int i = 0; i < retrySize; i++) {
// maybe should add HTTP schema to the address
// actually, at this time we can only process http protocol
// NOTE. currentNode may have some spaces.
// User may set a config like described below:
// hosts: "http://192.168.0.1:8200, http://192.168.0.2:8200"
// then currentNode will be "http://192.168.0.1:8200", " http://192.168.0.2:8200"
LOG.warn("CUSTOM_LOG: currentNode = " + currentNode);
currentNode = currentNode.trim();
if (!(currentNode.startsWith("http://") || currentNode.startsWith("https://"))) {
currentNode = "http://" + currentNode;
}
Request request = builder.get()
.url(currentNode + "/" + path) //path /_cat/indices?h=index&format=json&s=index:asc
.build();
Response response = null;
if (LOG.isTraceEnabled()) {
LOG.trace("es rest client request URL: {}", currentNode + "/" + path);
}
try {
response = client.newCall(request).execute();
LOG.warn("CUSTOM_LOG: response = " + response);
LOG.warn("CUSTOM_LOG: response.isSuccessful = " + response.isSuccessful());
LOG.warn("CUSTOM_LOG: response.body = " + response.body().string());
if (response.isSuccessful()) {
return response.body().string();
}
} catch (IOException e) {
LOG.warn("request node [{}] [{}] failures {}, try next nodes", currentNode, path, e);
scratchExceptionForThrow = new StarRocksConnectorException(e.getMessage());
} finally {
if (response != null) {
response.close();
}
}
selectNextNode();
}
LOG.warn("try all nodes [{}],no other nodes left", nodes);
if (scratchExceptionForThrow != null) {
throw scratchExceptionForThrow;
}
LOG.warn("CUSTOM_LOG: return = null");
return null;
}
====
fe异常日志:
2024-08-19 14:34:29.641+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():211] CUSTOM_LOG: currentNode = 10.10.0.43:9200
2024-08-19 14:34:29.724+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():226] CUSTOM_LOG: response = Response{protocol=http/1.1, code=401, message=Unauthorized, url=http://10.10.0.43:9200/_cat/indices?h=index&format=json&s=index:asc}
2024-08-19 14:34:29.725+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():227] CUSTOM_LOG: response.isSuccessful = false
2024-08-19 14:34:29.725+08:00 WARN (starrocks-mysql-nio-pool-3|200) [EsRestClient.execute():228] CUSTOM_LOG: response.body = {“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“unable to authenticate user [elastic] for REST request [/_cat/indices?h=index&format=json&s=index:asc]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}}],“type”:“security_exception”,“reason”:“unable to authenticate user [elastic] for REST request [/_cat/indices?h=index&format=json&s=index:asc]”,“header”:{“WWW-Authenticate”:"Basic realm=“security” charset=“UTF-8"”}},“status”:401}
另外,验证了es的密码无论写对写错,都报这个异常。
但是通过curl命令封装的话
curl -u ‘elastic:xxxxx’ ‘http://10.1xxxx:9200/_cat/indices?h=index&format=json&s=index:asc’
这个shell脚本还是有返回索引列表的。