export导出数据至MiniO

[一剑飘血]

2.2.x版本，按S3方式配置，使用broker导出数据至minio，apache_hdfs_broker.log 部分报错如图，全部报错在附件。
apache_hdfs_broker.zip (8.2 KB) 。

mysql> show EXPORT\G
*************************** 1. row ***************************
JobId: 1571604
QueryId: 1ee160a0-8819-11ed-9e81-525400ec31ae
State: CANCELLED
Progress: 100%
TaskInfo: {“partitions”:[“p20221121”,“p20221122”],“column separator”:",",“columns”:["*"],“tablet num”:8,“broker”:“broker1”,“coord num”:1,“db”:“default_cluster:ods”,“tbl”:“ods_y_pc_award_num1”,“row delimiter”:"\n",“mem limit”:2147483648}
Path: s3a://starrocks/
CreateTime: 2022-12-30 16:08:19
StartTime: 2022-12-30 16:08:24
FinishTime: 2022-12-30 16:30:58
Timeout: 3600
ErrorMsg: type:RUN_FAIL; msg:export job fail. query id: deb90db4-8ca5-4254-8e84-f7f82081a264, fail msg: No broker permission, java.net.SocketTimeoutException: doesBucketExist on starrocks: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider InstanceProfileCredentialsProvider : com.amazonaws.SdkClientException: Failed to connect to service endpoint: , cause by: doesBucketExist on starrocks: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider InstanceProfileCredentialsProvider : com.amazonaws.SdkClientException: Failed to connect to service endpoint:
1 row in set (0.01 sec)

[一剑飘血]

可以帮忙看下吗？MiniO是兼容S3协议的 为啥遇到这个报错呢？

[一剑飘血]

有人帮忙看看吗？

[dongquan]

看上去是权限认证有问题，权限如果是公开的话，2.2.最新版本可以在broker中添加配置
WITH BROKER “broker1” (
…
“fs.s3a.aws.credentials.provider” = “org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider”,
…
);

[一剑飘血]

apache_hdfs_broker.log (102.7 KB) 还是报错，是2.2.5版本太老不支持吗？

[dongquan]

2.2的最新版本是2.2.9，下载链接如下：
wget “http://cdn-release.starrocks.com/StarRocks-2.2.9.tar.gz?OSSAccessKeyId=LTAI5tLYAjAmZTbSaqwYLBcD&Expires=1673088226&Signature=ViaYmqofwvXTvpIn%2Bz6hCCKsZcs%3D” -O StarRocks-2.2.9.tar.gz

[U_1642562074482_9550]

这个问题解决了吗，我在V3.0也有这个问题，不知道startRock是否支持minio导出

[Doni]

是支持 minio 的，这边用的 3.0 哪个版本
导出语句最后的配置 新增这两个配置 看一下还会不会报错，如果还报错的话 贴一下报错信息
“aws.s3.enable_path_style_access”=“true”,
“aws.s3.enable_ssl”=“true”

[U_1642562074482_9550]

加上这个配置可以了，非常感谢