【报错：Fail to get file status: listObjects() on oss://*/user/hive/warehouse/ods.db/arch_account_type_all_temp4: software.amazon.awssdk.services.s3.model.S3Exception: The specified bucket is not valid. :InvalidBucketName: The specified bucket is not valid.】

💬 StarRocks 用户问答
#1

–oss-hdfs的文件导入标签sr
–sr建表
CREATE TABLE test.arch_account_type2
(
id int(11) NOT NULL COMMENT “主键”,
name varchar(65533) NULL COMMENT “来源类型名称”,
url varchar(65533) NULL COMMENT “来源类型的图片的url”,
create_time datetime NULL COMMENT “创建时间”,
update_time datetime NULL COMMENT “更新时间”,
kudu_time datetime NULL COMMENT “kudu存储时间”,
kafka_time datetime NULL COMMENT “kafka发送时间”
) ENGINE=OLAP
PRIMARY KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES (
“compression” = “LZ4”,
“enable_persistent_index” = “true”,
“fast_schema_evolution” = “true”,
“replicated_storage” = “true”,
“replication_num” = “1”
);

2、use test;
LOAD LABEL arch_account_type_003912
(
DATA INFILE(“oss:///user/hive/warehouse/ods.db/arch_account_type_all_temp4/*")
INTO TABLE arch_account_type2
format as “parquet”
)
WITH BROKER
(
“aws.s3.enable_ssl” = “false”,
“aws.s3.enable_path_style_access” = “true”,
“aws.s3.endpoint” = "”,
“aws.s3.access_key” = “",
“aws.s3.secret_key” = "”
);

show load where label=‘arch_account_type_003912’;

报错信息：
– 报错：type:ETL_RUN_FAIL; msg:Fail to get file status: listObjects() on oss://*/user/hive/warehouse/ods.db/arch_account_type_all_temp4: software.amazon.awssdk.services.s3.model.S3Exception: The specified bucket is not valid. :InvalidBucketName: The specified bucket is not valid.

【报错：The arguments of blob store(S3/Azure) may be wrong. You can check the arguments like region, IAM, instance profile and so on.】
#2

bucket 名称确认是正确的不，使用ossutil64访问这个路径可以查看文件么

#3

增加配置已完成（并重启了FE）
fe conf/core-site.xml 配下

fs.oss.impl
org.apache.hadoop.fs.s3a.S3AFileSystem


fs.AbstractFileSystem.oss.impl
org.apache.hadoop.fs.s3a.S3A

#4

这个oss是oss-hdfs,（来自阿里官网：OSS-HDFS服务完全兼容HDFS接口，同时支持目录层级的操作(
OSS-HDFS服务的能力，像使用Hadoop分布式文件系统(HDFS)一样管理和访问数据。）

#5

操作过程为：
1、云上使用阿里云sr（存算分离模式），通过sr表生成oss-hdfs表（create table ods. arch_account_type_all_temp4 as select * from arch_account_type）.
2、开源版本sr（存算一体模式：v3.3.1）创建表test.arch_account_type2。
3、开源版本sr（存算一体模式：v3.3.1）执行
LOAD LABEL arch_account_type_003912
(
DATA INFILE(“oss:// /user/hive/warehouse/ods.db/arch_account_type_all_temp4/*")
INTO TABLE arch_account_type2
format as “parquet”
)
WITH BROKER
(
“aws.s3.enable_ssl” = “false”,
“aws.s3.enable_path_style_access” = “true”,
“aws.s3.endpoint” = " ”,
“aws.s3.access_key” = “ ",
“aws.s3.secret_key” = " ”
);

show load where label=‘arch_account_type_003912’;

就出现报错了

#6

use test;
LOAD LABEL arch_account_type_003913
(
DATA INFILE(“oss:///user/hive/warehouse/ods.db/arch_account_type_all_temp4/*")
INTO TABLE arch_account_type2
format as “parquet”
)
WITH BROKER
(
“aws.s3.enable_ssl” = “false”,
“aws.s3.enable_path_style_access” = “true”,
“aws.s3.endpoint” = "”,
“aws.s3.access_key” = “",
“aws.s3.secret_key” = "”,
“aws.s3.bucket” = “*****”
);

show load where label=‘arch_account_type_003913’;
报错一样的

京ICP备2022026421号-1