【Bug】4.1.1 使用Paimon Catalog查询带主键的Paimon表报错 All columns must be nullable for external table

💬 StarRocks 用户问答
#1

【详述】通过Paimon Catalog访问带primary key的Paimon主键表，执行SELECT直接报错：
All columns must be nullable for external table. Column ‘id’ is not nullable … BE:11001
【背景】做过哪些操作？
【业务影响】是
【是否存算分离】否
【StarRocks版本】4.1.1-14b7e3f
【集群规模】1fe+1be（fe与be混部）
【机器信息】CPU虚拟核/内存/网卡，例如：48C/64G/万兆
【复现步骤】

  1. Flink SQL 创建Paimon主键表，DDL字段不加NOT NULL，仅配置primary key(id)
    CREATE CATALOG IF NOT EXISTS paimon_catalog WITH (
    ‘type’ = ‘paimon’,
    ‘warehouse’ = ‘s3://paimon-warehouse/’,
    ‘s3.endpoint’ = ‘http://xx:9000’,
    ‘s3.access-key’ = ‘xxx’,
    ‘s3.secret-key’ = ‘xxxx’,
    ‘s3.path.style.access’ = ‘true’
    );
    USE CATALOG paimon_catalog;
    CREATE DATABASE IF NOT EXISTS cdc_db;
    USE cdc_db;
    CREATE TEMPORARY TABLE mysql_orders_cdc (
    id BIGINT,
    order_no STRING,
    customer_name STRING,
    amount DECIMAL(10, 2),
    status STRING,
    create_time TIMESTAMP(3),
    update_time TIMESTAMP(3),
    PRIMARY KEY (id) NOT ENFORCED
    ) WITH (
    ‘connector’ = ‘mysql-cdc’,
    ‘hostname’ = ‘xxx’,
    ‘port’ = ‘3306’,
    ‘username’ = ‘root’,
    ‘password’ = ‘xxx’,
    ‘database-name’ = ‘cdc_test’,
    ‘table-name’ = ‘orders’,
    ‘scan.startup.mode’ = ‘initial’,
    ‘server-time-zone’ = ‘Asia/Shanghai’
    );
    CREATE TABLE IF NOT EXISTS paimon_orders (
    id BIGINT,
    order_no STRING,
    customer_name STRING,
    amount DECIMAL(10, 2),
    status STRING,
    create_time TIMESTAMP(3),
    update_time TIMESTAMP(3),
    PRIMARY KEY (id) NOT ENFORCED
    ) WITH (
    ‘connector’ = ‘paimon’,
    ‘file.compression’ = ‘zstd’
    );
    INSERT INTO paimon_orders
    SELECT id, order_no, customer_name, amount, status, create_time, update_time
    FROM mysql_orders_cdc;
  2. StarRocks 创建Paimon Filesystem Catalog指向s3 warehouse
    CREATE EXTERNAL CATALOG paimon_catalog
    PROPERTIES
    (
    “type” = “paimon”,
    “paimon.catalog.type” = “filesystem”,
    “paimon.catalog.warehouse” = “s3://paimon-warehouse/”,
    “aws.s3.enable_ssl” = “true”,
    “aws.s3.enable_path_style_access” = “true”,
    “aws.s3.endpoint” = “http://xxx:9000”,
    “aws.s3.access_key” = “xx”,
    “aws.s3.secret_key” = “xx”
    );
  3. 查询语句：
    SELECT * FROM paimon_catalog.cdc_db.paimon_orders;
    【联系方式】418736719@qq.com
    【附件】

京ICP备2022026421号-1