配置项
| Option | Required | Default | Type | Description |
|---|---|---|---|---|
| connector | YES | NONE | String | starrocks |
| scan-url | YES | NONE | String | Hosts of the fe node like: fe_ip1:http_port,fe_ip2:http_port.... |
| jadc-url | YES | NONE | String | Hosts of the fe node like: fe_ip1:query_port,fe_ip2: query_port.... |
| username | YES | NONE | String | StarRocks user name. |
| password | YES | NONE | String | StarRocks user password. |
| database-name | YES | NONE | String | Database name |
| table-name | YES | NONE | String | Table name |
| scan.connect.timeout-ms | NO | 1000 | String | Connect timeout |
| scan.params.properties | NO | NONE | String | Reserved params for use |
| scan.params.keep-alive-min | NO | 10 | String | Max keep alive time min |
| scan.params.query-timeout-s | NO | 600(5min) | String | Query timeout for a single query(The value of this parameter needs to be longer than the estimated period of the source) |
| scan.params.mem-limit-byte | NO | 102410241024(1G) | String | Memory limit for a single query |
| scan.max-retries | NO | 1 | String | Max request retry times. |
使用示例
CREATE TABLE flink_test (
date_1 DATE,
datetime_1 TIMESTAMP(6),
char_1 CHAR(20),
varchar_1 VARCHAR,
boolean_1 BOOLEAN,
tinyint_1 TINYINT,
smallint_1 SMALLINT,
int_1 INT,
bigint_1 BIGINT,
largeint_1 STRING,
float_1 FLOAT,
double_1 DOUBLE,
decimal_1 DECIMAL(27,9)
) WITH (
'connector'='starrocks',
'scan-url'='172.26.92.152:8030,172.26.92.153:8030,172.26.92.154:8030',
'jdbc-url'='jdbc:mysql://172.26.92.152:9030',
'username'='root',
'password'='',
'database-name'='flink_test',
'table-name'='flink_test'
);
select date_1, smallint_1 from flink_test where char_1 <> 'beijing' and int_1 = -126
使用说明
- 请不要在flink sql查询语句中加入 limit(会在 StarRocks 中扫全表返回全量数据后，由flink执行 limit 过滤)。
- 请确保各个参数中 port 配置的正确性
- 目前不支持聚合查询
- 目前在任务失败的情况下，无法保证数据一致性
flink-connector-starrocks-1.1.14-snapshot_flink-1.13_2.11.jar (8.8 MB)
flink-connector-starrocks-1.1.14-snapshot_flink-1.12_2.11.jar (8.8 MB)