Routine 任务连接kafka失败

💬 StarRocks 用户问答 安装部署与监控
#1

【详述】Routine 任务连接kafka失败
【StarRocks版本】3.4.0
【联系方式】vfight@hotmail.com

docker安装starrocks和kafka
创建 Routine任务导入数据，实测在starrocks的docker内可以连接到kafka的docker，IP端口配置都没问题，但是运行任务报错，什么情况
CREATE ROUTINE LOAD task_sync_status ON sync_status
COLUMNS(chain, blocknumber, blocktime, sol_price)
PROPERTIES
(
“format” = “json”,
“jsonpaths” = “[”$.chain","$.blocknumber","$.blocktime","$.sol_price"]"
)
FROM KAFKA
(
“kafka_broker_list” = “172.17.0.3:9092”,
“kafka_topic” = “rsquantex_sync_status”,
“property.kafka_default_offsets” = “OFFSET_BEGINNING”
);

ErrorReason{errCode = 2, msg=‘Failed to send get kafka partition info request to BE TNetworkAddress(hostname:127.0.0.1, port:8060), err: Ocurrs time out with specfied time 12 SECONDS’}

我特意测试配错kafka的IP，会报另一个错：
ErrorReason{errCode = 4, msg=‘Job failed to fetch all current partition with error [failed to get kafka topic: rsquantex_sync_status meta, err: Local: Broker transport failure, event: [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT), BE: TNetworkAddress(hostname:127.0.0.1, port:8060)]’}

#2

请问您的问题解决了吗？我也遇到同样的问题，查看be的日志发现一直在连127.0.0.1:9092，而不是制定的broker_list

#3

创建Kafka时换成具体ip不要用localhost试试呢？

#4

同样存在这个问题, 查看BE日之后后, 还是链接127.0.0.1:9092, 而不是配置中的ip,
我将be的环境的127.0.0.1:9092 转发到目标的kafka server后, 数据接入正常了

apt update
apt install socat
socat TCP-LISTEN:9092,bind=127.0.0.1,fork TCP:172.31.96.1:9092

2025年9月27日18:16:53更新

通过调整kafka服务的配置也解决了

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.6.7
    container_name: my-zookeeper
    restart: unless-stopped
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - "2181:2181"

  kafka:
    image: confluentinc/cp-kafka:7.6.7 # Pin to a ZooKeeper-compatible version
    container_name: my-kafka
    restart: unless-stopped
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://172.31.96.1:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      # Explicitly disable KRaft mode
      KAFKA_NODE_ROLE: broker  # Optional, for older Confluent images

京ICP备2022026421号-1