为了更快的定位您的问题,请提供以下信息,谢谢
【详述】重新搭建了一个环境,除了starrocks版本不同,数据库中的表数据结构完全一样。将旧环境中使用的datax脚本全部复制迁移到新环境中,在新环境执行datax的python脚本,该脚本进行插入数据的操作,会一直报错,只有在starrocks版本为3.0.2时能够正常执行
【StarRocks版本】旧环境3.0.2 新环境版本3.0.8
datax报错日志
2023-11-30 14:47:22 [JobThread.run-130]
----------- datax-web job execute start -----------
----------- Param:
2023-11-30 14:47:22 [ScriptJobHandler.execute-82] ----------- script file:/usr/local/datax-web/modules/datax-executor/bin/…/data/applogs/executor/jobhandler/gluesource/1051_1685443120000.py -----------
2023-11-30 14:47:22 [ScriptUtil.execToFile-81] ------------------Process id: 8715
2023-11-30 14:47:22 [ProcessCallbackThread.callbackLog-186]
----------- datax-web job callback finish.
Traceback (most recent call last):
File “/usr/local/datax-web/modules/datax-executor/bin/…/data/applogs/executor/jobhandler/gluesource/1051_1685443120000.py”, line 34, in
db.execute(“INSERT INTO datax_m_user_attribute_tag_manger SELECT NULL as id,a1.org_id,a1.external_user_id,”
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/cursors.py”, line 148, in execute
result = self._query(query)
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/cursors.py”, line 310, in _query
conn.query(q)
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/connections.py”, line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/connections.py”, line 775, in _read_query_result
result.read()
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/connections.py”, line 1156, in read
first_packet = self.connection._read_packet()
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/connections.py”, line 725, in _read_packet
packet.raise_for_error()
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File “/usr/local/python3/lib/python3.9/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, ‘Insert has filtered data in strict mode, txn_id = 1488 tracking sql = select tracking_log from information_schema.load_tracking_logs where job_id=27859’)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/datax-web/modules/datax-executor/bin/…/data/applogs/executor/jobhandler/gluesource/1051_1685443120000.py”, line 48, in
six.reraise(exc)
TypeError: reraise() missing 1 required positional argument: ‘value’
2023-11-30 14:47:22 [JobThread.run-165]
----------- datax-web job execute end(finish) -----------
----------- ReturnT:ReturnT [code=500, msg=script exit value(1) is failed, content=null]
2023-11-30 14:47:22 [TriggerCallbackThread.callbackLog-186]
----------- datax-web job callback finish
同类型脚本有多个,只有这一个脚本出现这个问题
脚本执行sql如下:
INSERT INTO datax_m_user_attribute_tag_manger
SELECT NULL as id,a1.org_id,a1.external_user_id,
ifnull(array_join(array_distinct(array_agg(a2.user_id)),’,’),’’) virtual_manger,
ifnull(array_join(array_distinct(array_agg(a2.real_name)),’,’),’’) virtual_manger_name,
ifnull(array_join(array_distinct(array_agg(a1.rely_id)),’,’),’’) virtual_manger_acct_id,
JSON_ARRAY(if(a1.conversation_id IS NOT null AND a1.conversation_id !=’’,
JSON_OBJECT(‘acct_id’, a1.rely_id
,‘user_id’,a2.user_id,‘acct_name’,a2.real_name,‘conversation_id’, a1.conversation_id
),’’)) AS relation_json,NOW()
FROM (SELECT org_id
, external_user_id
, rely_id
,conversation_id
FROM label_m_follow_user
WHERE deleted
=0 AND status
=0 AND external_user_id
IS NOT NULL GROUP BY org_id
, external_user_id
, rely_id
) a1
JOIN (SELECT org_id
, rely_id
, user_id
, real_name
FROM label_u_manager
WHERE deleted
=0 GROUP BY org_id
, rely_id
, user_id
, real_name
) a2
ON a1.org_id=a2.org_id and a1.rely_id=a2.rely_id GROUP BY a1.org_id,a1.external_user_id
sql所涉及的表,在两个环境中数据结构完全一致,区别就是版本不同。求大佬帮忙看下是什么原因
【联系方式】邮箱:aupatientmu@163.com