DolphinSchedule执行Spark任务失败

最新推荐文章于 2026-06-23 17:59:46 发布

原创最新推荐文章于 2026-06-23 17:59:46 发布 · 2.8k 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

大数据

本文介绍了解决DolphinSchedule执行Spark任务时显示失败但实际上YARN执行成功的问题。通过修改DS集群的配置文件common.properties并重启服务，确保yarn.resourcemanager.ha.rm.ids正确设置。

现象：

DolphinSchedule执行Spark任务失败,但yarn实际是执行成功的,且目标库中有执行结果.

yarn：

ds 的worker报错日志：

[ERROR] 2021-06-28 14:59:41.693 org.apache.dolphinscheduler.common.utils.HttpUtils:[73] - null: Name or service not known
java.net.UnknownHostException: null: Name or service not known
   at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
   at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
   at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
   at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
   at java.net.InetAddress.getAllByName(InetAddress.java:1193)
   at java.net.InetAddress.getAllByName(InetAddress.java:1127)
   at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
   at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
   at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
   at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
   at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
   at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   at org.apache.dolphinscheduler.common.utils.HttpUtils.get(HttpUtils.java:60)
   at org.apache.dolphinscheduler.common.utils.HadoopUtils.getApplicationStatus(HadoopUtils.java:423)
   at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.isSuccessOfYarnState(AbstractCommandExecutor.java:403)
   at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:233)
   at org.apache.dolphinscheduler.server.worker.task.AbstractYarnTask.handle(AbstractYarnTask.java:57)
   at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:133)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
[ERROR] 2021-06-28 14:59:41.739 - [taskAppId=TASK-4-584-607]:[417] - yarn applications: [application_1623892380354_0315] status failed
java.lang.NullPointerException: null
   at org.apache.dolphinscheduler.common.utils.HadoopUtils.getApplicationStatus(HadoopUtils.java:426)
   at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.isSuccessOfYarnState(AbstractCommandExecutor.java:403)
   at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:233)
   at org.apache.dolphinscheduler.server.worker.task.AbstractYarnTask.handle(AbstractYarnTask.java:57)
   at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:133)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)

参考了这个问题,https://github.com/apache/dolphinscheduler/issues/2611

最终解决:

第一步：修改ds的common.properties配置文件，添加yarn.resourcemanager.ha.rm.ids

修改 ds路径/conf/common.properties文件中，yarn.resourcemanager.ha.rm.ids=host1,host2 注意，if resourcemanager HA is enabled, please set the HA IPs; if resourcemanager is single, keep this value empty，也就是说如果yarn是HA模式，就设置成yarn HA中的host设置，如果是单节点的，就设置为空。

yarn HA中的设置文件在hadoop路径/etc/hadoop/yarn-site.xml

yarn.resourcemanager.ha.enabled 属性为true，代表yarn开启了ha模式

yarn.resourcemanager.ha.rm-ids 属性的值就是ds中要设置的值（最好写hostname，这里可能会有映射）