datanode日志连接被对方重设/断开的管道

最新推荐文章于 2021-07-20 10:09:35 发布

原创

最新推荐文章于 2021-07-20 10:09:35 发布 · 3.7k 阅读

标签

#hadoop #datanode #hdfs

博客探讨了datanode日志中出现的'断开的管道'和'连接被对方重设'错误，这些实际上是'Broken pipe'和'Connection reset by peer'异常。作者分析了Hadoop源码，发现相关异常处理逻辑，并引用了社区中关于此问题的JIRA issues。解决方案是将Hadoop环境的语言设置从中文更改为英文。

背景

datanode日志中经常发现大量以下报错：

2021-04-07 14:59:35,131 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: BlockSender.sendChunks() exception:
java.io.IOException: 连接被对方重设
    at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
    at sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:433)
    at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:565)
    at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:223)
    at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:578)
    at org.apache.hadoop.hdfs.server.datanode.BlockSender.doSendBlock(BlockSender.java:758)
    at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:705)
    at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:544)
    at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:148)
    at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
    at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:244)
    at java.lang.Thread.run(Thread.java:745)
2021-04-07 14:59:36,273 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: BlockSender.sendChunks() exception:
java.io.IOException: 断开的管道
    at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
    at sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:433)
    at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:565)
    at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:223)
    at org.apache.hadoop.hdfs.server.d