当从本地上传文件到HDFS中时报错
fs.FSInputChecker: Found checksum error: b[0, 69]=6d6f77656968616861686168616868616686168616861686861680a
org.apache.hadoop.fs.ChecksumException: Checksum error: file:/hyk/data/hyk.txt
[root@node01 data]# hadoop fs -put hyk.txt /hyk/test
20/02/18 12:54:39 INFO fs.FSInputChecker: Found checksum error: b[0, 69]=6d6f77656968616861686168616868616686168616861686861680a
org.apache.hadoop.fs.ChecksumException: Checksum error: file:/hyk/data/hyk.txt at 0 exp: 599154106 got: -6
at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:323)
at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:279)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:228)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:196)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:86)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:60)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:120)
at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWit
at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.jav
at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:
at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:263)
at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:248)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:317)
at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:289)
at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.ja
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:271)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:255)
at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:
at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:267)
at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201)
at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
put: Checksum error: file:/hyk/data/hyk.txt at 0 exp: 599154106 got: -685625431
hadoop客户端将本地文件hyk.txt上传到集群上时,会通过fs.FSInputChecker判断需要上传的文件是否存在.crc校验文件。
如果存在.crc校验文件，则会进行校验。如果校验失败，就不会上传该文件.
解决办法如下：
# 查看是否存在.hyk.txt.crc
[root@node01 data]# ls -a
. .. hyk.txt .hyk.txt.crc hyw.txt
# 删除.hyk.txt.crc文件
[root@node01 data]# rm .hyk.txt.crc
rm：是否删除普通文件 ".hyk.txt.crc"？y
[root@node01 data]#
再次从本地上传文件到HDFS中去就可以了
