开发环境:
hadoop-2.7.3
spark-2.2.1
scala-2.11.11
jdk-1.8
报错信息如下:
Exception in thread "main" java.lang.UnsatisfiedLinkError:org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
原因:
我在window要环境配置的hadoop版本与linux测试机上的不一致,较低,下载同版本的即可 hadoop-common-2.7.3-bin-master,将hadoop.dll文件复制到c:\windows\system32中即可