[spark running error]

If you use Hadoop-related things in the program, such as writing files to HDFS, you will encounter the following exception: The insert image description here
reason for this problem is not a program error, but the use of Hadoop-related services. The solution:
configure HADOOP_HOME environment variable
insert image description here
If this problem occurs, it is because the hadoop permission on windows is not enough.

Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
	at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
	at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:645)
	at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:1230)
	at org.apache.hadoop.fs.FileUtil.list(FileUtil.java:1435)
	at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:493)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1868)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1910)
	at org.apache.hadoop.fs.FileSystem$4.<init>(FileSystem.java:2072)

The solution is to copy hadoop.dll in the hadoop bin folder installed on windows to the C:\Windows\System32 folder.
insert image description here
If this problem still occurs, it is because there is no C++ dependent library locally.

Guess you like

Origin blog.csdn.net/Tonystark_lz/article/details/127078505