ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException

window

在启动Pyspark时，会报下面错误

ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

原因，我们搭建的远程Spark集群，在window本地是无法获取hadoop的配置，我们需要在hadoop\bin目录下载winutils.exe，再重启Hadoop和spark即可。

linux

zailinux需要将hadoop解压包放回window，配置hadoop_home环境变量，还是要winutils.exe

下载链接： https://github.com/sdravida/hadoop2.6_Win_x64/blob/master/bin/winutils.exe