ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

15/06/11 15:35:50 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
    at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:272)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:790)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:760)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:633)
    at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2001)
    at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2001)
    at scala.Option.getOrElse(Option.scala:120)
    at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2001)
    at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:207)
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:218)
    at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:163)
    at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:269)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:272)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:154)
    at SparkFromHbase$.main(SparkFromHbase.scala:15)
    at SparkFromHbase.main(SparkFromHbase.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

Obviously it should be HADOOP_HOME problem. If HADOOP_HOME is empty, the inevitable fullExeName is null \ bin \ winutils.exe. The solution is simple, configuration environment variable, do not want to restart the computer can be added in the program:

 
        System.setProperty( 
        "hadoop.home.dir" 
        ,  
        "E:\\Program Files\\hadoop-2.7.0" 
        );

Note: E: \\ Program Files \\ hadoop-2.7.0 is the path of my native hadoop decompression.

And do it later, you may still be the same error occurs, this time you might want to blame me. In fact, the beginning, I was rejected, because you get to see under your hadoop-xxx / bin directory, you will find that you are fundamentally not winutils.exe this stuff.

So I tell you, you can go to download a github, the Earth people know the address you send a.

Address: https://github.com/srccodes/hadoop-common-2.2.0-bin

Do not worry about its version, do not be afraid, because the latest hadoop-2.7.0 I use no problem! Once you have downloaded, the winutils.exe join under your hadoop-xxx / bin.

ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

Guess you like