Error:
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.
Description: There is an error calculating wordcount, the error prompts to supplement the mapred-site.xml configuration
Solution: 1 :
This is here for meit does not work(Many blogs say to change this)
Insert the following code in mapred-site-xml
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
Solution 2: (Pro-test effective)
- 1: Enter the command:hadoop classpath
For example:
- 2:Copy the return value and modify it to the following format
<property>
<name>yarn.application.classpath</name>
<value>
/opt/module/hadoop-3.1.3/etc/hadoop,
/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*,
/opt/module/hadoop-3.1.3/share/hadoop/common/*,
/opt/module/hadoop-3.1.3/share/hadoop/hdfs,
/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*,
/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*,
/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*,
/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*,
/opt/module/hadoop-3.1.3/share/hadoop/yarn,
/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*,
/opt/module/hadoop-3.1.3/share/hadoop/yarn/*,
</value>
</property>
- 3: Then add the code toyarn-site.xml,mapred-site.xmlFor
example, yarn-site.xml, mapred-site.xml is the same
! ! ! ! Remember to distribute these two modified files to other machines , and then restart yarn, dfs
success! ! ! (Happy)
Results on HDFS! ! !