Open Hadoop / Yarn log monitoring

配置 yarn-site.xml

<!-- Site specific YARN configuration properties -->
<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>

  <!-- 开启日志聚合 如果没有设置的话，会显示3个目录 -->
  <property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
  </property>
</configuration>

配置mapred-site.xml

<property>
    <!-- 表示提交到hadoop中的任务采用yarn来运行，要是已经有该配置则无需重复配置 -->
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
</property>
<property>
  <!--日志监控服务的地址，一般填写为nodenode机器地址 -->
    <name>mapreduce.jobhistroy.address</name>
    <value>master:10020</value>
</property>
<property>
    <name>mapreduce.jobhistroy.webapp.address</name>
    <value>master:19888</value>
</property>

重启yarn

stop-yarn.sh
start-yarn.sh

开启日志监控服务进程
Nodenode performed on the machine sbin/mr-jobhistory-daemon.sh start historyservercommand, use the jps command to see if started successfully After execution, if successful will start showing JobHistoryServer Service