内存计算框架Spark学习笔记（3）—— Spark Application监控 HistoryServer 配置

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/u012292754/article/details/86592517

1 Spark HistoryServer 配置

• spark-env.sh

SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=file:///home/hadoop/appsData/sparkData/historylogs

• spark-defaults.conf

spark.eventLog.enabled           true
spark.eventLog.dir               file:///home/hadoop/appsData/sparkData/historylogs
spark.eventLog.compress          true

---

• 启动 sbin/start-history-server.sh

[hadoop@node1 spark-2.2.2-bin-2.6.0-cdh5.7.0]$ sbin/start-history-server.sh 
starting org.apache.spark.deploy.history.HistoryServer, logging to /home/hadoop/apps/spark-2.2.2-bin-2.6.0-cdh5.7.0/logs/spark-hadoop-org.apache.spark.deploy.history.HistoryServer-1-node1.out
[hadoop@node1 spark-2.2.2-bin-2.6.0-cdh5.7.0]$ jps
1825 Worker
1906 HistoryServer
1702 Master
1965 Jps

• http://node1:18080/