问:
Spark 在YARN上运行,以集群模式启动Spark应用程序:
$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi \
--master yarn \
--deploy-mode cluster \
--driver-memory 4g \
--executor-memory 2g \
--executor-cores 1 \
--queue default \
../examples/jars/spark-examples*.jar \
10
如何查看结果?
答:
打开Hadoop日志文件夹下面的stdout文件,例如:
/usr/hadoop/hadoop-3.2.1/logs/userlogs/application_1580631306621_0002/container_1580631306621_0002_01_000001/stdout
stdout文件内容:
Pi is roughly 3.141115141115141