cdh跑任务spark on yarn

版权声明：https://blog.csdn.net/lvtula https://blog.csdn.net/lvtula/article/details/88746658

参考

https://www.cnblogs.com/ptbx/p/5853106.html

https://www.cnblogs.com/fastzhe/p/7444439.html

均切换到hdfs用户才行总结: su hdfs

spark1.x

spark-shell

spark-submit --class org.apache.spark.examples.SparkPi --master yarn-cluster /opt/cloudera/parcels/CDH-5.15.2-1.cdh5.15.2.p0.3/lib/spark/lib/spark-examples.jar 10

spark2.x

spark2-shell

spark2-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster /opt/cloudera/parcels/CDH-5.15.2-1.cdh5.15.2.p0.3/lib/spark/lib/spark-examples.jar 10