How to install and config spark on Ubuntu/Linux

版权声明: https://blog.csdn.net/u011263794/article/details/81503969

1. Download spark-2.3.1-bin-hadoop2.7.tgz from http://spark.apache.org/downloads.html

2. unzip the file spark-2.3.1-bin-hadoop2.7.tgz

3. Set the SPARK_HOME and PATH
export SPARK_HOME=/home/myname/spark-2.3.1-bin-hadoop2.7
export PATH=$PATH:$SPARK_HOME/bin

4. Set python3 as the default python version
export PYSPARK_PYTHON=python3

5. Enable our sets
source /etc/profile

issues:
1) py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
Please try djk8

猜你喜欢

转载自blog.csdn.net/u011263794/article/details/81503969