spark ubuntu cluster installation

Scala installation

1. Mirroring address http://distfiles.macports.org/scala2.13/

#创建文件夹
mkdir sparkWork
cd sparkWork
#下载scala
wget http://distfiles.macports.org/scala2.13/scala-2.13.0.tgz
#解压
tar -xvf scala-2.13.0.tgz

2. Configure

#获取当前路径
pwd

#为系统每一个用户配置环境
sudo nano /etc/profile
#进入文件最后添加如下
export SCALA_HOME=/home/ubuntu/sparkWork/scala-2.12.10 #此为解压scala文件夹位置
export PATH=$PATH:$SCALA_HOME/bin
#使环境变量生效
source /etc/profile

#为当前用户配置环境
sudo nano ~/.bashrc
#进入文件最后添加如下
export SCALA_HOME=/home/ubuntu/sparkWork/scala-2.12.10 #此为解压scala文件夹位置
export PATH=$PATH:$SCALA_HOME/bin
#使环境变量生效
source ~/.bashrc

3. Verify

scala -version

Shown below indicates that the installation was successful

Installation spark

1. Install
foreign installation is too slow, you can use a mirror image of the Northern Polytechnic select the appropriate version of the service

http://mirror.bit.edu.cn/apache/spark/

#创建文件夹
mkdir sparkWork
cd sparkWork
#下载spark
wget http://mirror.bit.edu.cn/apache/spark/spark-2.4.4/spark-2.4.4-bin-hadoop2.6.tgz
#解压 
tar -xvf spark-2.4.4-bin-hadoop2.6.tgz
#修改文件名
mv spark-2.4.4-bin-hadoop2.6 spark-2.4.4

2. Configure

#获取当前路径
pwd

#为系统每一个用户配置环境
sudo nano /etc/profile
#进入文件最后添加如下
export SPARK_HOME=/home/ubuntu/sparkWork/spark-2.4.4 #此为解压spark文件夹位置
export PATH=$PATH:$SPARK_HOME/bin
#使环境变量失效
source /etc/profile

#为当前用户配置环境
sudo nano ~/.bashrc
#进入文件最后添加如下
export SPARK_HOME=/home/ubuntu/sparkWork/spark-2.4.4 #此为解压spark文件夹位置
export PATH=$PATH:$SPARK_HOME/bin
#使环境变量失效
source ~/.bashrc

3. Verify

spark-shell

I.e., successful installation is shown below

Any bug occurs during installation, an error will copy the information into Google search to search, most likely reason is to install a version mismatch problem