The premise is to install hadoop, JDK environment
Download address scala:
http://www.scala-lang.org/download/2.11.6.html
Unzip: tar –zxvf scala-2.11.6.tgz
Configure environment variables: vi /etc/profile
export SCALA_HOME=/opt/scala-2.11.6
export PATH=$SCALA_HOME/bin:$PATH
Execute (effective immediately): source /etc/profile
Execute: cd /opt/scala-2.11.6/bin
./scala –version
Output (successful installation): Scala code runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL
Download address spark:
http://apache.fayea.com/spark/spark-1.4.0/spark-1.4.0-bin-hadoop2.6.tgz
Unzip: tar –zxvf spark-1.4.0-bin-hadoop2.6.tgz
Configure environment variables: vi /etc/profile
export SPARK_HOME=/opt/spark-1.4.0-bin-hadoop2.6
export PATH=$SPARK_HOME/bin:$PATH
Execute (effective immediately): source /etc/profile
Enter the spark conf directory: cd /opt/spark-1.4.0-bin-hadoop2.6/conf
Execute: cp spark-env.sh.template spark-env.sh
Execute: vi spark-env.sh
Add the following:
export JAVA_HOME=/usr/java/jdk1.7.0_67
export SCALA_HOME=/opt/scala-2.11.6
export SPARK_MASTER_IP=192.168.68.84
export SPARK_WORKER_MEMORY=2g
export HADOOP_CONF_DIR=/opt/soft-228238/hadoop-2.6.0/etc/hadoop
Note: SPARK_MASTER_IP is the host IP in the spark cluster; SPARK_WORKER_MEMORY is the maximum memory used by the worker
执行： cp slaves.template slaves
Execute: vi slaves
Add two slave nodes:
192.168.68.84
192.168.68.85
Go to: cd /opt/spark-1.4.0-bin-hadoop2.6/sbin
Execute: ./start-all.sh
Execution: jps result as shown in the red box indicates success
Visit: http://192.168.68.84:8080/
