Spark 简洁集群搭建(1分钟)

1. 准备环境
2. 安装JDK， 不需要安装Scala， Spark内置scala, 最好搭配JDK8, 搭配8以上版本， 依据个人经验可能出问题。
3. 上传spark安装包
4. 解压spark并修改配置文件（两个配置文件，第一个配置文件添加了3个配置文件）

Spark.env.sh 文件，增加如下3个属性， 指定JDK, 指定Spark master, 指定Spark集群rpc通讯端口

export JAVA_HOME=/root/apps/jdk1.8.0_65
export SPARK_MASTER_HOST=hadoop-01
export SPARK_MASTER_PORT=7077

slaves文件

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# A Spark Worker will be started on each of the machines listed below.
hadoop-03
hadoop-04
hadoop-05

6. 将配置好的spark安装程序拷贝给其他机器， 可以写一个循环的shell

 for i in {3..5}; 
 do 
 scp -r /root/apps/spark-2.1.1-bin-hadoop2.7/  root@hadoop-0$i:/root/apps/; 
 done 

8. 启动spark (sbin/start-all.sh)
   Worker通过读取spark-env.sh文件得知Master在哪里的
9. 通过web页面访问spark管理页面（master所在机器的地址+8080端口）， 注意： 此8080并非tomcat或者啥乱七八糟的， 而是netty通讯端口。