主节点是 server,其他节点是 agent
1:apache服务器
sudo rpm -qa|grep http
sudo service httpd status
sudo service httpd start
sudo chkconfig httpd on
访问 : spark1 主机名(先重启再访问),要能访问通
2:搭建本地源
>>软件下载
>>网站:
http://archive.cloudera.com/cm5/
>>cm-5.3.6 bin文件
http://archive.cloudera.com/cm5/installer/5.3.6/cloudera-manager-installer.bin
>>cm-5.3.6 依赖的rpm包
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.3.6/
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.3.6/RPMS/x86_64/*
>>依赖其他包
postgrep-sql
repo-libs
3:下载包
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.3.6/RPMS/x86_64/ 下的包，
新建目录:
cd /var/www/html/
mkdir -p cm5/redhat/6/x86_64/cm/5/RPMS/x86_64/
mkdir /var/www/html/cm5/redhat/6/x86_64/cm/5/repodata/
把包移动到 linux目录下 cm5/redhat/6/x86_64/cm/5/RPMS/x86_64/,
把 文件移动到 /var/www/html/cm5/redhat/6/x86_64/cm/5/repodata/下
4:创建cloudera-manager.repo
cd /etc/yum.repos.d/
touch cloudera-manager.repo
写入:
[cloudera-manager]
name=Cloudera Manager
baseurl=http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/
gpgkey = http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1
5:修改本机域名
window域名 192.168.1.181 sparkproject1
sparkproject1/cm5/redhat/6/x86_64/cm/5/RPMS/x86_64/
6:安装
./cloudera-manager-installer.bin
到此，CM安装完成
7:安装 CDH
CDH-5.3.6-1.cdh5.3.6.p0.11-el6.parcel
CDH-5.3.6-1.cdh5.3.6.p0.11-el6.parcel.sha1
manifest.json
把这三个包移动到 /opt/cloudera/parcel-repo 文件夹，修改 CDH-5.3.6-1.cdh5.3.6.p0.11-el6.parcel.sha1 文件名为
CDH-5.3.6-1.cdh5.3.6.p0.11-el6.parcel.sha
然后重启 CM，让他自动去找这个包
sudo service cloudera-scm-server status
sudo service cloudera-scm-server start
sudo service cloudera-scm-server restart
注意:
先下载到 parcel-cache缓存包，再解压到 parcels包里面
然后运行 命令:sudo sysctl -w vm.swappiness=0(此时重启就不生效)
设置重启生效:echo "vm.swappiness=0" >> /etc/sysctl.conf
然后在spark2上面也运行
sudo sysctl -w vm.swappiness=0，echo "vm.swappiness=0" >> /etc/sysctl.conf命令
然后 vi /etc/rc.local ，
添加 echo never > /sys/kernel/mm/transparent_hugepage/defrag