1. Download good installation package on the host: hadoop-2.8.5.tar.gz; jdk1.8.0_131.tar.gz total of two packets
2. Docker got me in the mirror repository:
docker pull centos
3. Create a container:
docker run -i -t -d --name centos_hdp centos:centos
4. From the host copy hadoop-2.8.5.tar.gz, jdk1.8.0_131.tar.gz two packages to the vessel:
docker cp jdk-8u131-linux-x64.tar.gz centos_hdp:/usr/local
docker cp hadoop-2.8.5.tar.gz centos_hdp:/opt/
The runs in the background into the container:
docker exec -it centos_hdp bash
tar -zxvf /usr/local/jdk1.8.0_131.tar.gz -C /usr/local/
tar -zxvf /opt/hadoop-2.8.5.tar.gz -C /opt/
vim /etc/proflie
source /etc/profile
vim /opt/hadoop-2.8.5/etc/hadoop/core-site.xml
vim /opt/hadoop-2.8.5/etc/hadoop/hdfs-site.xml
vim /opt/hadoop-2.8.5/etc/hadoop/mapred-site.xml
vim /opt/hadoop-2.8.5/etc/hadoop/yarn-site.xml
vim /opt/hadoop-2.8.5/etc/hadoop/hadoop-env.sh
cat /opt/hadoop-2.8.5/etc/hadoop/slave
6. Quit container, the container packed into the current mirror:
docker commit -a "mayunzhen" -m "hadoop base images" centos_hdp centos_hdp:2.8.5
7. Export into jar package mirror
docker save centos_hdp:2.8.5 -o centos_hdp.jar
8.centos_hdp.jar image package spread to other host
scp centos_hdp.jar [email protected]:/
scp centos_hdp.jar [email protected]7:/
scp centos_hdp.jar [email protected]8:/
9. Run the imported image on three (192.168.130.166,192.168.130.167,192.168.130.168) host
docker load -i centos_hdp.jar
10. On each host configured docker weave, so that the container can communicate on a different host
详情见链接:https://www.cnblogs.com/kevingrace/p/6859173.html
11.运行hadoop容器命令:
(192.168.130.166主机上)docker run -itd -h master --name master -v /etc/localtime:/etc/localtime:ro centos_hdp:2.8.5
weave attach 192.168.1.1/24 master
(192.168.130.167主机上)docker run -itd -h slave1 --name slave1 -v /etc/localtime:/etc/localtime:ro centos_hdp:2.8.5
weave attach 192.168.1.2/24 slave1
(192.168.130.168主机上)docker run -itd -h slave2 --name slave2 -v /etc/localtime:/etc/localtime:ro centos_hdp:2.8.5
weave attach 192.168.1.3/24 slave2
测试三个容器IP(192.168.1.1,192.168.1.2,192.168.1.3)互相能通信:
12.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置/etc/hosts
13.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置无密码登录
详情:https://www.cnblogs.com/shuochen/p/10441455.html
14.master容器中执行启动hadoop
source /etc/profile
start-all.sh
15.查看集群状态:
制作好的docker hadoop:2.8.5镜像已经上传到https://hub.docker.com/repository/docker/iammayunzhen/centos_hdp(或者可 docker pull iammayunzhen/centos_hdp:2.8.5进行下载),有需要,可供大家下载,参考。
Refereces:
docker hadoop镜像制作:https://www.jianshu.com/p/bf76dfedef2f
docker容器时间和宿主机同步:https://www.cnblogs.com/kevingrace/p/5570597.html
多宿主机容器之间通信:https://www.cnblogs.com/kevingrace/p/6859173.html
docker容器中IP无法ssh登录:http://blog.chinaunix.net/uid-26168435-id-5732463.html
查看端口情况:netstat -tulnp |grep 22
安装passwd命令:yum install -y passwd
多容器IP无密码登录:https://www.cnblogs.com/shuochen/p/10441455.html