本文是我学习Hadoop搭建过程中的各种笔记，内容来自于各种公开的教程，起点非常低，从Linux基础开始，直至在PC上搭建Hadoop成功，是真正的从零开始。
感谢过程中帮助我的各位认识的和不认识的老师。
52、Hdfs的ha测试：
解压后：
配置环境变量：
[root@hadoop01 zookeeper-3.4.12]# cd ../hadoop-2.7.1
[root@hadoop01 hadoop-2.7.1]# rm -rf ./share/doc/
[root@hadoop01 hadoop-2.7.1]# vi /etc/profile
export JAVA_HOME=/usr/local/jdk1.8.0_144/ （配这个）
[root@hadoop01 hadoop-2.7.1]# vi ./etc/hadoop/core-site.xml
[root@hadoop01 hadoop-2.7.1]# vi ./etc/hadoop/hdfs-site.xml
做免登陆：
在hadoop01上分发到hadoop02上：
[root@hadoop01 hadoop-2.7.1]# scp -r ../hadoop-2.7.1 hadoop02:/usr/local/
此时在hadoop03上which hadoop是没有的：
[root@hadoop03 ~]# which hadoop
/usr/bin/which: no hadoop in (/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/jdk1.8.0_144//bin:/usr/local/hadoop-2.7.1//bin:/usr/local/hadoop-2.7.1//sbin:/usr/local/zookeeper-3.4.12//bin::/root/bin)
在hadoop01上分发到hadoop03上：
[root@hadoop01 hadoop-2.7.1]# scp -r ../hadoop-2.7.1 hadoop03:/usr/local/
做免登陆的配置，做2到1,2,3的，因为2也有老大：
[root@hadoop02 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
fe:bc:6a:98:93:a0:76:26:9c:ec:5d:45:9f:38:22:c7 root@hadoop02
The key's randomart image is:
+--[ RSA 2048]----+
| |
| |
| . |
| . . o . |
| . E S o |
| .o + . |
| o o ..+. |
| B.o.= .o |
| o.+. o..+. |
+-----------------+
此时登录自己，也需要输密码：
[root@hadoop02 ~]# ssh hadoop02
The authenticity of host 'hadoop02 (192.168.216.112)' can't be established.
RSA key fingerprint is 58:0e:71:78:09:8c:54:ed:43:16:e3:71:eb:5c:20:57.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop02,192.168.216.112' (RSA) to the list of known hosts.
root@hadoop02's password:
Last login: Thu May 3 13:58:12 2018 from 192.168.216.1
退出去：
[root@hadoop02 ~]# exit
logout
Connection to hadoop02 closed.
[root@hadoop02 ~]# ssh-copy-id hadoop02
root@hadoop02's password:
Now try logging into the machine, with "ssh 'hadoop02'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
现在再登录就可以了，不需要再输密码：
[root@hadoop02 ~]# ssh hadoop02
Last login: Thu May 3 19:05:27 2018 from hadoop02
退出去，同样的方法配置hadoop01免登陆配置
[root@hadoop02 ~]# exit
logout
Connection to hadoop02 closed.
[root@hadoop02 ~]# ssh-copy-id hadoop01
输入： 一次yes 一次root
[root@hadoop02 ~]# ssh hadoop01
Last login: Thu May 3 16:49:43 2018 from 192.168.216.1
退出去，同样的方法配置hadoop03免登陆配置
[root@hadoop01 ~]# exit
[root@hadoop02 ~]# ssh-copy-id hadoop03
[root@hadoop02 ~]# ssh hadoop03
Last login: Thu May 3 13:58:21 2018 from 192.168.216.1
现在1和2老大间的通信也配好：（在hadoop01，可到达hadoop02）
[root@hadoop01 hadoop-2.7.1]# ssh hadoop02
Last login: Thu May 3 19:08:43 2018 from hadoop02
[root@hadoop02 ~]#
[root@hadoop02 ~]# exit
logout
Connection to hadoop02 closed.
远程发送配置好的hadoop到另外的服务器上 (也做好)
按照步骤依次执行并启动：
1、 启动zk
2、 启动journalnode服务（单个启动，多个进程启动）
./sbin/hadoop-deamon.sh start journalnode
./sbin/hadoop-deamons.sh start journalnode
3、 挑选两个namenode之中的一台格式化，并且启动
4、 在另外一台namenode的机子上拉去元数据（也可以使用复制）
hdfs namenode –bootstrapStandby
5、 格式化zkfc
hdfs zkfc –formatZK
6、 启动
7、 测试
先看对应的进程是否启动
再看web ui是否正常
在hdfs中读写文件
关闭一个namenode失败，查看是否能自动切换（制造一个namenode失败，看能不能转换过去）
第一次配高可用要执行的东西：↓
1、启动zk
启用前，保证三台机子的zk都在，分别查看jps：
[root@hadoop01 hadoop-2.7.1]# jps
14145 QuorumPeerMain
[root@hadoop02 hadoop-2.7.1]# jps
14145 QuorumPeerMain
[root@hadoop03 hadoop-2.7.1]# jps
14145 QuorumPeerMain
2、启动journalnode服务（多个进程启动）
./sbin/hadoop-deamons.sh start journalnode
[root@hadoop01 hadoop-2.7.1]# hadoop-daemons.sh start journalnode
hadoop01: starting journalnode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop01.out
hadoop02: starting journalnode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop02.out
hadoop03: starting journalnode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-journalnode-hadoop03.out
查看hadoop01 02 03 jps: 都起起来了JournalNode
[root@hadoop01 hadoop-2.7.1]# jps
14145 JournalNode
3、挑选两个namenode之中的一台格式化，并且启动
[root@hadoop01 hadoop-2.7.1]# hdfs namenode -format
[root@hadoop01 hadoop-2.7.1]# ll /home/hahadoopdata/dfs/name （下面有东西）
此时hadoop02下面没有，（02下面也有hahadoopdata，但是没有，原因：没有同步元数据）
[root@hadoop01 hadoop-2.7.1]# ll /home/hahadoopdata/dfs/name
单个启动，不起动起来，去同步是不行的：
[root@hadoop01 hadoop-2.7.1]# hadoop-daemon.sh start namenode
[root@hadoop01 hadoop-2.7.1]# jps
18618 NameNode （namenode启动起来了）
4、在另外一台namenode的机子上拉去元数据（也可以使用复制）
hdfs namenode –bootstrapStandby
[root@hadoop02 ~]# hdfs namenod -bootstrapStandby
[root@hadoop01 hadoop-2.7.1]# ll /home/hahadoopdata/dfs/name
此时hadoop02上就有了和hadoop01一样的元数据
5、格式化zkfc
hdfs zkfc –formatZK
在克隆的hadoop01上：
此处有问题，前面就有，没能解决
2018-05-03 20:31:42,583 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@878] - Socket connection established to localhost/127.0.0.1:2181, initiating session
正确的应该出现下面这个目录：
[zk: hadoop01:2181 (CONNECTED) 0]
[zk: hadoop01:2181 (CONNECTED) 0] ls /
[name0000000002,name0000000003, zookeeper]
删除：name0000000002
[zk: hadoop01:2181 (CONNECTED) 1] rmr /name0000000002
删除：name0000000003
[zk: hadoop01:2181 (CONNECTED) 2] rmr /name0000000003
查看：
[zk: hadoop01:2181 (CONNECTED) 3] ls /
[zookeeper]
在hadoop01上格式化：
[root@hadoop01 hadoop-2.7.1]# hdfs zkfc -formatZK
在克隆的hadoop01上查看：
[zk: hadoop01:2181 (CONNECTED) 4] ls /
[hadoop-ha,zookeeper]
[zk: hadoop01:2181 (CONNECTED) 5] ls /hadoop-ha
[qf]
[zk: hadoop01:2181 (CONNECTED) 6] ls /hadoop-ha/qf
[ ]
6、启动
[root@hadoop01 hadoop-2.7.1]# start-dfs.sh
hadoop01配置了5个，但上面有6个，有一个是连接客户端
[root@hadoop01 hadoop-2.7.1]# jps
hadoop02上5个：
[root@hadoop02 hadoop-2.7.1]# jps
hadoop03上3个：
[root@hadoop02 hadoop-2.7.1]# jps
7、测试
先看对应的进程是否启动; 再看web ui是否正常
在hdfs中读写文件
在hadoop03上查看（现在没有任何东西，因为是重新搭了一个高可用）
[root@hadoop03 ~]# hdfs dfs –ls
上传：
[root@hadoop03 ~]# hdfs dfs –put /home/shell/if.sh /
此时在web ui可以看到
也可以查看，能查看的，就没问题：
[root@hadoop03 ~]# hdfs dfs –cat /if.sh
关闭一个namenode失败，查看是否能自动切换（制造一个namenode失败，看能不能转换过去）
在hadoop01上：
[root@hadoop01 hadoop-2.7.1]# jps
18618 NameNode
[root@hadoop01 hadoop-2.7.1]# kill -9 18618
再查看jps，就没有NameNode了
但是，在hadoop03上仍然能读取，这就是高可用
老大坏一个，立马把另一个起起来
[root@hadoop03 ~]# hdfs dfs –cat /if.sh
在hadoop01上重新启动：
[root@hadoop01 hadoop-2.7.1]# hadoop-daemon.sh start namenode
此时jps,有namenode:
[root@hadoop01 hadoop-2.7.1]# jps
19451 NameNode
在网址上查看，此时hadoop01是standby
在hadoop02上，杀掉namenode：
[root@hadoop01 hadoop-2.7.1]# jps
14456 NameNode
[root@hadoop01 hadoop-2.7.1]# kill -9 14456
在网址上查看，此时hadoop01就变成active，hadoop02就访问不进来了。
在hadoop03上读取文件，仍然正常
[root@hadoop03 ~]# hdfs dfs –cat /if.sh