1. Overview
Partly reproduced from: http://eksliang.iteye.com/blog/2226986
1.1 The single point problem of hadoop1.0 The
NameNode in Hadoop is like the heart of a human being. It is very important and must not stop working. In the hadoop1 era, there was only one NameNode. If the NameNode data is lost or becomes inoperable, the entire cluster cannot be recovered. This is a single point problem in hadoop1 and a manifestation of hadoop1 being unreliable. As shown in the figure below, it is the architecture diagram of hadoop1.0;
1.2 The solution of hadoop2.0 to the single-point problem of hadoop1.0
In order to solve the single-point problem in hadoop1, the new NameNode in hadoop2 is no longer only one, there can be multiple (currently only two are supported). Each has the same function. One is in active state and the other is in standby state. When the cluster is running, only the NameNode in the active state is working normally, and the NameNode in the standby state is in the standby state, and the data of the NameNode in the active state is synchronized at all times. Once the NameNode in the active state cannot work, the NameNode in the standby state can be changed to the active state through manual or automatic switching, and it can continue to work. This is high reliability.1.3 Using JournalNode to Share NameNode (Active and Standby) Data
In Hadoop 2.0, the data of two NameNodes is actually shared in real time. The new HDFS adopts a sharing mechanism, Quorum Journal Node (JournalNode) cluster or Nnetwork File System (NFS) for sharing. NFS is at the operating system level, and JournalNode is at the hadoop level. We use the JournalNode cluster for data sharing (this is also the mainstream practice). As shown in the figure below, it is the architecture diagram of JournalNode.
For data synchronization, two NameNodes communicate with each other through a set of independent processes called JournalNodes. Most of the JournalNodes processes are notified when the namespace of the active NameNode is modified. The NameNode in the standby state has the ability to read the change information in the JNs, and always monitor the changes of the edit log, and apply the changes to its own namespace. standby ensures that in the event of a cluster failure, the namespace state is fully synchronized
1.4 Failover between NameNodes
For an HA cluster, it is crucial to ensure that only one NameNode is active at the same time. Otherwise, the data states of the two NameNodes will diverge, data may be lost, or erroneous results may be produced. In order to ensure this, this requires the use of ZooKeeper. First, both NameNodes in the HDFS cluster are registered in ZooKeeper. When the NameNode in the active state fails, ZooKeeper can detect this situation, and it will automatically switch the NameNode in the standby state to the active state.
2. Reference document for building a Hadoop (HDFS-style HA) cluster
* Architecture diagram
* 2.1 Detailed configuration
Four machines: ndoe1, ndoe2, ndoe3, node4
node1:192.168.116.3
node2:192.168.116.4
node3:192.168.116.5
node4:192.168. 116.6
Build a zookeeper cluster, reference link
HDFS High Availability
reference document path: hadoop-2.5.2/share/doc/hadoop/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.htmlDetailed configuration
1. Modify the hadoop file hdfs-site.xml
<configuration>
<property>
<name>dfs.nameservices</name>
<value>bjsxt</value>
</property>
<property>
<name>dfs.ha.namenodes.bjsxt</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.bjsxt.nn1</name>
<value>node1:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.bjsxt.nn2</name>
<value>node2:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.bjsxt.nn1</name>
<value>node1:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.bjsxt.nn2</name>
<value>node2:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://node2:8485;node3:8485;node4:8485/bjsxt</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.bjsxt</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_dsa</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/hadoop-2.5/data</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
</configuration>2、core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://bjsxt</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop-2.5</value>
</property>
</configuration>3. Copy the hadoop configured under the current node to other node2\3\4
scp /home/hadoop-2.5.1/etc/hadoop/ root@node2:/home/hadoop-2.5.1/etc/
scp /home/hadoop-2.5.1/etc/hadoop/ root@node3:/home/hadoop-2.5.1/etc/
scp /home/hadoop-2.5.1/etc/hadoop/ root@node4:/home/hadoop-2.5.1/etc/4. Start journalnode, that is, 234 (single node startup)
Enter the following commands in node2, 3, and 4 respectively
hadoop-daemon.sh start journalnode5. Check:
In order to prevent errors, you need to check the logs after each step is modified. If there is no abnormality, it is correct, and continue to
check the journalnode and namenode:
tail -100 /home/hadoop-2.5.1/logs/hadoop-root-journalnode-node3.log
tail -100 /home/hadoop-2.5.1/logs/hadoop-root-namenode-node3.log 6. Synchronize the data of two namenodes, namely node1 and node2.
In order to synchronize the data, you first need to select a namenode for data formatting
hdfs namenode -format7. Initializing HA state in ZooKeeper
Select a namenode and enter the following command:
hdfs zkfc -formatZKEnter start-dfs.shstart cluster
8. Verification
(if it cannot be switched normally, check the log, it may be that the password-free login between nodes is not possible)
- Add ResourceManager for resource management
1. Official architecture
2. Configure ResourceManager nodes
node1 and node2 as ResourceManager nodes
3. Modify yarn-default.xml
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>cluster1</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>node1</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>node2</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>node2:2181,node3:2181,node4:2181</value>
</property>4. Single node configuration
etc/hadoop/core-site.xml:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
etc/hadoop/hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>Copy configuration files to other nodes
scp /home/hadoop-2.5.1/etc/hadoop/ root@node2:/home/hadoop-2.5.1/etc/
scp /home/hadoop-2.5.1/etc/hadoop/ root@node2:/home/hadoop-2.5.1/etc/
scp /home/hadoop-2.5.1/etc/hadoop/ root@node2:/home/hadoop-2.5.1/etc/5. Start yarn
1、主节点node1启动：
start-yarn.sh
2、启动备用yarn（node2上）：
yarn-daemon.sh start resourcemanager 6. Verify resourcemanager
Error record
1. Firewall closed
2. Password-free login between nodes
3. Word error in configuration file and word consistency
4. Check log
Possibly better blog link:
https://blog.csdn.net/q361239731/article/details/53559681
https://blog.csdn.net/q361239731/article/details/53559866