hadoop2.x Yarn HA搭建

1.在worker1和worker2上配置resourcemanager和nodemanager，resourcemanager一主一备，所以配置这两台机器免密钥登录，前面已经配置过，不再赘述。
2.先拷贝一份mapred-site.xml，配置mapreduce使用yarn框架

[hadoop@master hadoop]$ vim mapred-site.xml

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

3.配置yarn-site.xml

[hadoop@master hadoop]$ vim yarn-site.xml

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>/data/hadoopha/yarn/log/userlogs</value>
  </property>
  <property>
    <name>yarn.resourcemanager.ha.enabled</name>
    <value>true</value>
  </property>
  <property>
    <name>yarn.resourcemanager.cluster-id</name>
    <value>cluster1</value>
  </property>
  <property>
    <name>yarn.resourcemanager.ha.rm-ids</name>
    <value>rm1,rm2</value>
  </property>
  <property>
    <name>yarn.resourcemanager.hostname.rm1</name>
    <value>worker1</value>
  </property>
  <property>
    <name>yarn.resourcemanager.hostname.rm2</name>
    <value>worker2</value>
  </property>
  <property>
    <name>yarn.resourcemanager.zk-address</name>
    <value>master:2181,worker1:2181,worker2:2181</value>
  </property>
</configuration>

4.启动yarn

[hadoop@master hadoop]$ start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-master.out
worker1: starting nodemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-worker1.out
worker2: starting nodemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-worker2.out
[hadoop@master hadoop]$ jps
7920 NameNode
5393 JournalNode
15651 Jps
5093 QuorumPeerMain
10429 DFSZKFailoverController

查看worker1，worker2节点，jps是没有resourcemanager进程的，需要手动启动
5.在worker1和worker2上启动

[hadoop@worker1 ~]$ yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-worker1.out
[hadoop@worker1 ~]$ jps
5024 JournalNode
14802 Jps
14580 ResourceManager
8868 DFSZKFailoverController
14438 NodeManager
4824 QuorumPeerMain
5436 DataNode
5373 NameNode

[hadoop@worker2 logs]$ yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-worker2.out
[hadoop@worker2 logs]$ jps
4978 JournalNode
9474 Jps
5331 DataNode
9414 ResourceManager
9272 NodeManager
4783 QuorumPeerMain
[hadoop@worker2 logs]$

查看jps，ResourceManager和Nodemanager都已经启动。
6.在webUI界面查看
worker1：8088
这里写图片描述
但在worker2：8088查看时直接跳到了worker1，worker2处于standby。

hadoop2.x Yarn HA搭建

猜你喜欢