ResourceManager-HA搭建

在HDFS-HA的基础上搭建
etc/hadoop/mapred-site.xml:

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.application.classpath</name>
        <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
    </property>
</configuration>

etc/hadoop/yarn-site.xml:

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
  <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.cluster-id</name>
  <value>rmhacluster1</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.rm-ids</name>
  <value>rm1,rm2</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm1</name>
  <value>node3</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm2</name>
  <value>node4</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm1</name>
  <value>node3:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm2</name>
  <value>node4:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.zk-address</name>
  <value>node2:2181,node3:2181,node4:2181</value>
</property>
</configuration>

/etc/profile：

export HADOOP_HOME=/opt/hadoop-3.1.1
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_NODEMANAGER_USER=root
export YARN_RESOURCEMANAGER_USER=root

export HADOOP_MAPRED_HOME=/opt/hadoop-3.1.1
export HADOOP_COMMON_HOME=/opt/hadoop-3.1.1
export HADOOP_HDFS_HOME=/opt/hadoop-3.1.1
export HADOOP_CONF_DIR=/opt/hadoop-3.1.1/etc/hadoop
export HADOOP_YARN_HOME=/opt/hadoop-3.1.1

以上配置在所有的节点上都要配置

在node3或node4上执行：

start-yarn.sh

启动resource manager和nodemanager

停止的时候在node3或node4上执行

stop-yarn.sh

停止resource manager和nodemanager

运行自己编斜体样式写的wordcount程序

yarn jar wordcount.jar com.bjsxt.mr.wordcount.MainClass /hello.txt /mr/myoutput

ResourceManager-HA搭建

猜你喜欢