大数据(八)Hadoop YARN配置

 

 1、配置mapred-site.xml

[root@node01 hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@node01 hadoop]# vi mapred-site.xml
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

 2、配置yarn-site.xml

[root@node01 hadoop]# vi yarn-site.xml
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>   <name>yarn.resourcemanager.ha.enabled</name>   <value>true</value>   </property>
  <property>   <name>yarn.resourcemanager.cluster-id</name>   <value>cluster1</value>   </property>   <property>   <name>yarn.resourcemanager.ha.rm-ids</name>   <value>rm1,rm2</value>   </property>   <property>   <name>yarn.resourcemanager.hostname.rm1</name>   <value>node03</value>   </property>   <property>   <name>yarn.resourcemanager.hostname.rm2</name>   <value>node04</value>   </property>   <property>   <name>yarn.resourcemanager.zk-address</name>   <value>node02:2181,node03:2181,node04:2181</value>   </property>

3、将上述两个文件分发到其他Hadoop节点

[root@node01 hadoop]# scp mapred-site.xml yarn-site.xml node02:`pwd`

4、让两个ResourceManager之间互相免密钥

[root@node03 .ssh]# ssh-keygen -t dsa -P '' -f ./id_dsa
[root@node03 .ssh]# cat id_dsa.pub >> authorized_keys
[root@node03 .ssh]# scp id_dsa.pub node04:`pwd`/node03.pub
[root@node04 .ssh]# cat node03.pub >> authorized_keys

另一个节点执行相同步骤

5、启动

启动顺序:

①zookeeper,zkServer.sh start
②hdfs (注意,有一个脚本不要用,start-all)start-dfs.sh
   如果nn 和 nn2没有启动,需要在node01,node02分别手动启动:
   hadoop-daemon.sh start namenode
③start-yarn.sh (启动nodemanager)
④在03,04节点分别执行脚本: yarn-daemon.sh start resourcemanager

6、启动后各节点角色进程

[root@node01 ~]# jps
3136 Jps
2612 NameNode
2955 DFSZKFailoverController
2798 JournalNode

[root@node02 ~]# jps
2306 DFSZKFailoverController
2388 NodeManager
2056 NameNode
2120 DataNode
2202 JournalNode
2524 Jps
1388 QuorumPeerMain

[root@node03 ~]# jps
1875 DataNode
1957 JournalNode
1414 QuorumPeerMain
2409 Jps
2185 ResourceManager
2044 NodeManager

[root@node04 ~]# jps
1890 NodeManager
1794 DataNode
2085 Jps
1415 QuorumPeerMain
2031 ResourceManager

7、http访问端口:8088

8、停止:

node01: stop-dfs.sh
node01: stop-yarn.sh (停止nodemanager)
node03,node04: yarn-daemon.sh stop resourcemanager (停止resourcemanager

ZooKeeper:zkServer.sh stop

猜你喜欢

转载自www.cnblogs.com/mstoms/p/11830588.html
今日推荐