hadoop大数据平台手动搭建(八)HDFS High Availability Using the Quorum Journal Manager

前面我们已经搭建了hadoop基本平台。由于namenode节点唯一,出现故障,就会瘫痪,恢复时间长。而在生产环境需要做到高可用。

第一种方式:

使用Quorum Journal Manager(QJM)功能配置和管理HA HDFS集群。

第二种方式:

使用NFS作为NameNodes所需的共享存储。

这里讲第一种方式。

修改两配置文件:红色标识在原来基础上变化的地方

1.core-site.xml

<configuration>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/hadoop-2.6.0-cdh5.8.0/tmp</value>
  </property>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://mycluster</value>
  </property>

  <property>
  <name>hadoop.proxyuser.hadoop.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hadoop.groups</name>
<value>*</value>
</property>
 <property>
   <name>ha.zookeeper.quorum</name>
   <value>master:2181,slave1:2181,slave2:2181</value>
 </property>

</configuration>

2.hdfs-site.xml

扫描二维码关注公众号,回复: 4539540 查看本文章

<configuration>
<property>
<name>
dfs.replication
</name>
<value>
3
</value>
</property>
<property>
<name>
dfs.namenode.name.dir
</name>
<value>
file:/opt/hdfs/name
</value>
</property>
<property>
<name>
dfs.namenode.edits.dir
</name>
<value>
file:/opt/hdfs/nameedit
</value>
</property>
<property>
<name>
dfs.datanode.data.dir
</name>
<value>
file:/opt/hdfs/data
</value>
</property>
<property>
<name>dfs.namenode.rpc-address</name>
<value>master:9000</value>
</property>
<property>
<name>dfs.http.address</name><value>master:50070</value></property>
<property><name>dfs.namenode.secondary.http-address</name><value>master:50090</value></property>
<property><name>dfs.webhdfs.enabled</name><value>true</value></property>
<property><name>dfs.permissions</name><value>false</value></property>
<property>
  <name>dfs.nameservices</name>
  <value>mycluster</value>
</property>
<property>
  <name>dfs.ha.namenodes.mycluster</name>
  <value>master,slave1</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.mycluster.master</name>
  <value>master:8020</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.mycluster.slave1</name>
  <value>slave1:8020</value>
</property>
<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://master:8485;slave1:8485;slave2:8485/mycluster</value>
</property>
<property>
  <name>dfs.client.failover.proxy.provider.mycluster</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
  <name>dfs.ha.fencing.methods</name>
  <value>shell(/bin/true)</value>
</property>
<property>
  <name>dfs.journalnode.edits.dir</name>
  <value>/opt/hdfs/journal/data</value>
</property>
<property>
  <name>dfs.namenode.http-address.mycluster.master</name>
  <value>master:50070</value>
</property>
<property>
  <name>dfs.namenode.http-address.mycluster.slave1</name>
  <value>slave1:50070</value>
</property>

</configuration>

3.启动hadoop平台

a ..在slave1节点执行,我把slave1作为另一namenode:

[hadoop@slave1 bin]$ ./hdfs namenode -bootstrapStandby
17/04/26 19:16:02 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   user = hadoop
STARTUP_MSG:   host = slave1/192.168.145.129
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.6.0-cdh5.8.0
STARTUP_MSG:   classpath = /opt/hadoop-2.6.0-cdh5.8.0/etc/hadoop:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/junit-4.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/activation-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/logredactor-1.0.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/gson-2.2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hadoop-auth-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/zookeeper-3.4.5-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-el-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hadoop-annotations-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/avro-1.7.6-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-nfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-common-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/htrace-core4-4.0.1-incubating.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/zookeeper-3.4.5-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jline-2.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/avro-1.7.6-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.8.0.jar:/opt/hive-1.1.0-cdh5.8.0/conf:/opt/hive-1.1.0-cdh5.8.0/lib:/opt/hadoop-2.6.0-cdh5.8.0//contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://github.com/cloudera/hadoop -r 042da8b868a212c843bcbf3594519dd26e816e79; compiled by 'jenkins' on 2016-07-12T23:02Z
STARTUP_MSG:   java = 1.7.0_79
************************************************************/
17/04/26 19:16:02 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
17/04/26 19:16:02 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
=====================================================
About to bootstrap Standby ID slave1 from:
           Nameservice ID: mycluster
        Other Namenode ID: master
  Other NN's HTTP address: http://master:50070
  Other NN's IPC  address: master/192.168.145.128:8020
             Namespace ID: 536081338
            Block pool ID: BP-1319270767-192.168.145.128-1488325418583
               Cluster ID: CID-dcf4f2d4-62ab-4da2-a27b-9a1c5f531dec
           Layout version: -60
       isUpgradeFinalized: true
=====================================================
17/04/26 19:16:04 INFO common.Storage: Storage directory /opt/hdfs/name has been successfully formatted.
17/04/26 19:16:04 INFO common.Storage: Storage directory /opt/hdfs/nameedit has been successfully formatted.
17/04/26 19:16:06 INFO namenode.TransferFsImage: Opening connection to http://master:50070/imagetransfer?getimage=1&txid=20178&storageInfo=-60:536081338:0:CID-dcf4f2d4-62ab-4da2-a27b-9a1c5f531dec&bootstrapstandby=true
17/04/26 19:16:06 INFO namenode.TransferFsImage: Image Transfer timeout configured to 60000 milliseconds
17/04/26 19:16:06 INFO namenode.TransferFsImage: Transfer took 0.11s at 245.45 KB/s
17/04/26 19:16:06 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000020178 size 27760 bytes.
17/04/26 19:16:06 INFO util.ExitUtil: Exiting with status 0
17/04/26 19:16:06 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at slave1/192.168.145.129

************************************************************/

b..如果将非HA NameNode转换为HA,则应运行命令“hdfs -initializeSharedEdits”,该命令将使用本地NameNode编辑目录中的编辑数据初始化JournalNodes。


4. 下载zookeeper-3.4.5-cdh5.8.0.tar.gz并在master,slave1,slave2上解压安装。

 a.   tar -zxvf  zookeeper-3.4.5-cdh5.8.0.tar.gz

 b.  cp /opt/zookeeper-3.4.5-cdh5.8.0/conf/zoo_sample.cfg /opt/zookeeper-3.4.5-cdh5.8.0/conf/zoo.cfg

c. 编辑zoo.cfg如下:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/opt/zookeeper-3.4.5-cdh5.8.0/data
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888

d:创建myid文件,文件内容分别为1,2,3分别上传至master,slave1,slave2

e:在三台机器上执行

  ./zkServer.sh start

f: 在master执行 hdfs zkfc -formatZK

[hadoop@master bin]$ ./hdfs zkfc -formatZK
17/05/04 10:24:12 INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at master/192.168.145.128:8020
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.8.0--1, built on 07/12/2016 20:39 GMT
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:host.name=master
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_79
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.7.0_79/jre
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hadoop-2.6.0-cdh5.8.0/etc/hadoop:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hadoop-auth-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/avro-1.7.6-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/hadoop-annotations-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/junit-4.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/activation-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/zookeeper-3.4.5-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/logredactor-1.0.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/gson-2.2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-el-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-nfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/common/hadoop-common-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/htrace-core4-4.0.1-incubating.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jetty-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jline-2.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/zookeeper-3.4.5-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/avro-1.7.6-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.0-tests.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.0.jar:/opt/hadoop-2.6.0-cdh5.8.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.8.0.jar:/opt/hive-1.1.0-cdh5.8.0/conf:/opt/hive-1.1.0-cdh5.8.0/lib:/opt/hadoop-2.6.0-cdh5.8.0//contrib/capacity-scheduler/*.jar
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/hadoop-2.6.0-cdh5.8.0/lib/native
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-573.el6.x86_64
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/hadoop-2.6.0-cdh5.8.0/bin
17/05/04 10:24:12 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181,slave1:2181,slave2:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@2619d129
17/05/04 10:24:12 INFO zookeeper.ClientCnxn: Opening socket connection to server slave2/192.168.145.130:2181. Will not attempt to authenticate using SASL (unknown error)
17/05/04 10:24:12 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /192.168.145.128:33676, server: slave2/192.168.145.130:2181
17/05/04 10:24:13 INFO zookeeper.ClientCnxn: Session establishment complete on server slave2/192.168.145.130:2181, sessionid = 0x35bd142a0ce0000, negotiated timeout = 5000
17/05/04 10:24:13 INFO ha.ActiveStandbyElector: Session connected.
17/05/04 10:24:13 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/mycluster in ZK.
17/05/04 10:24:13 INFO zookeeper.ClientCnxn: EventThread shut down
17/05/04 10:24:13 INFO zookeeper.ZooKeeper: Session: 0x35bd142a0ce0000 closed

[hadoop@master bin]$ cd ../sbin
[hadoop@master sbin]$ ./start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [slave1 master]
master: starting namenode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-namenode-master.out
slave1: starting namenode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-namenode-slave1.out
slave1: starting datanode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-datanode-slave1.out
slave2: starting datanode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-datanode-slave2.out
Starting journal nodes [master slave1 slave2]
slave2: starting journalnode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-journalnode-slave2.out
slave1: starting journalnode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-journalnode-slave1.out
master: starting journalnode, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-journalnode-master.out
Starting ZK Failover Controllers on NN hosts [slave1 master]
master: starting zkfc, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-zkfc-master.out
slave1: starting zkfc, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/hadoop-hadoop-zkfc-slave1.out
starting yarn daemons
starting resourcemanager, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/yarn-hadoop-resourcemanager-master.out
slave1: starting nodemanager, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/yarn-hadoop-nodemanager-slave1.out
slave2: starting nodemanager, logging to /opt/hadoop-2.6.0-cdh5.8.0/logs/yarn-hadoop-nodemanager-slave2.out
[hadoop@master sbin]$ jps
3827 NameNode
4235 ResourceManager
4150 DFSZKFailoverController
4017 JournalNode
4546 Jps
3366 QuorumPeerMain
[hadoop@master sbin]$ 



5.验证可用性


5.验证可用性 从上两图。知道master namenode是active.

现在kill master 的namenode进程。

则slave1的namenode应友standby变为active

[hadoop@master sbin]$ jps
9920 DFSZKFailoverController
9756 JournalNode
9563 NameNode
10098 Jps
10023 ResourceManager
3366 QuorumPeerMain
[hadoop@master sbin]$ jps
9920 DFSZKFailoverController
9756 JournalNode
9563 NameNode
10305 Jps
10023 ResourceManager
3366 QuorumPeerMain
[hadoop@master sbin]$ kill -9 9563
[hadoop@master sbin]$ 


猜你喜欢

转载自blog.csdn.net/feilong2483/article/details/70821314
今日推荐