Hbase1.2.2 cluster + Hadoop cluster construction

HBase installation

There are also three modes of HBase installation: stand-alone mode, pseudo-distributed mode and fully distributed mode. Only the fully distributed mode is introduced here. The premise is that the Hadoop cluster and Zookeeper have been installed and run correctly. 


Step 1: Download the installation package, unzip it to a suitable location, and assign permissions to the hadoop user (the account running hadoop).
Here, hbase-1.2.2 is downloaded, and the Hadoop cluster uses 2.7.2. Unzip it to / usr/local and renamed to hbase

wget http://mirror.bit.edu.cn/apache/hbase/1.2.2/hbase-1.2.2-bin.tar.gz
sudo cp hbase-1.2.2-bin.tar.gz /usr/local
sudo tar -zxf hbase-1.2.2-bin.tar.gz
sudo mv hbase-1.2.2 hbase
sudo chown -R hadoop:hadoop hbase

 

 Step 2 : Configure related files
(1) Configure hbase-env.sh, which is in /usr/local/hbase/conf

export JAVA_HOME=/usr/local/java/jdk1.8.0_77 #Java installation path
export HBASE_CLASSPATH=/usr/local/hadoop/conf #HBase class path
export HBASE_MANAGES_ZK=true #HBase is responsible for starting and closing Zookeeper

 (2) Configure hbase-site.xml, which is located in /usr/local/hbase/conf 

   <property>
           <name>hbase.master</name>
           <value>master:6000</value>
   </property>
   <property>
           <name>hbase.master.maxclockskew</name>
           <value>180000</value>
   </property>
   <property>
           <name>hbase.rootdir</name>
           <value>hdfs://master:9000/hbase</value>
   </property>
   <property>
           <name>hbase.cluster.distributed</name>
           <value>true</value>
   </property>
   <property>
           <name>hbase.zookeeper.quorum</name>
           <value>master,node1,node2</value>
   </property>
   <property>
           <name>hbase.zookeeper.property.dataDir</name>
           <value>/home/hadoop/zookeeper</value>
   </property>
   <property>
           <name>dfs.replication</name>
           <value>1</value>
   </property>
  其中,hbase.master是指定运行HMaster的服务器及端口号; hbase.master.maxclockskew是用来防止HBase节点之间时间不一致造成regionserver启动失败,默认值是30000 hbase.rootdir指定HBase的存储目录; hbase.cluster.distributed设置集群处于分布式模式; hbase.zookeeper.quorum设置Zookeeper节点的主机名,它的值个数必须是奇数;hbase.zookeeper.property.dataDir设置Zookeeper的目录,默认为/tmp; dfs.replication设置数据备份数,集群节点小于3时需要修改,本次试验是一个节点,所以修改为1
(3)配置regionservers,该文件位于/usr/local/hbase/conf

设置所运行HBase的机器,此文件配置和hadoop中的slaves类似,一行指定一台机器,本次试验仅用一台机器,设置master即可。

node1
node2

 (4)设置HBase环境变量,文件位于/etc/profile在文件末尾添加:  

#hbase Env
export HBASE_HOME=/usr/local/hbase
export PATH=$PATH:$HBASE_HOME/bin

 使之生效:source /etc/profile

 

(5)修改hadoop集群及hbase集群pid文件存放位置(如果不设置话),停止hbase集群会报如下错误 

stopping hbasecat: /var/hadoop/pids/hbase-hadoop-master.pid: 没有那个文件或目录

 

  1. 在集群各个节点的/var目录下创建一个文件夹  
    sudo mkdir -p /var/hadoop/pids chown -R hadoop:hadoop /var/hadoop
  2. 修改hadoop-env.sh(/usr/local/hadoop/etc/hadoop/hadoop-env.sh)
    export HADOOP_PID_DIR=/var/hadoop/pids 
  3. 修改yarn-env.sh(/usr/local/hadoop/etc/hadoop/yarn-env.sh)
    export YARN_PID_DIR=/var/hadoop/pids
  4. 修改hbase-env.sh(/usr/local/hbase/conf/hbase-env.sh)
    export HBASE_PID_DIR=/var/hadoop/pids 

第三步:同步hbase到其他机器 

scp -r hbase hadoop@node1:/usr/local
scp -r hbase hadoop@node2:/usr/local

   

第四步:启动hadoop和hbase集群(启动顺序hadoop>hbase)

start-all.sh
start-hbase.sh

第五步:查看pid文件(/var/hadoop/pids) 

master机器 

hadoop-hadoop-namenode.pid  hadoop-hadoop-secondarynamenode.pid  
hbase-hadoop-master.pid hbase-hadoop-master.znode  
hbase-hadoop-zookeeper.pid  yarn-hadoop-resourcemanager.pid

 slave机器

hadoop-hadoop-datanode.pid  hbase-hadoop-regionserver.pid  
hbase-hadoop-regionserver.znode  hbase-hadoop-zookeeper.pid  
yarn-hadoop-nodemanager.pid

第六步:查看hbase运行情况 

hadoop@master:/var/hadoop/pids$ jps
26499 ResourceManager
26995 HQuorumPeer
27059 HMaster
27339 Jps
26123 NameNode
26335 SecondaryNameNode

hadoop@node1:/var/hadoop/pids$ jps
10678 HQuorumPeer
10775 HRegionServer
10362 DataNode
10493 NodeManager
10990 Jps

第七步:关闭hbase集群(关闭顺序hbase>hadoop)

stop-hbase.sh
stop-all.sh

 

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=326713238&siteId=291194637