Hbase--architecture and deployment

1. Architecture

• 分布式主从架构
  • Master: HMaster
    • Manage slave nodes
  • From: HRegionServer
    • Responsible for work: storage
    • Data is stored in the memory of RegionServer
• HDFS
  • 用于实现Hbase中数据的持久化
• Zookeeper
  • 存储关键性数据
  • 辅助选举: Two HMasters, one active, one standby
  • Hbase is highly dependent on Zookeeper
    • Most distributed frameworks rely on Zookeeper
    • Unless you implement a mechanism similar to Zookeeper
      

2. Distributed cluster deployment

First select a machine for installation and configuration

• Upload

cd /export/software/
rz

• Unzip and install

tar -zxvf hbase-1.2.0-cdh5.14.0.tar.gz -C /export/servers/

• Directory Structure
  • bin: The directory used to store commands
  • conf: configuration file directory
  • lib: the directory of the dependent package
  • logs: The directory where logs are stored

Change setting

• hbase-env.sh: modify the environment variables of Hbase runtime

#修改第27行
export JAVA_HOME=/export/servers/jdk1.8.0_141
#修改128行
export HBASE_MANAGES_ZK=false

• hbase-site.xml: Configure the properties of Hbase
  • For all hbase programs, load hbase-default.xml first, and then load hbase-site.xml
  • Create a directory

cd /export/servers/hbase-1.2.0-cdh5.14.0/
mkdir  datas

<!--指定Hbase 临时存储目录的-->
  <property >
    <name>hbase.tmp.dir</name>
    <value>/export/servers/hbase-1.2.0-cdh5.14.0/datas</value>
  </property>
  <!--指定Hbase在HDFS上的存储目录-->
  <property >
    <name>hbase.rootdir</name>
    <value>hdfs://node-01:8020/hbase</value>
  </property>
  <property >
    <name>hbase.fs.tmp.dir</name>
    <value>/user/${user.name}/hbase-staging</value>
  </property>
  <property >
    <name>hbase.bulkload.staging.dir</name>
    <value>${hbase.fs.tmp.dir}</value>
  </property>
  <!--指定Hbase的运行模式是否是分布式-->
  <property >
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <!--指定Zookeeper的地址-->
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>node-01:2181,node-02:2181,node-03:2181</value>
  </property>

• If hdfs is configured with HA
  • Note that to put core-site.xml and hdfs-site.xml into the conf directory of hbase
• regionservers: Configure the address of the hbase slave node, similar to the slaves file of Hadoop

node-01
node-02
node-03

Why is the address of the master node not specified?

• Because the bottom layer of Hbase is highly dependent on Zookeeper, it is directly associated with zookeeper
• You can start the master node on any machine
• The first machine to start is the active master node
• Other machines start the master node is the master node of the standby
• If you have to define which machines can start the master node process HMaster
  
• achieve

cd /export/servers/hbase-1.2.0-cdh5.14.0/
vim conf/backup-masters

node-02

distribution

cd /export/servers/
scp -r hbase-1.2.0-cdh5.14.0 node-02:$PWD
scp -r hbase-1.2.0-cdh5.14.0 node-03:$PWD

start up

• Start HDFS first
  • Hbase can be started only after HDFS exits safe mode

start-dfs.sh 

• Then start Zookeeper
  • Without scripts, start one process at a time
  • After the startup is complete, be sure to check the status of zookeeper

/export/servers/zookeeper-3.4.5-cdh5.14.0/bin/start-zk-all.sh 

• Then start Hbase
  • Method 1: The first machine starts the processes of all machines at once

cd /export/servers/hbase-1.2.0-cdh5.14.0/
bin/start-hbase.sh 

• shut down
  • Close hbase first
  • Turn off hdfs or zookeeper again

bin/stop-hbase.sh

• note:
  • Started process
    
• Warning don't worry
  
• Method 2: Start a single process

cd /export/servers/hbase-1.2.0-cdh5.14.0/
bin/hbase-daemon.sh start master
bin/hbase-daemon.sh start regionserver

Test visit

• jps: Check whether the process is complete
  • If a process disappears automatically, please check the log
  • Where is the log?
    

tail -100f logs/hbase-root-master-node-01.log

• Visit WebUI

node-01:60010
node-02:60010

注意：hbase我们使用的是1.x版本
	apache从1.x开始网页端口更改为16010
	cdh延续以前老版本的端口：60010

• The first configuration environment variable

vim /etc/profile

export HBASE_HOME=/export/servers/hbase-1.2.0-cdh5.14.0
export PATH=$PATH:$HBASE_HOME/bin

source /etc/profile