HBase2.2.2 installation and programming practice guide
1. Introduction to HBase
HBase is a distributed, column-oriented open source database, derived from a Google paper "BigTable: A Distributed Storage System for Structured Data". HBase stores data in the form of tables, which consist of rows and columns, and columns are divided into several column families/column families. For official information about HBase, please visit the official website of HBase .
HBase operates in three modes: stand-alone mode, pseudo-distributed mode, and distributed mode.
Stand-alone mode: install and use HBase on one computer without distributed storage of data; pseudo-distributed mode: simulate a small cluster on one computer; distributed mode: use multiple computers to achieve physical Distributed storage. For learning purposes here, we only focus on the stand-alone mode and the pseudo-distributed mode. The operating environment of this tutorial is under the Ubuntu-64-bit system (such as Ubuntu18.04, Ubuntu16.04 or Ubuntu14.04), and the HBase version is hbase-2.2.2, which is a compiled and stable version that has been released so far. The file with src is the uncompiled version, here we just need to download the bin version hbase-2.2.2-bin.tar.gz
1. If the reader is a user who uses a virtual machine to install the Ubuntu system, please use the Firefox browser that comes with Ubuntu in the virtual machine to access this guide, and then click the address below to download the HBase file to the virtual machine Ubuntu. Please do not use the browser under the Windows system to download, the file will be downloaded to the Windows system, and the Ubuntu in the virtual machine cannot access the files of the external Windows system, causing unnecessary trouble.
2. If the reader is a user who installs the Ubuntu system in dual-system mode, please run the Ubuntu system, open the firefox browser on the Ubuntu system to access this guide, and then click the link below to download the HBase download link
2. Install and configure HBase2.2.2
Before installing HBase, please make sure that Hadoop3.1.3 has been installed on your computer. Since HBase has a version dependency on Hadoop, when installing HBase2.2.2, you must first install Hadoop3.1.3 (see the installation method).
- HBase2.2.2 installation
Click the address below to download the HBase2.2.2 installation file HBase official website download address
You can also directly click here to download the software from Baidu cloud disk (extraction code: lnwl). After entering the Baidu network disk, enter the "software" directory, find the hbase-2.2.2-bin.tar.gz file, and download it locally.
1.1 Unzip the installation package hbase-2.2.2-bin.tar.gz to the path /usr/local, the command is as follows:
1.cd ~
2.sudo tar -zxf ~/下载/hbase-2.2.2-bin.tar.gz -C /usr/local
Shell command
1.2 Change the decompressed file name from hbase-2.2.2 to hbase for ease of use. The command is as follows:
1.cd /usr/local
2.sudo mv ./hbase-2.2.2 ./hbase Under
the Shell command, hbase
The directory permission is given to the hadoop user:
1.cd /usr/local
2.sudo chown -R hadoop ./hbase
Shell command
1.3 Configure environment variables
Add the bin directory under hbase to the path, so that you don’t need to go to /usr to start hbase The /local/hbase directory greatly facilitates the use of hbase. The following part of the tutorial still switches to the /usr/local/hbase directory operation, which is helpful for beginners to understand the running process, and it is not necessary to switch after being proficient.
Edit ~/.bashrc file
1.vim ~/.bashrc
Shell command
If PATH has not been introduced, please add the following content at the end of ~/.bashrc file:
1.export PATH=$PATH:/usr/local/hbase/bin
Shell
if If PATH has been introduced, please add /usr/local/hbase/bin to the export PATH line, where ":" is a separator. As shown below:
After editing, execute the source command to make the above configuration take effect immediately on the current terminal. The command is as follows:
1.source ~/.bashrc
Shell command
Extended reading: The method and difference of setting Linux environment variables
1.4 Add HBase permission
1.cd /usr/ local
2.sudo chown -R hadoop ./hbase #Change the owner of all files under hbase to hadoop, where hadoop is the username of the current user.
Shell command
1.5 Check the HBase version to confirm that the hbase installation is successful. The command is as follows:
1./usr/local/hbase/bin/hbase version
The shell command
sees the output version message, indicating that HBase has been installed successfully. Next, HBase stand-alone mode and Configuration for pseudo-distributed mode.
2. HBase configuration
HBase has three operating modes, stand-alone mode, pseudo-distributed mode, and distributed mode. As a study, we focus on the stand-alone mode and the pseudo-distributed mode.
The following prerequisites are very important. For example, if the JAVA_HOME environment variable is not configured, an error will be reported.
– jdk
– Hadoop (not required for stand-alone mode, but required for pseudo-distributed mode and distributed mode) – If the above three
SSH are not installed, please refer to the installation of Hadoop3.1.3 for how to install.
2.1 Stand-alone mode configuration
- Configure /usr/local/hbase/conf/hbase-env.sh. Configure the JAVA environment variable, and add the configuration HBASE_MANAGES_ZK to true, use the vi command to open and edit hbase-env.sh, the command is as follows:
1. vim /usr/local/hbase/conf/hbase-env.sh
Shell command
to configure the JAVA environment variable . If you have installed Hadoop3.1.3 according to the Hadoop installation tutorial on this website, you have already installed JDK1.8. The installation directory of JDK is /usr/lib/jvm/jdk1.8.0_162, then JAVA _HOME =/usr/lib/jvm/jdk1.8.0_162; configure HBASE_MANAGES_ZK to true, which means that zookeeper is managed by hbase itself, and no separate zookeeper is needed . The configuration of these variables already exists in hbase-env.sh, you only need to delete the preceding # and modify the configuration content (# represents comments):
1.export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_162
2 .export HBASE_MANAGES_ZK=true Save and exit after
the Shell
is added.
2. Configure /usr/local/hbase/conf/hbase-site.xml
Open and edit hbase-site.xml, the command is as follows:
1.vim /usr/local/hbase/conf/hbase-site.xml
The shell command
needs to set the attribute hbase.rootdir before starting HBase, which is used to specify the storage location of HBase data , because if not set, hbase.rootdir defaults to /tmp/hbase-${user.name}, which means that data will be lost every time the system is restarted. Here it is set to the hbase-tmp folder under the HBase installation directory (/usr/local/hbase/hbase-tmp), add the configuration as follows: 1. 2. 3. hbase.rootdir 4.
file
:
///
usr/ local/hbase/hbase-tmp
5.
6.
XML
3. Next test run. First switch the directory to the HBase installation directory /usr/local/hbase; then start HBase. The command is as follows:
1.cd /usr/local/hbase
2.bin/start-hbase.sh
3.bin/hbase shell
Shell command
Among the above three commands, sudo bin/start-hbase.sh is used to start HBase, and bin/hbase shell is used to Open the shell command line mode, and users can operate the HBase database by entering shell commands.
Start HBase successfully, the screenshot is as follows:
To stop HBase running, the command is as follows:
1.bin/stop-hbase.sh
Shell command
Note: If an error occurs during the operation of HBase, you can log in the logs subdirectory under the {HBASE_HOME} directory (/usr/local/hbase) Check the log file for the cause of the error.
2.2 Pseudo-distributed mode configuration
1. Configure /usr/local/hbase/conf/hbase-env.sh. The command is as follows:
1.vim /usr/local/hbase/conf/hbase-env.sh
Shell command
to configure JAVA_HOME, HBASE_CLASSPATH, HBASE_MANAGES_ZK.
HBASE_CLASSPATH is set to the conf directory under the local HBase installation directory (ie /usr/local/hbase/conf)
1 .export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_162
2.export HBASE_CLASSPATH=/usr/local/hbase/conf
3.export HBASE_MANAGES_ZK=true
Shell
2. Configure /usr/local/hbase/conf/hbase-site.xml
Use the command vi to open and edit hbase-site.xml, the command is as follows:
1. vim /usr/local/hbase/conf/hbase-site.xml
Shell command
to modify hbase.rootdir, specify the storage path of HBase data on HDFS; set The property hbase.cluter.distributed is set to true. Assume that the current Hadoop cluster is running in pseudo-distributed mode, running on the local machine, and the NameNode is running on port 9000.
1.
2.
3. hbase.rootdir
4. hdfs://localhost:9000/hbase
5.
6.
7. hbase.cluster.distributed
8. true
9.
10.
11. hbase.unsafe.stream.capability.enforce
12 . false
13.
14.
XML
hbase.rootdir specifies the storage directory of HBase; hbase.cluster.distributed sets the cluster to be in distributed mode.
In addition, in the configuration file above, the property hbase.unsafe.stream.capability.enforce is set to avoid startup errors. That is to say, if hbase.unsafe.stream.capability.enforce is not set to false, then after starting HBase, there will be an error that the HMaster process cannot be found. After starting, check the system startup log (/usr/local/hbase/logs /hbase-hadoop-master-ubuntu.log), the following error will be found:
2020-01-25 15:04:56,916 ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to become active master
java.lang. IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
3. Next, test and run HBase.
Step 1: Log in to ssh first, and no password is set to log in before, so no password is required here; then switch the directory to /usr/local/hadoop; then start hadoop, if you have already started hadoop, please skip this step. The command is as follows:
1.ssh localhost
2.cd /usr/local/hadoop
3.../sbin/start-dfs.sh
Shell command
input command jps, you can see that the NameNode, DataNode and SecondaryNameNode have been successfully started, indicating that hadoop started successfully, Screenshot below:
Step 2: Switch the directory to /usr/local/hbase; start HBase again. The command is as follows:
1.cd /usr/local/hbase
2.bin/start-hbase.sh
The shell command
starts successfully, enter the command jps, and see The following interface shows that hbase started successfully
Enter the shell interface:
1.bin/hbase shell
Shell command
screenshot is as follows:
4. Stop HBase running, the command is as follows:
1.bin/stop-hbase.sh
Shell command
Note: If an error occurs during the operation of HBase, you can view the cause of the error through the log files in the logs subdirectory under the {HBASE_HOME} directory (/usr/local/hbase).
The order of starting and closing Hadoop and HBase here must be:
start Hadoop—>start HBase—>close HBase—>close Hadoop
