Basic knowledge of HBase (2): HBase cluster deployment, HBaseShell operation

1. HBase installation and deployment

1.1 Normal deployment of Zookeeper

First ensure the normal deployment of the Zookeeper cluster and start it:

Create a cluster startup script:

#!/bin/bash
case $1 in
"start"){
for i in hadoop100 hadoop101 hadoop102
do
echo----------zookeeper $i 启动------------
ssh $i "/opt/module/zookeeper-3.4.10/bin/zkServer.sh start"
done
};;
​
"stop"){
for i in hadoop100 hadoop101 hadoop102
do
echo----------zookeeper $i 停止------------
ssh $i "/opt/module/zookeeper-3.4.10/bin/zkServer.sh stop"
done
};;
​
"status"){
for i in hadoop100 hadoop101 hadoop102
do
echo----------zookeeper $i 状态------------
ssh $i "/opt/module/zookeeper-3.4.10/bin/zkServer.sh status"
done
};;
esac

1.2 Hadoop normal deployment

Normal deployment and startup of Hadoop cluster:

sbin/start-dfs.sh
sbin/start-yarn.sh

1.3 HBase decompression

Unzip Hbase to the specified directory:

tar -zxvf hbase-1.3.1-bin.tar.gz -C /opt/module

1.4 HBase configuration file

Modify the corresponding configuration file of HBase.

1) hbase-env.sh modification content:

First get the java information of your own environment

echo $JAVA_HOME
/opt/module/jdk1.8.0_161

Annotate the following information:

Modify the information:

export JAVA_HOME=/opt/module/jdk1.8.0_161
export HBASE_MANAGES_ZK=false

2) hbase-site.xml modified content:

<?xmlversion="1.0"?>
<configuration>
<property>
    <name>hbase.rootdir</name>
    <value>hdfs://hadoop100:8020/HBase</value>
</property>
    
<property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
</property>
<!--0.98后的新变动，之前版本没有.port,默认端口为60000-->
<property>
    <name>hbase.master.port</name>
    <value>16000</value>
</property>
<property>
    <name>hbase.zookeeper.quorum</name>
    <value>hadoop100,hadoop101,hadoop102</value>
</property>
<property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/opt/module/zookeeper-3.4.10/zkData</value>
</property>
</configuration>

3）regionservers：

hadoop100
hadoop101
hadoop102

4) Softly connect the hadoop configuration file to HBase:

ln -s /opt/module/hadoop-3.1.3/etc/hadoop/core-site.xml/opt/module/hbase-1.3.1/conf/core-site.xml
ln -s /opt/module/hadoop-3.1.3/etc/hadoop/hdfs-site.xml/opt/module/hbase-1.3.1/conf/hdfs-site.xml

You can see that both files are referenced:

1.5 HBase remotely sends to other clusters

xsync.sh hbase-1.3.1

1.6 Starting the HBase service

1. Start mode

bin/hbase-daemon.sh start master
startingmaster,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-master-hadoop100.out
bin/hbase-daemon.sh start regionserver
startingregionserver,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-regionserver-hadoop100.out

Tip: If the node times between clusters are not synchronized, the region server will fail to start and a ClockOutOfSyncException exception will be thrown. Repair tips:

a. For time synchronization service, please refer to the help document: "Getting Started with Hadoop in Silicon Valley Big Data Technology"

b. Attribute: hbase.master.maxclockskew set a larger value

<property>
    <name>hbase.master.maxclockskew</name>
    <value>180000</value>
    <description>Timedifferenceofregionserverfrommaster</description>
</property>

2. Start mode

bin/start-hbase.sh

You can see the output:

startingmaster,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-master-hadoop100.out
hadoop101:startingregionserver,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-regionserver-hadoop101.out
hadoop102:startingregionserver,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-regionserver-hadoop102.out
hadoop100:startingregionserver,loggingto/opt/module/hbase-1.3.1/bin/../logs/hbase-atguigu-regionserver-hadoop100.out

Corresponding stop service:

bin/stop-hbase.sh

2. HBaseShell operation

2.1 Basic operations

1. Enter the HBase client command line

bin/hbaseshell

2. View help commands

hbase(main):001:0>help
HBaseShell,version1.3.1,r930b9a55528fe45d8edce7af42fef2d35e77677a,ThuApr619:36:54PDT2017
Type'help"COMMAND"',(e.g.'help"get"'--thequotesarenecessary)forhelponaspecificcommand.
Commandsaregrouped.Type'help"COMMAND_GROUP"',(e.g.'help"general"')forhelponacommandgroup.
​
COMMANDGROUPS:
Groupname:general
Commands:status,table_help,version,whoami
​
#Groupname:ddl
#Commands:alter,alter_async,alter_status,create,describe,disable,disable_all,drop,drop_all,enable,enable_all,exists,get_table,is_disabled,is_enabled,list,locate_region,show_filters
​
Groupname:namespace
Commands:alter_namespace,create_namespace,describe_namespace,drop_namespace,list_namespace,list_namespace_tables
​
#Groupname:dml
#Commands:append,count,delete,deleteall,get,get_counter,get_splits,incr,put,scan,truncate,truncate_preserve
​
Groupname:tools
Commands:assign,balance_switch,balancer,balancer_enabled,catalogjanitor_enabled,catalogjanitor_run,catalogjanitor_switch,close_region,compact,compact_rs,flush,major_compact,merge_region,move,normalize,normalizer_enabled,normalizer_switch,split,splitormerge_enabled,splitormerge_switch,trace,unassign,wal_roll,zk_dump
​
Groupname:replication
Commands:add_peer,append_peer_tableCFs,disable_peer,disable_table_replication,enable_peer,enable_table_replication,get_peer_config,list_peer_configs,list_peers,list_replicated_tables,remove_peer,remove_peer_tableCFs,set_peer_tableCFs,show_peer_tableCFs
​
Groupname:snapshots
Commands:clone_snapshot,delete_all_snapshot,delete_snapshot,delete_table_snapshots,list_snapshots,list_table_snapshots,restore_snapshot,snapshot
​
Groupname:configuration
Commands:update_all_config,update_config
​
Groupname:quotas
Commands:list_quotas,set_quota
​
Groupname:security
Commands:grant,list_security_capabilities,revoke,user_permission
​
Groupname:procedures
Commands:abort_procedure,list_procedures
​
Groupname:visibilitylabels
Commands:add_labels,clear_auths,get_auths,list_labels,set_auths,set_visibility
​
SHELLUSAGE:
QuoteallnamesinHBaseShellsuchastableandcolumnnames.Commasdelimit
commandparameters.Type<RETURN>afterenteringacommandtorunit.
Dictionariesofconfigurationusedinthecreationandalterationoftablesare
RubyHashes.Theylooklikethis:
​
{'key1'=>'value1','key2'=>'value2',...}
​
andareopenedandclosedwithcurley-braces.Key/valuesaredelimitedbythe
'=>'charactercombination.Usuallykeysarepredefinedconstantssuchas
NAME,VERSIONS,COMPRESSION,etc.Constantsdonotneedtobequoted.Type
'Object.constants'toseea(messy)listofallconstantsintheenvironment.
​
Ifyouareusingbinarykeysorvaluesandneedtoenterthemintheshell,use
double-quote'dhexadecimalrepresentation.Forexample:
​
hbase>get't1',"key\x03\x3f\xcd"
hbase>get't1',"key\003\023\011"
hbase>put't1',"test\xef\xff",'f1:',"\x01\x33\x40"
​
TheHBaseshellisthe(J)RubyIRBwiththeaboveHBase-specificcommandsadded.
FormoreontheHBaseShell,seehttp://hbase.apache.org/book.html

3．Check what tables are in the current database

hbase(main):002:0>list
TABLE
0row(s)in0.6350seconds
​
=>[]

2.2 Table operations

1. Create table

hbase(main):003:0>create 'student','info'
0row(s)in2.4680seconds
​
=>Hbase::Table-student
hbase(main):004:0>create 'stu','info1','info2'
0row(s)in1.2800seconds
​
=>Hbase::Table-stu
hbase(main):005:0>list
TABLE
stu
student
2row(s)in0.0200seconds
​
=>["stu","student"]

2. Insert data into table

hbase(main):003:0>put 'student','1001','info:sex','male'
hbase(main):004:0>put 'student','1001','info:age','18'
hbase(main):005:0>put 'student','1002','info:name','Janna'
hbase(main):006:0>put 'student','1002','info:sex','female'
hbase(main):007:0>put 'student','1002','info:age','20'

3．Scan to view table data

hbase(main):008:0>scan 'student'
hbase(main):009:0>scan 'student',{STARTROW=>'1001',STOPROW=>'1001'}
hbase(main):010:0>scan 'student',{STARTROW=>'1001'}

4．View table structure

describe ‘student’

5．Update the data of the specified field

hbase(main):012:0>put 'student','1001','info:name','Nick'
hbase(main):013:0>put 'student','1001','info:age','100'

6．View data for "specified row" or "specified column family: column"

hbase(main):014:0>get 'student','1001'
hbase(main):015:0>get 'student','1001','info:name'

7．Number of rows of statistics table data

hbase(main):021:0>count 'student'

8. delete data

Delete all data of a rowkey:

hbase(main):016:0>deleteall 'student','1001'

Delete a column of data for a rowkey:

hbase(main):017:0>delete 'student','1002','info:sex'

9. Clear table data

hbase(main):018:0>truncate 'student'

Tip: The order of operations to clear the table is disable first, then truncate.

10．Delete table

First, you need to put the table in the disabled state:

hbase(main):019:0>disable 'student'

Then you can drop this table:

hbase(main):020:0>drop 'student'

Tip: If you drop the table directly, an error will be reported: ERROR: Tablestudentisenabled.Disableitfirst.

11．Change table information

Store the data in the info column family in 3 versions:

hbase(main):022:0>alter 'student',{NAME=>'info',VERSIONS=>3}
hbase(main):022:0>get 'student','1001',{COLUMN=>'info:name',VERSIONS=>3}