HBase and Hive interoperate Case

HBase and Hive interaction

1, environment preparation

Because we follow may also affect HBase Hive while operating, it is required to hold the operation of HBase Hive Jar, then the next copy Hive Jar package depends (or in the form of soft-wired).

$ export HBASE_HOME=/opt/modules/cdh/hbase-0.98.6-cdh5.3.6/
$ export HIVE_HOME=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/
$ ln -s $HBASE_HOME/lib/hbase-common-0.98.6-cdh5.3.6.jar  $HIVE_HOME/lib/hbase-common-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/hbase-server-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-server-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/hbase-client-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-client-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/hbase-protocol-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-protocol-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/hbase-it-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-it-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/htrace-core-2.04.jar $HIVE_HOME/lib/htrace-core-2.04.jar
$ ln -s $HBASE_HOME/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar $HIVE_HOME/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar
$ ln -s $HBASE_HOME/lib/high-scale-lib-1.1.1.jar $HIVE_HOME/lib/high-scale-lib-1.1.1.jar

While Hive -site.xml modify properties of zookeeper, as follows:

 

2, Case 1: CV table Hive, HBase association table, the data inserted into the table while Hive can affect HBase

The STEP 1 , in the creation Hive in the table at the same time associated with HBase

CREATE TABLE hive_hbase_emp_table(

empno int,

ename string,

job string,

mgr int,

hiredate string,

sal double,

comm double,

deptno int)

STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,info:ename,info:job,info:mgr,info:hiredate,info:sal,info:comm,info:deptno")

TBLPROPERTIES ("hbase.table.name" = "hbase_emp_table");

(Screaming Tip: After completion, and enter HBase Hive view respectively, generates the corresponding table)

STEP 2 , to create a temporary table in the intermediate Hive, the data for (Note: not load the data directly into the HBase Hive associated in that table) load file .

CREATE TABLE emp(

empno int,

ename string,

job string,

mgr int,

hiredate string,

sal double,

comm double,

deptno int)

row format delimited fields terminated by '\t';

Step3、向Hive中间表中load数据

hive> load data local inpath '/home/admin/Desktop/emp.txt' into table emp;

Step4、通过insert命令将中间表中的数据导入到Hive关联HBase的那张表中

hive> insert into table hive_hbase_emp_table select * from emp;

Step5、测试,查看Hive以及关联的HBase表中是否已经成功的同步插入了数据

如图所示:

Hive中:

 

HBase中:

 

3、案例2:比如在HBase中已经存储了某一张表hbase_emp_table,然后在Hive中创建一个外部表来关联HBase中的hbase_emp_table这张表,使之可以借助Hive来分析HBase这张表中的数据。

该案例2紧跟案例1的脚步,所以完成此案例前,请先完成案例1。

Step1、在Hive中创建外部表

CREATE EXTERNAL TABLE relevance_hbase_emp(

empno int,

ename string,

job string,

mgr int,

hiredate string,

sal double,

comm double,

deptno int)

STORED BY

'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

WITH SERDEPROPERTIES ("hbase.columns.mapping" =

":key,info:ename,info:job,info:mgr,info:hiredate,info:sal,info:comm,info:deptno") 

TBLPROPERTIES ("hbase.table.name" = "hbase_emp_table");

Step2、关联后就可以使用Hive函数进行一些分析操作了

在此,我们查询一下所有数据试试看

hive (default)> select * from relevance_hbase_emp;

  

Guess you like

Origin www.cnblogs.com/alexzhang92/p/10940428.html