Hive-1.2.1与HBase-1.1.2的整合
这里的整合是指,将HBase作为存储数据的库,由Hive作为连接桥梁
在Hive中创建表时,出现了
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
这个问题是版本适配问题,关键在 Hive 下 hive-hbase-handler-1.2.1.jar 这个jar需要将HBase的以下jar包放到Hive的lib下重新进行编译,
如果以下报错:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/Connection
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:225)
at org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:64)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10767)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.client.Connection
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 27 more
粗暴的解决办法:
将如下 Hbase 安装目录lib下面的jar包全部导入到hive安装目录的lib下
hbase-common-1.1.2.jar
hbase-server-1.1.2.jar
hbase-client-1.1.2.jar
hbase-protocol-1.1.2.jar
htrace-core-3.1.0-incubating.jar
netty-all-4.0.23.Final.jar
hbase-hadoop2-compat-1.1.2.jar
hbase-hadoop-compat-1.1.2.jar
metrics-core-2.2.0.jar