Java在Windows下使用Hbase操作Hadoop示例

（1）解决jar包依赖问题，可以使用两种方式

1、直接创建maven工程，编写pom文件，添加依赖的jar包声明

2、直接下载对应的hbase的linux压缩包，解压后拷贝文件夹lib下的所有jar包

注意：

    在运行过程中会报错：java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing，因为缺少了hadoop的jar包），

可以从解压hadoop的安装包中的子目录获得（此处我获取路径）：hadoop-2.9.1\share\hadoop\common\lib\hamcrest-core-1.3.jar

（2）运行报错问题

1、没有设置超时情况下，Java会报错（60s后超时-太长）：

java.io.IOException: Failed to get result within timeout, timeout=60000ms
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:206)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:162)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)

at com.donwait.hadoop.HBaseUtil.createTable(HBaseUtil.java:112)

仅仅报超时，都没有报具体错误！为了解决超时问题，设置如下超时：

configuration.set("zookeeper.znode.parent", "/hbase");
configuration.set("hbase.client.retries.number", "3"); 
configuration.set("hbase.rpc.timeout", "2000"); 
configuration.set("hbase.client.operation.timeout", "3000"); 

configuration.set("hbase.client.scanner.timeout.period", "10000"); 

此时报错：

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=3, exceptions:
Thu Jun 21 09:48:03 CST 2018, RpcRetryingCaller{globalStartTime=1529545680623, pause=100, retries=3}, java.net.UnknownHostException: hadoop2
Thu Jun 21 09:48:03 CST 2018, RpcRetryingCaller{globalStartTime=1529545680623, pause=100, retries=3}, java.net.UnknownHostException: hadoop2
Thu Jun 21 09:48:03 CST 2018, RpcRetryingCaller{globalStartTime=1529545680623, pause=100, retries=3}, java.net.UnknownHostException: hadoop2


at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:157)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: hadoop2
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.<init>(AbstractRpcClient.java:315)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.createBlockingRpcChannel(AbstractRpcClient.java:267)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getClient(ConnectionManager.java:1639)
at org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:163)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.prepare(ScannerCallableWithReplicas.java:376)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
... 4 more

2、原因是因为hbase配置的Zookeeper列表为主机名，Hbase客户端与Hbase的主节点通信后拿到的就是对应配置

这里hbase-site.xml配置如下：

<configuration>
	<property>
		<!-- 端口要和Hadoop的fs.defaultFS端口一致-->
		<!-- hbase存放数据目录 -->
		<name>hbase.rootdir</name> 
		<value>hdfs://hadoop1:9000/hbase</value>
	</property> 
	<property>
		<!-- 是否分布式部署 -->
		<name>hbase.cluster.distributed</name> 
		<value>true</value> 
	</property> 
	<property> 
		<!-- list of  zookooper -->
		<name>hbase.zookeeper.quorum</name> 
		<value>hadoop1,hadoop2,hadoop3</value> 
	</property> 　　　 
	<property>
		<!--zookooper配置、日志等临时文件的存储位置 -->
		<name>hbase.zookeeper.property.dataDir</name> 
		<value>/home/hadoop/hbase/zookeeper</value>
	</property>
	<property>
		<!--不设置将会使用一个默认端口，可能不是zookeeper使用的2181这些端口-->
		<name>hbase.zookeeper.property.clientPort</name>
		<value>2181</value>
	</property>
</configuration>

可以看到Zookeeper列表为hadoop1、hadoop2和hadoop3（因为我配置了3台集群），客户端获取到列表后无法解析对应主机，所以windows上运行对应代码必须让本机器能识别对应主机名，故修改本机的hosts文件，添加上对应主机和ip的映射即可（路径位置：C:\Windows\System32\drivers\etc\hosts）：

具体配置：

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#	127.0.0.1       localhost
#	::1             localhost
192.168.12.104	hadoop1
192.168.12.105	hadoop2
192.168.12.106	hadoop3

再次运行实例：

public class Test {
	@org.junit.Test
	public void testHBase(){
		HBaseUtil hbase = new HBaseUtil();
		// 通话记录，联系人
		hbase.createTable("CallLog", "Record,Contact");
	}
}

具体测试代码：

package com.donwait.hadoop;

import java.io.IOException;

public class Test {
	/**
	 * 做多表存储使用【相对于关系型数据库中的表通话记录表、联系人表】
	 * @throws IOException
	 */
	@org.junit.Test
	public void testCreateCallLog() throws IOException{
		HBaseUtil hbase = new HBaseUtil();
		// 通话记录，联系人
		hbase.createTable("CallLog", "Record,Contact");
		
		// 通话记录 from 指点给 to， from的具体联系人详情Contact(类似于RDBMS中的Record表和Contact表）
		hbase.putDataH("CallLog", "row1", "Record", "from", "13556892563");
		hbase.putDataH("CallLog", "row1", "Record", "to", "13256894751");
		hbase.putDataH("CallLog", "row1", "Contact", "name", "lixx");
		hbase.putDataH("CallLog", "row1", "Contact", "addr", "test address");
	}
	
	@org.junit.Test
	public void testDropCallLog() throws IOException{
		HBaseUtil hbase = new HBaseUtil();
		// 通话记录，联系人
		hbase.dropTable("CallLog");
	}
	
	/**
	 * 做多表存储使用
	 * @throws IOException
	 */
	@org.junit.Test
	public void testGetFromCallLog() throws IllegalArgumentException, IOException{
		HBaseUtil hbase = new HBaseUtil();	
		// Record表
		System.out.println(hbase.getValueBySeriesH("CallLog", "row1", "Record", "from"));
		System.out.println(hbase.getValueBySeriesH("CallLog", "row1", "Record", "to"));
		
		// Contact表
		System.out.println(hbase.getValueBySeriesH("CallLog", "row1", "Contact", "name"));
		System.out.println(hbase.getValueBySeriesH("CallLog", "row1", "Contact", "addr"));
	}
	
	/**
	 * 做单表存储使用
	 * @throws IOException
	 */
	@org.junit.Test
	public void testCreatePerson() throws IOException{
		HBaseUtil hbase = new HBaseUtil();
		// 通话记录，联系人
		hbase.createTable("Person", "name,sex,age");
		
		hbase.putDataH("Person", "id11", "name", null, "lixx");
		hbase.putDataH("Person", "id11", "sex", null, "20");
		hbase.putDataH("Person", "id11", "age", null, "1");
	}
	
	/**
	 * 做单表存储使用
	 * @throws IOException
	 */
	@org.junit.Test
	public void testGetFromPerson() throws IOException{
		HBaseUtil hbase = new HBaseUtil();
		// 通话记录，联系人
		hbase.createTable("Person", "name,sex,age");
		
		System.out.println(hbase.getValueBySeriesH("Person", "id11", "name", null));
		System.out.println(hbase.getValueBySeriesH("Person", "id11", "sex", null));
		System.out.println(hbase.getValueBySeriesH("Person", "id11", "age", null));
	}
}

代码实例下载链接：https://download.csdn.net/download/lixiang987654321/10489950