EndPoint相当于存储过程。0.94.x之前使用EndPoint需要实现CoprocessorProtocol接口,而0.96.x的EndPoint改为用protobufs作为RPC的协议。在此用一个具体的例子说明一下新版的EndPoint该怎么使用。
例如:统计一张表的行数。
首先首先编写protobuf文件并编译。
option java_package = "linecounter"; option java_outer_classname = "LineCounterServer"; option java_generic_services = true; option java_generate_equals_and_hash = true; option optimize_for=SPEED; message CountRequest { required string askWord = 1; } message CountResponse { required int64 retWord = 1; } service LineCounter { rpc countLine(CountRequest) returns (CountResponse); }
编译后会生成LineCounterServer.java
CountRequest是发送给服务端的消息,这里定义字符串askWord来存放具体消息内容。CounterResponse是返回的结果,统计的是行数,所以用long类型存放。LineCounter中定义一个方法countLine,传递请求,返回响应。具体说明请参见protobuf。
实现EndPoint
public class LineCounterEndPoint extends LineCounterServer.LineCounter implements Coprocessor, CoprocessorService { private RegionCoprocessorEnvironment env; @Override public void start(CoprocessorEnvironment coprocessorEnvironment) throws IOException { if (coprocessorEnvironment instanceof RegionCoprocessorEnvironment) this.env = (RegionCoprocessorEnvironment) coprocessorEnvironment; else throw new CoprocessorException("Must be loaded on a table region!!"); } @Override public void stop(CoprocessorEnvironment coprocessorEnvironment) throws IOException { } @Override public Service getService() { return this; } @Override public void countLine(RpcController controller, LineCounterServer.CountRequest request, RpcCallback<LineCounterServer.CountResponse> done) { RegionScanner scanner = null; LineCounterServer.CountResponse.Builder respBuilder = LineCounterServer.CountResponse.newBuilder(); if (!"count".equals(request.getAskWord())) { respBuilder.setRetWord(23333); } else { long count = 0; try { Scan scan = new Scan(); scan.setMaxVersions(1); scanner = env.getRegion().getScanner(scan); List<Cell> list = new ArrayList<>(); while (scanner.next(list)) count += 1; respBuilder.setRetWord(count); } catch (IOException e) { e.printStackTrace(); } finally { if (scanner != null) try { scanner.close(); } catch (IOException e) { e.printStackTrace(); } } } done.run(respBuilder.build()); } }
LineCounterEndPoint需要继承抽象类LineCounter并实现Coprocessor和CoprocessorService接口。LineCounter在刚才生成的java文件里。
start和stop方法分别负责endpoint执行前的初始化和结束后的清理工作。start方法的参数是一个接口,需要根据实际环境将其转成需要的类型。
主要需要实现的是countLine方法,这也刚才在protobuf中定义的方法。为了测试效果,这里对请求做了一个区分:如果收到的请求信息不是“count”,那么返回23333;否则统计region的记录行数并返回。
实现Client端
public class LineCounterClient { public static void main(String[] args) throws Throwable { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "zk_host1:2181,zk_host2:2181,zk_host3:2181"); conf.set("hbase.master", "host_master:60000"); HTable table = new HTable(conf, "count_test"); final LineCounterServer.CountRequest req = LineCounterServer.CountRequest.newBuilder().setAskWord("count").build(); Map<byte[], Long> tmpRet = table.coprocessorService(LineCounterServer.LineCounter.class, null, null, new Batch.Call<LineCounterServer.LineCounter, Long>() { @Override public Long call(LineCounterServer.LineCounter instance) throws IOException { ServerRpcController controller = new ServerRpcController(); BlockingRpcCallback<LineCounterServer.CountResponse> rpc = new BlockingRpcCallback<>(); instance.countLine(controller, req, rpc); LineCounterServer.CountResponse resp = rpc.get(); return resp.getRetWord(); } }); long ret = 0; for (long l : tmpRet.values()) ret += l; System.out.println("lines: " + ret); } }
首先设置zookeeper和master的地址和接口信息。然后构造请求即CountRequest,先将请求信息设置为“count”。调用HTable的coprocessorService方法
public <T extends Service, R> Map<byte[],R> coprocessorService(final Class<T> service, byte[] startKey, byte[] endKey, final Batch.Call<T,R> callable)
该方法有四个参数,第1个参数是protobuf生成的LineCounter类对象。第2个和第3个参数分别为起始和结束rowkey,这里的意思是范围内rowkey所在的region都会调用endpoint,这里设为null表明所有的region都会调用。第4个参数为接口,需要重写call方法。
方法的返回值是Map类型,Map的size与参与计算的region个数一致。所以最后需要做的一步是讲返回结果进行累加,得到最后的结果。
此程序返回5782,是表count_test的行数。若请求消息设置为“hello”,程序返回23333。
coprocessorService还有一个五参数方法,第五个参数是一个CallBack接口,还可以如此实现:
public class LineCounterClient { public static void main(String[] args) throws Throwable { Configuration conf = HBaseConfiguration.create(); conf.set("hbase.zookeeper.quorum", "zk_host1:2181,zk_host2:2181,zk_host3:2181"); conf.set("hbase.master", "host_master:60000"); HTable table = new HTable(conf, "count_test"); final LineCounterServer.CountRequest req = LineCounterServer.CountRequest.newBuilder().setAskWord("count").build(); final AtomicLong ret = new AtomicLong(); table.coprocessorService(LineCounterServer.LineCounter.class, null, null, new Batch.Call<LineCounterServer.LineCounter, Long>() { @Override public Long call(LineCounterServer.LineCounter instance) throws IOException { ServerRpcController controller = new ServerRpcController(); BlockingRpcCallback<LineCounterServer.CountResponse> rpc = new BlockingRpcCallback<>(); instance.countLine(controller, req, rpc); LineCounterServer.CountResponse resp = rpc.get(); return resp.getRetWord(); } }, new Batch.Callback<Long>() { @Override public void update(byte[] region, byte[] row, Long result) { ret.getAndAdd(result); System.out.println(Bytes.toString(row)+": "+result); } }); System.out.println("lines: " + ret.get()); } }
每调用一次call方法之后会调用一次update方法,因此在外部定义一个变量ret存放结果,每次调用update时更新ret的值即可。