I taught you how to roll up a ZkClient with netty

Foreword

There is a reason this idea a while ago whim, want to try can not be directly connected to the use of js zookeeper, so as to acquire the registration information dubbo.

Later, after some find information, we found that as pure js does not support tcp socket communication, so pure js can not be achieved, but found some great god but use nodeJs achieve zk client. This success aroused my interest. Simple after a little research zk communication protocol, I began to try a hand line and zk client of course is implemented in java

Idea

zookeeper communication protocol is a typical "header / content" structure, in which header specifies the number of bytes of content, content is a specific message data.

Since it is a header / content structure, then it is easy to think of using LengthFieldPrepender netty to encode, decode and use LengthFieldBasedFrameDecoder With netty this great artifact, can do something more with less. It was decided to use netty development.

Client selection decisions were finished, we will need to have a server-side debugging. From the protocol point of view, there should not have too many compatibility problems between different versions of zk, but the difference is certainly there. So for your convenience, we here define a zookeeper version of the server is 3.4.12, higher version or lower version not done rigorous compatibility testing.

Note: To simplify the work, in addition to the version, the client only tested in stand-alone mode, and not verified in the cluster mode is run through [Behind his].

Ready to work

	stat path [watch]
	set path data [version]
	ls path [watch]
	delquota [-n|-b] path
	ls2 path [watch]
	setAcl path acl
	setquota -n|-b val path
	history 
	redo cmdno
	printwatches on|off
	delete path [version]
	sync path
	listquota path
	rmr path
	get path [watch]
	create [-s] [-e] path data acl
	addauth scheme auth
	quit 
	getAcl path
	close 
	connect host:port
复制代码

As shown in the above list, zkCli provides many commands for us, we will achieve our three command representative:

1. connect host:port

  这个命令其实是用来跟服务端建立会话的.
  为了避免跟socket建立tcp连接的connect方法相混淆, 我更愿意把它称作"login", 
  所以在实现的时候, 它对应的方法名也就是login
复制代码

2. create [-s] [-e] path data acl

   这个命令是用来创建zk的node节点的.
   其中包括持久性节点, 持久性顺序节点, 临时性节点, 临时性顺序节点等,
   还可以指定ACL权限
   
复制代码

3. ls path [watch]

   ls命令就是列举zk某个路径下的所有子路径,
   在具体实现里, 我把这个命令叫做getChildren
复制代码

Inside zookeep communication protocol, connect command (login) is a necessary precondition for all other commands. Since then, as a client, you must establish a session with the server, the following command requests to be accepted and processed server.

And in addition to the connect command, all other commands are in fact very different. So you will find, understand and create ls command after command, then the command is very simple to achieve other things, they just need to understand the communication protocol other things are models to copy.

Of course, to understand their communication protocol is not a simple matter, but also the structure of each command packet is not quite the same. In fact, code code, seventy to eighty percent of all energy consumed in the basic understanding of each the above command message structure.

Code

Prior to look at specific implementation, the first look at the structure of the project summary:

  1. bean包
     封装了每个命令需要的字段参数, 在序列化报文时只需要序列化对应的bean即可. 同样, 在服务端返回内容时, 也只需要把报文序列化成对应的对象即可.
  2. factories包
      上面提到过, zk的每个命令的报文结构都是不一样的,所以在序列化和反序列化时, 对应到netty的codec也是不一样的.这个实现了一个codec静态方法工厂, 需要的时候直接从codec工厂拿对应的codec即可.
  3. registrys包
     其实就是一个缓存中心, 缓存了每个命令对应的requestId和codec, 在服务端返回时, 从这个缓存中心根据requestId拿到对应codec来进行反序列化
  4. utils包
     一些工具类, 不需要多解释
  
  5. zkcodec包
     每个命令对应的codec和handler实现
  6. NettyZkClient类
     就是本文要介绍的zk客户端了
  7. test    
     为了方便调试准备的单元测试, 先了解代码实现原理的可用直接从这个单元测试入手
     
复制代码

After reading the code structure, we will look at the specific implementation of each command.

login command

First look at the structure of a communication packet for the login command, as shown below:

Briefly explain the meaning of each field, specific meaning we can do online search deeper understanding of:

 1. header
    上面提到, zk的每个报文都是header/content模式, 其中header占用4个字节, 表示接下来的content的长度(字节数)
 2. protocolVersion
    顾名思义, 这个字段表示协议的版本号,用4个字节表示. 这里我们写死为0即可(好浪费~~~)
 3. lastZxidSeen
    等下我们会看到, zk服务端每次的响应都会返回一个zxid.顾名思义, 这个结构就是表示客户端接收到最新的zxid.用8个字节表示. 由于login一般都是第一次跟服务端通讯, 所以这里也是写死为0即可
 4. timeout 
    login请求的目的是为了跟zk服务端建立会话, 这个参数表示的是会话的有效时间, 用四个字节表示
 5. senssionId
    会话ID, 用8个字节表示, 用于我们还没有建立会话,所以这个也是直接写死为0即可
 6. passwordLen
    用4个字节来表示接下来的密码的字节数
 7. password
    passwordLen个字节的密码, 用bytes[]表示
 8. readOnly
    boolean类型的,所以用一个字节表示即可
复制代码

After know the message structure, we can start writing code. First, the definition of a ZkLoginRequest bean. In java inside, 8-byte type can be represented by long, 4 bytes can represent int, String type can be very simple converted into a byte array so that the final bean defined as follows:

public class ZkLoginRequest implements Serializable {
  private Integer protocolVersion;
  private Long lastZxidSeen;
  private int timeout;
  private Long sessionId;
  private String password;
  private boolean readOnly = true;
    
}
复制代码

Zk because communication is byte-based, so we only define the java object is not enough, also need to be converted into bytes java object can send the server. And the service receive only header / content in the form of packets, so we have to calculate the number of bytes after the java entire target sequence, assigning it to go into the header.

Fortunately, these two working netty provide a good tool for us, we directly use it.

Java objects achieved ZkLoginCodec converts into ByteBuf

ZkLoginCodec encode and decode includes two portions, wherein in response to decode for decoding the server, sending a request for encoding encode is as follows, to convert the ZkLoginRequest into ByteBuf netty

 @Override
 protected void encode(ChannelHandlerContext ctx, ZkLoginRequest msg, ByteBuf outByteBuf) throws Exception {
     outByteBuf.writeInt(msg.getProtocolVersion());
     outByteBuf.writeLong(msg.getLastZxidSeen());
     outByteBuf.writeInt(msg.getTimeout());
     outByteBuf.writeLong(msg.getSessionId());
     String passWord = msg.getPassword();
     SerializeUtils.writeStringToBuffer(passWord, outByteBuf);
     outByteBuf.writeBoolean(msg.isReadOnly());
 }
复制代码

Netty directly using the built-in LengthFieldPrepender

netty built LengthFieldPrepender packets can be converted into a header / content in the form of a structure wherein the configuration parameter indicates the number of bytes occupied by the header, 4 bytes here, it is four.

   // 编码器, 给报文加上一个4个字节大小的HEADER
   nioSocketChannel.pipeline().addLast(new LengthFieldPrepender(4))
   
复制代码

ZkLoginRequest target after these two codec encoding, zk server will be able to correctly resolve its message, and if nothing unexpected happens, then the server will establish a session for us this socket, then give us a response message, message sessionId structure will contain the following responses:

You can see, the response message with our request packet is about the same, except sessionId, the other is basically returned intact to us here. So much to do here to explain the meaning of the response message. Direct view look at how to parse the response message returned from the server.

We can see from the chart, the return message is header / content form, so we can use the built-in decoder netty to get the header and content.

Skip header using LengthFieldBasedFrameDecoder

LengthFieldBasedFrameDecoder not familiar with the netty students can take a look at the official website, where the parameters of this class does not do too much explanation, just know that we want to skip the header, you can get only the content portion of the response message

nioSocketChannel.pipeline()
                        // 解码器, 将HEADER-CONTENT格式的报文解析成只包含CONTENT
        .addLast(ZkLoginHandler.LOGIN_LENGTH_FIELD_BASED_FRAME_DECODER,new LengthFieldBasedFrameDecoder(2048, 0, 4, 0, 4))
复制代码

ZkLoginCodec the content sequence into inverse ZkLoginResp

That is, the decode section ZkLoginCodec

   @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        ZkLoginResp zkLoginResp = new ZkLoginResp();
        zkLoginResp.setProtocolVersion(in.readInt());
        zkLoginResp.setTimeout(in.readInt());
        zkLoginResp.setSessionId(in.readLong());
        String password = SerializeUtils.readStringToBuffer(in);
        zkLoginResp.setPassword(password);
        zkLoginResp.setReadOnly(in.readBoolean());
        out.add(zkLoginResp);
    }
复制代码

After this step, our client has successfully established a session with the server, and the back can happily send other requests

create command

opCode和requestHeader

Before we start implementing the create command, first look at two terms, two terms not only create the need to use the command, wait for the next command to be achieved getChildren also need to use.

1. opCode

Zk of the server with client good agreement, each command corresponds to a opCode, the client sends a command opCode must bring this request, the server in order to know how to deal with this command. For example create command corresponding opCode is 1, to achieve the getChildren wait for the next command is opCode 8

2. requestHeader

Do not confuse this header with header / content in the header or part of the requestHeader content, which contains two fields, each of 4 bytes, as shown below:

   1. xid, 通俗点理解的就是requestId, 客户端维护这个xid的唯一性, 服务端返回响应时会原封不动的返回这个xid,
   这样客户端就可以知道服务端返回的报文时对应哪个请求的了.毕竟socket通讯都是异步的.
   
   2. type
      这个更好理解, 就是上面的opcode
复制代码

Create command message structure

Really, there is a little bit complicated messages create command, so before you see the message structure createRequest, have to first understand another concept: ACL permissions;

ACL permissions zookeeper involves the following three points:

1. scheme to authenticate the identity There are four ways:
2. Authorization object id
3. permission authority

Specific may see this blog , to put it more clearly.

In this article we wrote scheme is dead "world", id is "anyone", permission is 31 (that is, converted into binary 11111, with CREATE, READ, WRITE, DELETE, ADMIN five kinds of permissions)

Zk ACL packet structure is as follows:

  1. perms是permission的简写, 4个字节表示
  2. 因为scheme是用字符串表示的, 所以需要用4个字节表示scheme字符串的长度, 用schemelen个字节表示scheme
  3. id也是用字符串表示的, 跟scheme同理
复制代码

Understanding of the structure requestHeader and the ACL, createRequest message structure will be better understood, as shown below:

1. requestHeader
   包含了xid和type
2. pathLen
   要创建的path的字符串长度
3. path
   要创建的path, 例如你想在zk上创建一个/dubbo节点, path就是"/path"
4. dataLen
   要创建的path下的data的长度
5. data
   要创建的path下的数据, 例如"192.168.99.101:2181"
6. aclListSize
   zk的ACL权限是list形式的,表示不同的权限控制维度
7. aclList
   aclListSize个ACL结构体
8. flags
   该节点的类型, 可以是持久性节点, 持久性顺序节点, 临时节点, 临时顺序节点4种
复制代码

The next job is almost login:

1. ZkCreateCodec achieve a conversion into the ZkCreateRequest ByteBuf
2. The ByteBuf constructed using LengthFieldPrepender Header / content of the message structure
3. LengthFieldBasedFrameDecoder parsed using the content from the content server response
4. ByteBuf content to convert content into ZkCreateResponse

Wherein createRes packet structure as shown below:

1. xid
   这个就是createReq中requestHeader的xid
2. zxid
   这个可以跟login报文中的lastZxidSeen关联起来, 可以理解为服务端的xid
3. errcode
   errcode为0表示正常响应, 如果不为0,说明有异常出现, 后面的path字段也会为空
4. pathLen和path
   其实也是createReuest中的path和pathLen
复制代码

3. getChildren commands (ls path)

If the above command can create understanding, then getChildren command and very easy to understand, only two packet structure is different, so there will be a little different codec.

getChildrenRequest message structure:

Response structure:

Run the code

To quickly experience the simple zkClient, you can start directly from the unit test:

public class ZkClientTest {
    @Test
    public void testZkClient() throws Exception {
        // NettyZkClient的构造方法里面会调用login() 跟服务端建立会话
        NettyZkClient nettyZkClient = new NettyZkClient(30000);

        // 创建一个临时顺序节点
        ZkCreateResponse createResponse = nettyZkClient.create("/as", 12312, CreateMode.EPHEMERAL_SEQUENTIAL);
        System.out.println(new Gson().toJson(createResponse));

        // 获取/下的所有子路径
        List<String> list =  nettyZkClient.getChildren("/");
        System.out.println(new Gson().toJson(list));

    }
}
复制代码

Implement new command

Because the logic between different commands most of the same, so I have some generic logic abstracted out, if you want to achieve other commands, then just do a few simple tasks. For example, I want to implement a get path command, then you only need:

1. Find documentation to determine the structure of the message get command, this step is the most troublesome
2. Creating GetRequest classes, inheritance RequestHeader class, implement the interface ZkRequest
3. Creating GetResp classes, inheritance AbstractZkResonse category, achieving ZkResponse
4. Write GetRequestCodec, achieve ByteBuf and GetRequest, ZkResponse conversion
5. Modify ZkCodecFactories class, and associate GetRequest GetRequestCodec

This can be achieved with a new command. Of course, it had to have to mention, the first step is the most troublesome, may have to spend seventy to eighty percent of the workload.

Speaking of which, some may ask, where to understand the structure of each command packet it? In fact, there are many methods, the official document may have (but I do not find). My approach is the easiest, is to read ZkClient existing code, but existing zkClient not very visually reflected, have combined to debug the code, read the server-side code (parse packet), and so capture method.

Source

After the line and finished the zkClient client, we found that not enough fun, so later line and a redis client and kafka of producer / consumer.

After line and found that as long as the communication protocol requirements, may be implemented in any Netty substantially C / S architecture client. Thus these several clients to organize together, put github above.

Later want to continue to achieve elastic-search, mysql, dubbo and so the client (in fact, after a study is feasible, but there was no effort to achieve)

Finally, attach the github source address:

github.com/NorthWard/a…

Interested students can refer to, common learning and common progress.