Article Directory
1. Codec overview
1. Codec Overview
When Netty sends or receives a message, a data conversion occurs. Inbound messages are decoded (converted from bytes to another format, such as java objects); outbound messages are encoded into bytes.
Netty provides a series of practical codecs, all of which implement the ChannelInboundHadnler or ChannelOutboundHandler interface. In these classes, the channelRead method has been overridden. Taking inbound as an example, this method will be called for every message read from the inbound Channel. Then, it will call the decode() method provided by the decoder to decode and forward the decoded bytes to the next ChannelInboundHandler in the ChannelPipeline.
2. Encoder class relationship diagram
Encoders all inherit the MessageToByteEncoder abstract class and need to rewrite its encode method.
We found that the encoder inherits ChannelOutboundHandlerAdapter, indicating that the outbound operation will only be performed.
3. Decoder class relationship diagram
Decoders all inherit the ByteToMessageDecoder abstract class and need to rewrite its decode method.
We found that the decoder inherits ChannelInboundHandlerAdapter, which means that the inbound operation will be executed.
2. Take the codec as an example to understand inbound and outbound
1. Server side
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
public class MyServer {
public static void main(String[] args) throws Exception{
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();//一会下断点
//入站的handler进行解码 MyByteToLongDecoder
pipeline.addLast(new MyByteToLongDecoder());
//出站的handler进行编码
pipeline.addLast(new MyLongToByteEncoder());
//自定义的handler 处理业务逻辑
pipeline.addLast(new MyServerHandler());
System.out.println("end");
}
}); //自定义初始化pipeline
ChannelFuture channelFuture = serverBootstrap.bind(7000).sync();
channelFuture.channel().closeFuture().sync();
}finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
public class MyServerHandler extends SimpleChannelInboundHandler<Long> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, Long msg) throws Exception {
System.out.println("从客户端" + ctx.channel().remoteAddress() + " 读取到long " + msg);
//给客户端发送一个long
ctx.writeAndFlush(98765L);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
cause.printStackTrace();
ctx.close();
}
}
2. Client side
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
public class MyClient {
public static void main(String[] args) throws Exception{
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap();
bootstrap.group(group).channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
//加入一个出站的handler 对数据进行一个编码
pipeline.addLast(new MyLongToByteEncoder());
//这时一个入站的解码器(入站handler )
pipeline.addLast(new MyByteToLongDecoder());
//加入一个自定义的handler ， 处理业务
pipeline.addLast(new MyClientHandler());
}
}); //自定义一个初始化类
ChannelFuture channelFuture = bootstrap.connect("localhost", 7000).sync();
channelFuture.channel().closeFuture().sync();
}finally {
group.shutdownGracefully();
}
}
}
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
public class MyClientHandler extends SimpleChannelInboundHandler<Long> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, Long msg) throws Exception {
System.out.println("服务器的ip=" + ctx.channel().remoteAddress());
System.out.println("收到服务器消息=" + msg);
}
//重写channelActive 发送数据
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
System.out.println("MyClientHandler 发送数据");
ctx.writeAndFlush(123456L); //发送的是一个long
}
}
3. Codec
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.MessageToByteEncoder;
public class MyLongToByteEncoder extends MessageToByteEncoder<Long> {
//编码方法
@Override
protected void encode(ChannelHandlerContext ctx, Long msg, ByteBuf out) throws Exception {
System.out.println("MyLongToByteEncoder encode 被调用");
System.out.println("msg=" + msg);
out.writeLong(msg);
}
}
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageDecoder;
import java.util.List;
public class MyByteToLongDecoder extends ByteToMessageDecoder {
/**
*
* decode方法 会根据接收的数据，被调用多次, 直到确定没有新的元素被添加到list
* , 或者是ByteBuf 没有更多的可读字节为止
* 如果list out 不为空，就会将list的内容传递给下一个 channelinboundhandler处理, 该处理器的方法也会被调用多次
*
* @param ctx 上下文对象
* @param in 入站的 ByteBuf
* @param out List 集合，将解码后的数据传给下一个handler
* @throws Exception
*/
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
System.out.println("MyByteToLongDecoder 被调用");
//因为 long 8个字节, 需要判断有8个字节，才能读取一个long
if(in.readableBytes() >= 8) {
out.add(in.readLong());
}
}
}
3. Execute and view the results
client side:
MyClientHandler sends data
MyLongToByteEncoder encode is called
msg=123456
MyByteToLongDecoder is called
server ip=localhost/127.0.0.1:7000
received server message=98765
server side:
MyByteToLongDecoder was called to
read from client/127.0.0.1:58478 to long 123456
MyLongToByteEncoder encode was called with
msg=98765
According to our execution results, we can simply draw a conclusion that the execution process is roughly like this:
4. Matters needing attention
The codec can only handle the data of the specified type. If the data type does not match, the codec will be skipped, but the data will not be lost.
For example, if the client ctx.writeAndFlush(Unpooled.copiedBuffer(“abcdabcdabcdabcd”, CharsetUtil.UTF_8)); sends a 16-bit string, although the encode method of the Encoder will be executed, it will not be encoded. Therefore, when we write Encoder, we should pay attention to the consistency of the incoming data type and the processed data type.
Let's take a look at the write method of MessageToByteEncoder:
// io.netty.handler.codec.MessageToByteEncoder#write
@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
ByteBuf buf = null;
try {
if (acceptOutboundMessage(msg)) {
//判断当前msg 是不是应该处理的类型，如果是就处理，不是就跳过encode
@SuppressWarnings("unchecked")
I cast = (I) msg;
buf = allocateBuffer(ctx, cast, preferDirect);
try {
encode(ctx, cast, buf);
} finally {
ReferenceCountUtil.release(cast);
}
if (buf.isReadable()) {
ctx.write(buf, promise);
} else {
buf.release();
ctx.write(Unpooled.EMPTY_BUFFER, promise);
}
buf = null;
} else {
ctx.write(msg, promise);
}
} catch (EncoderException e) {
throw e;
} catch (Throwable e) {
throw new EncoderException(e);
} finally {
if (buf != null) {
buf.release();
}
}
}
3. Other built-in codecs of Netty
Netty includes many codecs, which are usually enough for our daily work:
1、ReplayingDecoder
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ReplayingDecoder;
import java.util.List;
public class MyByteToLongDecoder extends ReplayingDecoder<Void> {
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
System.out.println("MyByteToLongDecoder2 被调用");
//在 ReplayingDecoder 不需要判断数据是否足够读取，内部会进行处理判断
out.add(in.readLong());
}
}
ReplayingDecoder is easy to use, but it also has some limitations:
1. Not all ByteBuf operations are supported. If an unsupported method is called, an UnsupportedOperationException will be thrown.
2. ReplayingDecoder may be slightly slower than ByteToMessageDecoder in some cases. For example, when the network is slow and the message format is complex, the message will be split into multiple fragments and the speed will slow down.
2. Other encoders
LineBasedFrameDecoder: This class is also used inside Netty, which uses end-of-line control characters (\n or \r\n) as delimiters to parse data.
DelimiterBasedFrameDecoder: Use custom special characters as message delimiters.
HttpObiectDecoder: A decoder for HTTP data.
LengthFieldBasedFrameDecoder: Identify the full packet message by specifying the length, so that sticky packets and half-packet messages can be automatically processed.
Encoders and decoders come in pairs.
3. The built-in codec handles the problem of sticking and unpacking
Four solutions for Netty to solve sticky and unpacking problems