Implement a simple RPC framework with Netty

Netty is an event-driven, asynchronous NIO network application framework that can be used to develop high-performance, high-reliability network IO programs. The underlying IO communication of many RPC frameworks on the market is implemented with Netty. The main purpose of this project is to strengthen the mastery of Netty applications through practice. In addition, this project can also be used to deepen the understanding of RPC.

what is RPC

RPC is short for Remote Procedure Call. It is a solution for calling between applications. By establishing a TCP connection between the client and the server, sending requests to serialize and deserialize exchange data, making it as easy for callers to use local APIs. People often wonder why HTTP is not used. In fact, HTTP is also a way to implement RPC.

Implementation process

Protocols
Before talking about server and client implementations, define the relevant RPC protocols.
SwiftMessage , the message POJO flowing in the pipeline:

public class SwiftMessage {

    private int length;
    private byte[] content;

    public SwiftMessage(int length, byte[] content) {
        this.length = length;
        this.content = content;
    }
    
    // 省略setter、getter
}
复制代码

RpcRequest , RpcResponse , the request response object processed in the handler:

public class RpcRequest implements Serializable {

    private static final long serialVersionUID = 2104861261275175620L;

    private String id;
    private String className;
    private String methodName;
    private Object[] parameters;
    private Class<?>[] parameterTypes;
}
复制代码

public class RpcResponse implements Serializable {

    private static final long serialVersionUID = -1921327887856337850L;

    private String requestId;
    private int code;
    private String errorMsg;
    private Object data;
}
复制代码

Encoder and decoder for ByteBuf to SwiftMessage conversion:

public class SwiftMessageEncoder extends MessageToByteEncoder<SwiftMessage> {

    @Override
    protected void encode(ChannelHandlerContext ctx, SwiftMessage msg, ByteBuf out) throws Exception {
        out.writeInt(msg.getLength());
        out.writeBytes(msg.getContent());
    }
}
复制代码

public class SwiftMessageDecoder extends ReplayingDecoder<Void> {

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        int length = in.readInt();
        byte[] content = new byte[length];
        in.readBytes(content);
        out.add(new SwiftMessage(length, content));
    }
}
复制代码

Server
In fact, RPC Server is simpler, just need to do the following things:

1. Start the service and register yourself with the RPC service center (so that the client can find and establish a connection)

@Component
public class SwiftServerRunner implements ApplicationRunner, ApplicationContextAware {

    private void start(int port, String serverName) {
        if (isRunning.get()) {
            LOGGER.info("SwiftRPC服务已经在运行中");
            return;
        }

        final NioEventLoopGroup bossGroup = new NioEventLoopGroup(1);
        final NioEventLoopGroup workerGroup = new NioEventLoopGroup(16);
        try {
            SwiftServerHandler handler = new SwiftServerHandler(rpcServiceMap);
            ServerBootstrap bootstrap = new ServerBootstrap();
            bootstrap.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG, 1024)
                    .childOption(ChannelOption.SO_KEEPALIVE, true)
                    .childOption(ChannelOption.TCP_NODELAY, true)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel channel) throws Exception {
                            ChannelPipeline pipeline = channel.pipeline();
                            pipeline.addLast(new SwiftMessageDecoder());
                            pipeline.addLast(new SwiftMessageEncoder());
                            pipeline.addLast(handler);
                        }
                    });
            ChannelFuture channelFuture = bootstrap.bind(port).sync();
            isRunning.set(true);

            String rpcServer = "127.0.0.1:" + port;
            registerRpcServer(serverName, rpcServer);

            LOGGER.info("SwiftRPC服务已经启动，端口：{}", port);

            channelFuture.channel().closeFuture().sync();
        } catch (Exception e) {
            isRunning.set(false);
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();

            throw new RuntimeException("启动SwiftRPC服务失败", e);
        }
    }

    private void registerRpcServer(String serverName, String rpcServer) throws Exception {
        // 简单的将rpc server列表保存到redis中
        redisTemplate.opsForHash().put("rpc-server", serverName, rpcServer);
    }
}
复制代码

2. Monitor client connections, receive request data, and deserialize into RpcRequest

@Override
protected void channelRead0(ChannelHandlerContext ctx, SwiftMessage msg) throws Exception {
    byte[] content = msg.getContent();
    RpcRequest request = Kryos.deserialize(content, RpcRequest.class);
}
复制代码

3. Use the parameters in RpcRequest to execute specific business implementation methods through reflection, and then write the results back to the connection with the client

@ChannelHandler.Sharable
public class SwiftServerHandler extends SimpleChannelInboundHandler<SwiftMessage> {
    
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, SwiftMessage msg) throws Exception {
        byte[] content = msg.getContent();
        RpcRequest request = Kryos.deserialize(content, RpcRequest.class);

        RpcResponse response = new RpcResponse();
        response.setRequestId(request.getId());

        try {
            Object result = handler(request);
            response.setData(result);
        } catch (Exception e) {
            response.setCode(-1);
            response.setErrorMsg(e.toString());
        }

        byte[] resContent = Kryos.serialize(response);
        SwiftMessage message = new SwiftMessage(resContent.length, resContent);
        ctx.writeAndFlush(message);
    }

    private Object handler(RpcRequest request) throws Exception {
        String className = request.getClassName();
        Object rpcService = rpcServiceMap.get(className);
        if (rpcService == null) {
            throw new RuntimeException("未找到RPC服务类：" + className);
        }

        // 通过反射调用业务层
        String methodName = request.getMethodName();
        Object[] parameters = request.getParameters();
        Class<?>[] parameterTypes = request.getParameterTypes();
        Class<?> clazz = rpcService.getClass();
        Method method = clazz.getMethod(methodName, parameterTypes);

        return method.invoke(rpcService, parameters);
    }
}
复制代码

Client
The client is a little more complicated, because the client needs to receive external call requests (such as a request from a rest interface), select different RPC servers to send requests according to different requests, and finally need to wait for the returned results.

1. 注册RPC接口。因为要想通过注入的方式调用service的方法，就需要将service注册成Spring的bean

@Autowired
private UserService userService;
复制代码

但是UserService具体的实现类在服务端，所以需要自定义一个ComponentScan，扫描所有的RPC接口，注册成Spring的bean，并设置对应的FactoryBean来完成实例化工作，可以参考Mybatis的执行过程。在FactoryBean中用动态代理来生成每个接口的代理类执行各个方法。所以最终的RPC调用是在这个代理类的invoke方法中发起的。

public class SwiftRpcServiceRegister implements ImportBeanDefinitionRegistrar {

    private static final Logger LOGGER = LoggerFactory.getLogger(SwiftRpcServiceRegister.class);

    @Override
    public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
        Map<String, Object> annotationAttributes = importingClassMetadata.getAnnotationAttributes(SwiftRpcServiceScan.class.getName());
        if (annotationAttributes != null) {
            String[] packages = (String[]) annotationAttributes.get("packages");
            SwiftRpcClassPathBeanDefinitionScanner scanner = new SwiftRpcClassPathBeanDefinitionScanner(registry);
            scanner.addIncludeFilter(new AnnotationTypeFilter(SwiftRpcService.class));

            if (packages == null || packages.length == 0) {
                LOGGER.warn("扫描RPC目录为空");
                return;
            }

            Set<BeanDefinitionHolder> definitionHolders = scanner.doScan(packages);
            for (BeanDefinitionHolder holder : definitionHolders) {
                GenericBeanDefinition beanDefinition = (GenericBeanDefinition) holder.getBeanDefinition();
                String className = beanDefinition.getBeanClassName();
                if (className == null) {
                    continue;
                }

                beanDefinition.getConstructorArgumentValues().addGenericArgumentValue(className);
                beanDefinition.setBeanClass(SwiftRpcFactoryBean.class);
                beanDefinition.getPropertyValues().add("rpcInterface", className);
                beanDefinition.setAutowireMode(GenericBeanDefinition.AUTOWIRE_BY_TYPE);
            }
        }
    }

    static class SwiftRpcClassPathBeanDefinitionScanner extends ClassPathBeanDefinitionScanner {

        SwiftRpcClassPathBeanDefinitionScanner(BeanDefinitionRegistry registry) {
            super(registry, false);
        }

        @Override
        protected boolean isCandidateComponent(AnnotatedBeanDefinition beanDefinition) {
            AnnotationMetadata metadata = beanDefinition.getMetadata();
            return metadata.isInterface() && metadata.isIndependent();
        }

        @Override
        protected Set<BeanDefinitionHolder> doScan(String... basePackages) {
            return super.doScan(basePackages);
        }
    }
}
复制代码

通过rpcInterface的代理来执行请求的调用。

public class SwiftRpcFactory<T> implements InvocationHandler {

    private Class<T> rpcInterface;
    
    @Override
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
        String className = rpcInterface.getName();
        String methodName = method.getName();

        SwiftRpcService rpcService = rpcInterface.getAnnotation(SwiftRpcService.class);
        String rpcServer = rpcService.server();
        RpcRequest request = new RpcRequest();
        request.setId(UUID.randomUUID().toString());
        request.setClassName(rpcInterface.getName());
        request.setMethodName(methodName);
        request.setParameters(args);
        request.setParameterTypes(method.getParameterTypes());

        SwiftClientRunner swiftClient = ApplicationContextHolder.getApplicationContext().getBean(SwiftClientRunner.class);
        return swiftClient.sendRequest(request, rpcServer);
    }
}
复制代码

2. 选择对应RPC Server的连接，发送请求，并等待结果返回

public Object sendRequest(RpcRequest request, String rpcServer) throws InterruptedException, IOException {
    Channel channel = getChannel(rpcServer);
    if (channel != null && channel.isActive()) {
        // 发送请求后阻塞等待返回结果
        SynchronousQueue<Object> queue = swiftClientHandler.sendRequest(request, channel);
        return queue.poll(60, TimeUnit.SECONDS);
    }

    return null;
}
复制代码

@Component
@ChannelHandler.Sharable
public class SwiftClientHandler extends SimpleChannelInboundHandler<SwiftMessage> {

    private final ConcurrentHashMap<String, SynchronousQueue<Object>> requests = new ConcurrentHashMap<>();

    @Override
    protected void channelRead0(ChannelHandlerContext channelHandlerContext, SwiftMessage swiftMessage) throws Exception {
        byte[] content = swiftMessage.getContent();
        RpcResponse response = Kryos.deserialize(content, RpcResponse.class);
        String requestId = response.getRequestId();
        SynchronousQueue<Object> queue = requests.get(requestId);
        queue.put(response.getData());
    }

    SynchronousQueue<Object> sendRequest(RpcRequest request, Channel channel) throws IOException {
        SynchronousQueue<Object> queue = new SynchronousQueue<>();
        requests.put(request.getId(), queue);

        byte[] content = Kryos.serialize(request);
        SwiftMessage message = new SwiftMessage(content.length, content);
        channel.writeAndFlush(message);

        return queue;
    }
}
复制代码

总结

这里只是实现了一个很简单的RPC调用，很多地方都是简单处理了，比如用redis的hash存储代替服务注册中心。但是可以借助这个项目对RPC的工作流程有一个全面的了解，在学习和使用其他RPC框架也很有帮助。下面是对服务端和客户端工作流程的总结： 服务端：启动Netty服务，注册服务信息到注册中心，接收请求反序列化成RpcRequest，通过反射执行本地业务代码，将结果写回到Socket。 客户端：扫描RpcInterface列表，注册到Spring容器中心，启动服务，读取注册中心的RPC Server地址，建立连接，通过代理发起远程调用，等待结果返回。

项目地址：github.com/Phantom0103…
参考&感谢：
github.com/mybatis/spr…
juejin.cn/post/684490…