笔记7：基于Netty的自定义RPC和Zookeeper实现简易版服务的注册与发现机制

首先，基于笔记5的代码进行改造
传送门：笔记5：Netty的自定义RPC(JSON序列化协议)
目标一：
1）启动2个服务端，可以将IP及端口信息自动注册到Zookeeper
2）客户端启动时，从Zookeeper中获取所有服务提供端节点信息，客户端与每一个服务端都建立连接
3）某个服务端下线后，Zookeeper注册列表会自动剔除下线的服务端节点，客户端与下线的服务端断开连接
4）服务端重新上线，客户端能感知到，并且与重新上线的服务端重新建立连接
目标二：
1）Zookeeper记录每个服务端的最后一次响应时间，有效时间为5秒，5s内如果该服务端没有新的请求，响应时间清零或失效
2）当客户端发起调用，每次都选择最后一次响应时间短的服务端进行服务调用，如果时间一致，随机选取一个服务端进行调用，从而实现负载均衡

改造客服端
因为客户端需要连接两个客服端，所以这里复制一个客服端，两个客服端代码基本一致，除了端口号。
客服端代码基本没变化，只是加入zookeeper的连接和监听操作(这里监听可省去)
1）启动类

@ComponentScan(value= "com.lossdate2")
	@SpringBootApplication
	public class ServerBoot1 {
      
      
	
	    public static void main(String[] args) throws Exception {
      
      
	        SpringApplication.run(ServerBoot1.class, args);
	
	        String ip = "127.0.0.1";
	        int port = 8998;
	
	        // 启动服务器
	        UserServiceImpl.startServer(ip, port);
	
	        //获取zk连接
	        ZkUtil1 zkUtil1 = new ZkUtil1();
	        //注册ip及端口信息
	        zkUtil1.createNode("/host1", ip + "#" + port);
	    }
	}

2）zookeeper连接类
建立临时节点，这样在服务端下线后，节点会自动剔除(有一定的延时)

public class ZkUtil1 {
      
      

    private static ZkClient ZK_CLIENT = null;

    /**
     * 根目录
     */
    private static final String PARENT_PATH = "/netty";

    /**
     * 本地储存
     */
    private static final Map<String, String> CHILDREN_NODE_MAP = new HashMap<>(16);

    public ZkUtil1() {
      
      
        connect();
    }

    public static ZkClient connect() {
      
      

        ZK_CLIENT = new ZkClient("127.0.0.1:2184");

        System.out.println("zk连接建立");

        //建立初始节点
        //判断是否存在
        boolean exists = ZK_CLIENT.exists(PARENT_PATH);
        if(!exists) {
      
      
            //节点不存在，创建临时节点
            ZK_CLIENT.createEphemeral(PARENT_PATH);
        }

        return ZK_CLIENT;
    }


    public void createNode(String path, String value) {
      
      

        CHILDREN_NODE_MAP.put(path, value);

        path = PARENT_PATH + path;

        //创建临时节点
        ZK_CLIENT.createEphemeral(path);
        System.out.println("zk创建节点" + path);
        ZK_CLIENT.writeData(path, value);
        System.out.println("节点" + path + " 写入值 " + value);

        //注册监听,监听子节点
        ZK_CLIENT.subscribeChildChanges(PARENT_PATH, new IZkChildListener() {
      
      
            @Override
            public void handleChildChange(String parentPath, List<String> list) throws Exception {
      
      
                //子节点变化，更新本地存储的节点列表，有少则删除zk上的节点
                Map<String, String> currentChildrenMap = new HashMap<>(16);
                if(list != null && list.size() > 0) {
      
      
                    list.forEach(node -> {
      
      
                        node = "/" + node;
                        //更新新增的节点
                        CHILDREN_NODE_MAP.putIfAbsent(node, node);
                        currentChildrenMap.put(node, node);
                    });
                }
                //剔除下线的节点 非持久节点断开后会自动删除
            }
        });

        //注册监听
        ZK_CLIENT.subscribeDataChanges(path, new IZkDataListener() {
      
      
            @Override
            public void handleDataChange(String s, Object o) throws Exception {
      
      
                System.out.println(s + "该节点内容被更新，更新的内容" + o);
            }

            @Override
            public void handleDataDeleted(String s) throws Exception {
      
      
                System.out.println(s + "该节点被删除");
            }
        });
    }
}

改造客户端
客户端的改造比较多
1）首先是启动类，这里是建立zookeeper连接和Netty连接的入口

public class ConsumerBoot {
      
      

    public static void main(String[] args) {
      
      

        //获取zk连接
        ZkUtilConsumer zkUtilConsumer = new ZkUtilConsumer();
        zkUtilConsumer.connect();
        //从zk上获取连接的ip和port
        Map<String, String> hostAndNodeMap = zkUtilConsumer.getChildrenHost(false);
        NettyConnection.createConnection(hostAndNodeMap);
    }
}

2）ZkUtilConsumer
用于初始化zookeeper连接、子节点的监听事件(主要时字节点的剔除和新增的监听，对应服务端的下线和重新上线)和5秒的轮询检查耗时，同时封装了更新节点数据和获取子节点的方法

public class ZkUtilConsumer {
      
      

    private static ZkClient ZK_CLIENT = null;

    /**
     * 根路径
     */
    private static final String PARENT_PATH = "/netty";

    /**
     * 节点的本地缓存
     */
    private static final Map<String, String> CHILDREN_NODE_MAP = new HashMap<>(16);

    /**
     * 定时任务线程池
     */
    private static ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(10);


    public ZkUtilConsumer() {
      
      
    }

    public ZkClient connect() {
      
      

        ZK_CLIENT = new ZkClient("127.0.0.1:2184");

        System.out.println("============= zk连接建立 =============");

        //建立初始节点
        //判断是否存在
        boolean exists = ZK_CLIENT.exists(PARENT_PATH);
        if(!exists) {
      
      
            //节点不存在，创建临时节点
            ZK_CLIENT.createEphemeral(PARENT_PATH);
        }

        //注册监听,监听子节点
        ZK_CLIENT.subscribeChildChanges(PARENT_PATH, new IZkChildListener() {
      
      
            @Override
            public void handleChildChange(String parentPath, List<String> list) throws Exception {
      
      
                //子节点变化，更新本地存储的节点列表，有少则删除zk上的节点
                Map<String, String> currentChildrenMap = new HashMap<>(16);
                if(list != null && list.size() > 0) {
      
      
                    list.forEach(node -> {
      
      
                        node = "/" + node;
                        //更新新增的节点
                        if(CHILDREN_NODE_MAP.get(node) == null) {
      
      
                            //不存在，新增
                            System.out.println("更新新增的节点" + node);
                            CHILDREN_NODE_MAP.put(node, node);
                            currentChildrenMap.put(node, node);
                            //建立连接
                            Object readData = ZK_CLIENT.readData(PARENT_PATH + node);

                            NettyConnection.addConnection(readData.toString());
                            System.out.println("============= 新增的节点"+node+"建立连接成功 =============");
                        }
                    });
                }
                //剔除下线的节点
                CHILDREN_NODE_MAP.forEach((key, value) -> {
      
      
                    if(currentChildrenMap.get(key) == null) {
      
      
                        //这个节点不存在了,删除
                        CHILDREN_NODE_MAP.put(key, null);
                    }
                });
            }
        });

        /*
        开启定时任务，5s执行获取一次zk的子节点的值，
        检查该节点的最后一次请求的时间与当前时间是否超过5s，
        超过则进行置空数据，没有则不处理
         */
        scheduledExecutorService.scheduleWithFixedDelay(new Runnable() {
      
      
            @Override
            public void run() {
      
      
                Map<String, String> hostAndNodeMap = getChildrenHost(true);
                long current = System.currentTimeMillis();
                System.out.println("定时5秒检测，当前时间：" + current);
                hostAndNodeMap.forEach((host, node) -> {
      
      
                    String[] arr = host.split("#");
                    //ip#port#time#dealTime
                    String ip = arr[0];
                    String port = arr[1];
                    if(arr.length > 2) {
      
      
                        long preTime = Long.parseLong(arr[2]);
                        if(current - preTime > 5000) {
      
      
                            ZkUtilConsumer.updateNodeVal(node, ip+"#"+port);
                            System.out.println("============= host:"+host+"超时" + (current - preTime) + "ms，时间置空 =============");
                        }
                    }
                });
            }
        }, 5, 5, TimeUnit.SECONDS);


        return ZK_CLIENT;
    }

    public Map<String, String> getChildrenHost(boolean needTime) {
      
      
        Map<String, String> hostAndNodeMap = new HashMap<>(6);
        List<String> children = ZK_CLIENT.getChildren(PARENT_PATH);
        if(children != null && children.size() > 0) {
      
      
            children.forEach(child -> {
      
      
                child = "/" + child;
                //读取节点内容
                Object readData = ZK_CLIENT.readData(PARENT_PATH + child);
                if(needTime) {
      
      
                    hostAndNodeMap.put(readData.toString(), PARENT_PATH + child);
                } else {
      
      
                    String host = readData.toString();
                    String[] arr = host.split("#");
                    //ip#port#time
                    String ip = arr[0];
                    String port = arr[1];
                    hostAndNodeMap.put(ip+"#"+port, PARENT_PATH + child);
                }

                //本地储存节点
                CHILDREN_NODE_MAP.putIfAbsent(child, child);
            });
        }

        return hostAndNodeMap;
    }

    static Stat updateNodeVal(String nodePath, String value) {
      
      
        if(!ZK_CLIENT.exists(nodePath)) {
      
      
            //节点不存在，创建临时节点
            ZK_CLIENT.createEphemeral(nodePath);
        }
        return ZK_CLIENT.writeData(nodePath, value);
    }

}

3）NettyConnection
处理主要逻辑，netty连接入口及发送连接数据，这里人为自造了5S空挡用于验证 “有效时间为5秒，5s内如果该服务端没有新的请求，响应时间清零或失效”。同时封装了删除链接和新增链接的方法。

public class NettyConnection {
      
      

    /**
     * 参数定义
     */
    private static final String PROVIDER_NAME = "UserService#sayHello#";

    /**
     * 连接
     */
    private static final Map<String, IUserService> USER_SERVICE_MAP = new HashMap<>(6);


    public static void createConnection(Map<String, String> hostAndNodeMap) {
      
      

        List<String> childrenHost = new ArrayList<>();
        hostAndNodeMap.forEach((host, node) -> childrenHost.add(host));
        //初始化连接
        RpcConsumer.clientBuild(childrenHost);

        if(childrenHost.size() > 0) {
      
      
            childrenHost.forEach(childHost -> {
      
      
                //1.创建代理对象
                IUserService userService = (IUserService) RpcConsumer.createProxy(IUserService.class, PROVIDER_NAME, childHost);
                USER_SERVICE_MAP.put(childHost, userService);
            });

            //2.循环给服务器写数据
            while (true) {
      
      
                if(USER_SERVICE_MAP.size() > 0) {
      
      
                    //任务一：连接2个客服端，向服务端发送消息
                    USER_SERVICE_MAP.forEach((host, userService) -> doNetty(host, userService, hostAndNodeMap));

                    //任务二，每次都选择最后一次响应时间短的服务端进行服务调用，如果时间一致，随机选取一个服务端进行调用，从而实现负载均衡
//                    doLoadBalance(userServiceMap, hostAndNodeMap);

                    //睡2s方便查看输出
                    try {
      
      
                        Thread.sleep(2000);
                    } catch (InterruptedException e) {
      
      
                        e.printStackTrace();
                    }
                }
            }
        }
    }

    /**
     * 任务二，每次都选择最后一次响应时间短的服务端进行服务调用，如果时间一致，随机选取一个服务端进行调用，从而实现负载均衡
     */
    private static void doLoadBalance(Map<String, IUserService> userServiceMap, Map<String, String> hostAndNodeMap) {
      
      
        Map<String, String> hostAndNodeMapWithTimeMap = new ZkUtilConsumer().getChildrenHost(true);
        //获取时间响应时间最短的,优先没有时间数据的
        if(hostAndNodeMapWithTimeMap.size() > 0) {
      
      
            String targetHost = null;
            long preDealTime = -1;
            for(Map.Entry<String, String> entry : hostAndNodeMapWithTimeMap.entrySet()) {
      
      
                String host = entry.getKey();

                String[] arr = host.split("#");
                //ip#port#time#dealTime
                String ip = arr[0];
                String port = arr[1];

                //延迟检测，服务端停止，zk上对应的node失效会有延迟性
                if(userServiceMap.get(ip+"#"+port) != null) {
      
      
                    //没有时间数据的，优先
                    if(arr.length > 2) {
      
      
                        long dealTime = Long.parseLong(arr[3]);
                        if(preDealTime == -1 || dealTime < preDealTime) {
      
      
                            targetHost = ip + "#" + port;
                            preDealTime = dealTime;
                        }
                    } else {
      
      
                        targetHost = ip + "#" + port;
                        preDealTime = 0;
                        break;
                    }
                }
            }

            System.out.println("选择响应时间最短的：host -> " + targetHost + "#" + preDealTime);

            doNetty(targetHost, userServiceMap.get(targetHost), hostAndNodeMap);
        }
    }

    /**
     * 任务一：连接2个客服端，向服务端发送消息
     */
    private static void doNetty(String host, IUserService userService, Map<String, String> hostAndNodeMap) {
      
      
        //断开的会被更新为null，所以这里要加个判断
        if(userService != null) {
      
      
            long start = System.currentTimeMillis();
            System.out.println("客户端开始");
            RpcResponse result = userService.sayHello("Hi I am Tom, I want to play a game with u !");
            System.out.println("客服端返回：" + result.toString());
            System.out.println("客户端结束");

            //人为自造5S空挡
            Random rd = new Random();
            int i = rd.nextInt(5);
            try {
      
      
                Thread.sleep(i*1000);
            } catch (InterruptedException e) {
      
      
                e.printStackTrace();
            }

            long end = System.currentTimeMillis();

            long dealTime = end - start;
            System.out.println(host+": 耗时：" + dealTime + "ms");

            //节点值绑定时间和耗时
            ZkUtilConsumer.updateNodeVal(hostAndNodeMap.get(host), host+"#"+end+"#"+dealTime);
            System.out.println("更新节点值时间：" + host+"#"+end+"#"+dealTime);
        }
    }

    /**
     * 链接断开时移除本地连接
     */
    public static synchronized void removeConnection(String host) {
      
      
        //用于连接
        USER_SERVICE_MAP.put(host, null);
    }

    /**
     * 新增连接
     */
    static synchronized void addConnection(String host) {
      
      
        String[] arr = host.split("#");
        //ip#port#time#dealTime
        String ip = arr[0];
        String port = arr[1];
        host = ip + "#" + port;

        //初始化连接
        List<String> childrenHost = new ArrayList<>();
        childrenHost.add(host);
        RpcConsumer.clientBuild(childrenHost);
        IUserService userService = (IUserService) RpcConsumer.createProxy(IUserService.class, PROVIDER_NAME, host);
        //用于连接
        USER_SERVICE_MAP.put(host, userService);
    }

}

4）修改RpcConsumer
对RpcConsumer进行修改，将Netty的初始化单独提出来封装成工具类，

public class RpcConsumer {
      
      

    /**
     * 1.创建一个线程池对象  -- 它要处理我们自定义事件
     */
    private static final ExecutorService EXECUTOR_SERVICE = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

    /**
     * 存储host和client的关联
     */
    private static final Map<String, Client> HOST_AND_CLIENT_MAP = new HashMap<>(6);

    public static void clientBuild(List<String> childrenHost) {
      
      

        if(childrenHost.size() > 0) {
      
      
            childrenHost.forEach(childHost -> {
      
      
                String[] arr = childHost.split("#");
                //ip#port#time
                String ip = arr[0];
                String port = arr[1];
                System.out.println("开始建立连接 ip:"+ip+" port:"+port);
                UserClientHandler userClientHandler = new UserClientHandler();
                Client client = new Client(ip, Integer.parseInt(port), userClientHandler);
                try {
      
      
                    client.initClient();
                } catch (InterruptedException e) {
      
      
                    e.printStackTrace();
                }
                HOST_AND_CLIENT_MAP.put(childHost, client);
            });
        }
    }

    /**
     * 4.编写一个方法,使用JDK的动态代理创建对象
     * serviceClass 接口类型,根据哪个接口生成子类代理对象;   providerParam :  "UserService#sayHello#"
     */
    public static Object createProxy(Class<?> serviceClass, final String providerParam, String childHost) {
      
      
        return Proxy.newProxyInstance(Thread.currentThread().getContextClassLoader(), new Class[]{
      
      serviceClass}, new InvocationHandler() {
      
      
            @Override
            public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
      
      
                //1)初始化客户端client
                Client client = HOST_AND_CLIENT_MAP.get(childHost);
                System.out.println("HOST:"+childHost + " " + client.getInfo());
                UserClientHandler userClientHandler = client.getUserClientHandler();

                //2)给UserClientHandler 设置param参数

                //修改为RpcRequest
                RpcRequest rpcRequest = new RpcRequest();
                rpcRequest.setRequestId(UUID.randomUUID().toString());
                String[] classNameAndMethod = providerParam.split("#");
                rpcRequest.setClassName(serviceClass.getName());
                rpcRequest.setMethodName(classNameAndMethod[1]);
                rpcRequest.setParameters(args);
                rpcRequest.setParameterTypes(new Class[]{
      
      String.class});
                userClientHandler.setParam(rpcRequest);

                //3).使用线程池,开启一个线程处理处理call() 写操作,并返回结果
                //4)return 结果
                return EXECUTOR_SERVICE.submit(userClientHandler).get();
            }
        });
    }
}

5）提取出来的Client初始化类
Client类会在RpcConsumer的一开始调用client.initClient()进行初始化连接，UserClientHandler通过new Client类时传入

public class Client {
      
      

    /**
     * 2.声明一个自定义事件处理器  UserClientHandler
     */
    private final UserClientHandler userClientHandler;


    private final String ip;

    private final int port;

    Client(String ip, int port, UserClientHandler userClientHandler) {
      
      
        this.ip = ip;
        this.port = port;
        this.userClientHandler = userClientHandler;
    }

    void initClient() throws InterruptedException {
      
      
        //1)创建连接池对象
        NioEventLoopGroup group = new NioEventLoopGroup();
        //2)创建客户端的引导对象
        Bootstrap bootstrap = new Bootstrap();
        //3)配置启动引导对象
        bootstrap.group(group)
                //设置通道为NIO
                .channel(NioSocketChannel.class)
                //设置请求协议为TCP
                .option(ChannelOption.TCP_NODELAY, true)
                //监听channel 并初始化
                .handler(new ChannelInitializer<SocketChannel>() {
      
      
                    @Override
                    protected void initChannel(SocketChannel socketChannel) throws Exception {
      
      
                        //获取ChannelPipeline
                        ChannelPipeline pipeline = socketChannel.pipeline();
                        //设置编码
                        pipeline.addLast(new RpcEncoder(RpcRequest.class, new JSONSerializer()));
                        pipeline.addLast(new RpcDecoder(RpcResponse.class, new JSONSerializer()));

                        //添加自定义事件处理器
                        pipeline.addLast(userClientHandler);
                    }
                });

        bootstrap.connect(ip, port).sync();
    }

    UserClientHandler getUserClientHandler() {
      
      
        return userClientHandler;
    }

    String getInfo() {
      
      
        return "UserClientHandler: ip-> "+this.ip + " port-> " + this.port;
    }

}

6）修改UserClientHandler类，加入对连接断开的监听，断开后要调用NettyConnection的removeConnection方法对断开的连接进行删除

	/**
     * 断开连接
     */
    @Override
    public void channelInactive(ChannelHandlerContext ctx) throws Exception {
      
      
        super.channelInactive(ctx);
        InetSocketAddress ipSocket = (InetSocketAddress) ctx.channel().remoteAddress();
        int port = ipSocket.getPort();
        String ip = ipSocket.getHostString();
        System.out.println("============= 与设备"+ip+":"+port+"连接断开! =============");
        final EventLoop eventLoop = ctx.channel().eventLoop();
        //移除本地存储的连接
        NettyConnection.removeConnection(ip+"#"+port);
    }

实现效果
目标一：先后启动客服客户端后，客户端会分别连接两个服务的并进行通信。当其中一个服务端下线后，客户端会断开于这个服务端的连接，当下线的服务端重新上线后，客户端会与之重新建立连接。
目标二：初始会连接两个服务端，同时将本次连接的时间和耗时存入节点。之后每次连接都会获取子节点列表获取耗时最小的进行连接。同时，客服端启动时同时启动了5秒的轮询获取子节点，当发现节点的最后一次连接的时间于当前时间相差大于5秒，则将节点的时间和耗时剔除。

笔记7：基于Netty的自定义RPC和Zookeeper实现简易版服务的注册与发现机制

猜你喜欢