Introduction to a learning mogoLink framework

https://github.com/penkee/mogoLink

mogoLink is an rpc framework that I started to design in 2016. At that time, I only came into contact with Netty technology and felt that it was very suitable for the underlying communication of the rpc framework. For the codec product, Google’s protoBuf was selected, but the problem of static compilation of its schema and each class could not be solved, and then it was delayed. At that time, the major companies had not yet servicing transformation, so there was insufficient experience in developing this framework and poor consideration.

This year, the company started servicing work. At this time, it came into contact with dubbo and spring cloud related books, and systematically introduced the theoretical basis of designing rpc. Therefore, this project has been re-maintained.

So what should be paid attention to when designing an rpc framework?

Socket communication framework: Generally, this framework is currently only open sourced by netty, which is quick to get started and has excellent performance.
Coding and decoding technology: java serialization, Google's protobuf, facebook thift, jboss marshaling, kryo, json, hession, aryo
Sticky packet processing: message fixed length, message tail plus separator, message header plus length field, and other complex application layer protocols
Service registry: used by distributed systems to manage available service addresses
Load balancing algorithm: used by the client, dispersing requests to different servers
Current limit fuse processing: used on the server side to prevent excessive requests from causing service interruption
Monitoring system: monitor the number of service calls, time-consuming, client connections and other information
Log tracking system: Since multiple layers of rpc will be called in cascade, it is necessary to generate a unique identifier to track the log collection system of the call chain

The above picture is the design flow chart of this system. Although it is rough, it is enough to be able to see clearly.

The FutureObject class is used to obtain future objects. Because netty receives messages asynchronously, this object can only be used as a communication tool. When the server receives the message and puts it into this object, the listener will immediately obtain the object, otherwise it will time out.

/**
 * @brief 获取未来对象
 * @details （必填）
 * @author 彭堃
 * @date 2016年8月26日下午5:56:29
 */
public class FutureObject<T> {
	private T value;

	public T get(long outTime) {
		if (value == null) {
			synchronized (this) {
				try {
					this.wait(outTime);
				} catch (InterruptedException e) {
					e.printStackTrace();
				}
			}
		}
		return value;
	}

	public void set(T value) {
		this.value = value;
		synchronized (this) {
			this.notify();
		}
	}
}

Client proxy method: In this way, all interfaces with @Autowird are automatically injected into the proxy object generated by the factory class. When the service method is called, the code executes the proxy request for the remote service

<bean id="userInfoRemoteService" class="com.eastorm.mogolink.client.proxy.ProxyFactory">
		<constructor-arg name="className" value="com.eastorm.mogolink.demo.client.service.api.IUserInfoService" />
	</bean>

/**
 * 创建动态代理对象
 * 动态代理不需要实现接口,但是需要指定接口类型
 * @author 慕容恪
 */
public class ProxyFactory  implements FactoryBean {
    private static final Logger logger = LoggerFactory.getLogger(ProxyFactory.class);

    private String className;

    public ProxyFactory(String className){
        this.className=className;
    }

    public Object getProxyInstance() throws ClassNotFoundException {
        Class target=Class.forName(className);

        return Proxy.newProxyInstance(target.getClassLoader(), new Class[]{target},
                new InvocationHandler() {
                    @Override
                    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {

                        //无需代理的父类方法
                        if("toString".equals(method.getName())){
                            return method.invoke(proxy,args);
                        }

                        BaseMessage req=new BaseMessage();
                        UUID uuid = UUID.randomUUID();
                        req.setRequestId(uuid.toString());

                        List<MethodParam> paramTypes=new ArrayList<>();
                        if(args!=null&&args.length>0){
                            for (Object arg : args) {
                                paramTypes.add(new MethodParam(arg.getClass().getName(),arg));
                            }
                        }

                        req.setParamTypes(paramTypes);
                        req.setServiceName(className);
                        req.setMethod(method.getName());

                        long s=System.currentTimeMillis();
                        ClientMsgHandler handler= ClientStarter.getHandler();
                        if(handler==null){
                            logger.info("请求失败req={}，耗时:{}ms",req.getRequestId(),System.currentTimeMillis()-s);
                            return null;
                        }
                        // Request and get the response.
                        BaseMessage resMsg = handler.getData(req);
                        if(resMsg!=null&&resMsg.getCode().equals(ServiceCodeEnum.SUCCCESS.getId())){
                            logger.info("req={}，耗时:{}ms",resMsg.getRequestId(),System.currentTimeMillis()-s);

                            return resMsg.getReturnData();
                        }else{
                            logger.info("失败req={}，耗时:{}ms",req.getRequestId(),System.currentTimeMillis()-s);
                        }
                        return null;
                    }
                });
    }

    @Override
    public Object getObject() throws Exception {
        return getProxyInstance();
    }

    @Override
    public Class<?> getObjectType() {
        Class target= null;
        try {
            target = Class.forName(className);
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
        return target;
    }

    @Override
    public boolean isSingleton() {
        return true;
    }
}

channel connection pool

If a client only connects to one channel, wouldn't it be a violent thing. A channel is blocked synchronously, so each client must have a channel connection pool. This system uses the common-pool tool of apache to maintain the channel connection.

Sticky bag problem

This framework is implemented by adding a message header to a message length field. The specific class is LengthFieldBasedFrameDecoder that comes with netty. We added a line header length field to the kryo encoder.

@Override
    protected void encode(ChannelHandlerContext channelHandlerContext, BaseMessage baseMessage, ByteBuf byteBuf) throws Exception {
        byte[] data= messageCodec.serialize(baseMessage);

        byteBuf.writeShort(data.length);
        byteBuf.writeBytes(data);
    }

If it can be determined that no separator is included after encoding, it can also be processed with separators, which is more economical.

The definition of the message body: the method supports overloading, so the class name of the method parameter is required

ublic class BaseMessage {
    private String requestId;
    private String code;
    private String msg;

    /**
     * 返回的信息
     */
    private Object returnData;
    /**
     * 服务名
     */
    private String serviceName;
    /**
     * 方法
     */
    private String method;
    /**
     * 方法的参数类型
     * 用来转型
     */
    private List<MethodParam> paramTypes;
}

Introduction to a learning mogoLink framework

Guess you like