premise
Front articles:
Github Page: "Netty and SpringBoot achieve a lightweight framework based on RPC - Agreement articles"Coding Page: "Netty and SpringBoot achieve a lightweight framework based on RPC - Agreement articles"
In the front of "and SpringBoot Netty achieve a lightweight framework based RPC - Protocol articles" herein has been defined in a relatively simple a RPCproprietary protocol, and realizes the corresponding encoding and decoding module. This article is based protocol papers, complete Serverwrite-side code calls. Given the current relatively mainstream IOCcontainers are Spring, where the choice of the spring-boot-starter(non- MVCcontainer, simply management Bean), dependent JDK1.8+.
Thinking
First of all RPCproprietary protocol defines the Clientend of the four characters will pass over and closely related to the service call: Interface full class name interfaceName, method name methodName, method parameter signature string array methodArgumentSignatures(optional, must be passed in this parameter is not) as well as a method parameter array methodArguments(available when selected, a list of empty method does not require passing parameters). The main process is as follows:
- The
Serverend of all server (achieve) class referred toIOCcontainer-managed. ClientInitiates aRPCrequest.- By up to four parameters mentioned above, from
Serverservice instanceIOCcontainers matching method of a highest degree of agreementjava.lang.reflect.Methodinstance, host-based and host-based instance of the process corresponding to theBeanexample, if the target method this step is more than one match or is 0, you can return exception information directly. - The previous step obtained
Methodinstance, host classBeaninstance, the method in conjunction with an array of parametersmethodArgumentsreflecting calls, call result obtained. ServerThe end result in response to the packagepayloadby a proprietary protocol to send backClientend.
Server-side code to achieve
For temporary convenience, reference is part of the array into the reseal ArrayList, in fact, written RPCwhen the frame should give priority to performance issues, as JDKprovided by a set of libraries and the like should be used sparingly as possible (in ArrayListan example, when the presence of the underlying expansion Object[]copies , resulting in an additional loss of performance and memory consumption), using very basic types and arrays as possible.
Define matching method isMethodMatcher associated with the class:
public interface MethodMatcher {
/**
* 查找一个匹配度最高的方法信息
*
* @param input input
* @return output
*/
MethodMatchOutput selectOneBestMatchMethod(MethodMatchInput input);
}
// 输入值
@EqualsAndHashCode
@Data
public class MethodMatchInput {
private String interfaceName;
private String methodName;
private List<String> methodArgumentSignatures;
private int methodArgumentArraySize;
}
// 输出值
@Data
public class MethodMatchOutput {
/**
* 目标方法实例
*/
private Method targetMethod;
/**
* 目标实现类 - 这个有可能是被Cglib增强过的类型,是宿主类的子类,如果没有被Cglib增强过,那么它就是宿主类
*/
private Class<?> targetClass;
/**
* 宿主类
*/
private Class<?> targetUserClass;
/**
* 宿主类Bean实例
*/
private Object target;
/**
* 方法参数类型列表
*/
private List<Class<?>> parameterTypes;
}Logic matching target method is as follows:
- Type of host and the method name as part of the method of Example matching certain criteria.
- If you passed in the parameter list of signatures, use priority list type parameter signature match.
- If you do not pass parameters signature list, then the number of parameters used for matching.
- If the parameter signature list and the list of parameters are not passed, then the only host names that match the type and method by means of examples.
- Considering the method of matching resolution process is relatively time consuming, the results need to be cached.
Analysis of this point, based on reflection, an abstract method for the preparation matcher BaseMethodMatcher, and the host delegate class information acquisition function to subclasses:
public class MethodMatchException extends RuntimeException {
public MethodMatchException(String message) {
super(message);
}
public MethodMatchException(String message, Throwable cause) {
super(message, cause);
}
public MethodMatchException(Throwable cause) {
super(cause);
}
}
@Data
public class HostClassMethodInfo {
private Class<?> hostClass;
private Class<?> hostUserClass;
private Object hostTarget;
}
@Slf4j
abstract class BaseMethodMatcher implements MethodMatcher {
private final ConcurrentMap<MethodMatchInput, MethodMatchOutput> cache = Maps.newConcurrentMap();
@Override
public MethodMatchOutput selectOneBestMatchMethod(MethodMatchInput input) {
return cache.computeIfAbsent(input, in -> {
try {
MethodMatchOutput output = new MethodMatchOutput();
Class<?> interfaceClass = Class.forName(in.getInterfaceName());
// 获取宿主类信息
HostClassMethodInfo info = findHostClassMethodInfo(interfaceClass);
List<Method> targetMethods = Lists.newArrayList();
ReflectionUtils.doWithMethods(info.getHostUserClass(), targetMethods::add, method -> {
String methodName = method.getName();
Class<?> declaringClass = method.getDeclaringClass();
List<Class<?>> inputParameterTypes = Optional.ofNullable(in.getMethodArgumentSignatures())
.map(mas -> {
List<Class<?>> list = Lists.newArrayList();
mas.forEach(ma -> list.add(ClassUtils.resolveClassName(ma, null)));
return list;
}).orElse(Lists.newArrayList());
output.setParameterTypes(inputParameterTypes);
// 如果传入了参数签名列表，优先使用参数签名列表类型进行匹配
if (!inputParameterTypes.isEmpty()) {
List<Class<?>> parameterTypes = Lists.newArrayList(method.getParameterTypes());
return Objects.equals(methodName, in.getMethodName()) &&
Objects.equals(info.getHostUserClass(), declaringClass) &&
Objects.equals(parameterTypes, inputParameterTypes);
}
// 如果没有传入参数签名列表，那么使用参数的数量进行匹配
if (in.getMethodArgumentArraySize() > 0) {
List<Class<?>> parameterTypes = Lists.newArrayList(method.getParameterTypes());
return Objects.equals(methodName, in.getMethodName()) &&
Objects.equals(info.getHostUserClass(), declaringClass) &&
in.getMethodArgumentArraySize() == parameterTypes.size();
}
// 如果参数签名列表和参数列表都没有传入，那么只能通过方法名称和方法实例的宿主类型匹配
return Objects.equals(methodName, in.getMethodName()) &&
Objects.equals(info.getHostUserClass(), declaringClass);
});
if (targetMethods.size() != 1) {
throw new MethodMatchException(String.format("查找到目标方法数量不等于1,interface:%s,method:%s",
in.getInterfaceName(), in.getMethodName()));
}
Method targetMethod = targetMethods.get(0);
output.setTargetClass(info.getHostClass());
output.setTargetMethod(targetMethod);
output.setTargetUserClass(info.getHostUserClass());
output.setTarget(info.getHostTarget());
return output;
} catch (Exception e) {
log.error("查找匹配度最高的方法失败,输入参数:{}", JSON.toJSONString(in), e);
if (e instanceof MethodMatchException) {
throw (MethodMatchException) e;
} else {
throw new MethodMatchException(e);
}
}
});
}
/**
* 获取宿主类的信息
*
* @param interfaceClass interfaceClass
* @return HostClassMethodInfo
*/
abstract HostClassMethodInfo findHostClassMethodInfo(Class<?> interfaceClass);
}Next, the function acquired by the host interface type based on the delegates to Springachieve, from the IOCacquisition of the container, define SpringMethodMatcher:
@Component
public class SpringMethodMatcher extends BaseMethodMatcher implements BeanFactoryAware {
private DefaultListableBeanFactory beanFactory;
@Override
public void setBeanFactory(@NonNull BeanFactory beanFactory) throws BeansException {
this.beanFactory = (DefaultListableBeanFactory) beanFactory;
}
@Override
HostClassMethodInfo findHostClassMethodInfo(Class<?> interfaceClass) {
HostClassMethodInfo info = new HostClassMethodInfo();
// 从容器中通过接口类型获取对应的实现,实现必须只有一个
Object bean = beanFactory.getBean(interfaceClass);
info.setHostTarget(bean);
info.setHostClass(bean.getClass());
info.setHostUserClass(ClassUtils.getUserClass(bean.getClass()));
return info;
}
}At this point, the module target matching method had been prepared, we need to approach the argument list deserialization. When writing the agreement, the author lists the method parameters methodArgumentsstored in Objectan array, when the transmission sequence into bytean array, after a protocol analysis, the actual type of the method parameter list to ByteBufan array (this is because Nettythe byte container is ByteBuf), you need to consider the ByteBufparameter type convert an array of instances of the target method. The main steps are as follows:
- If the method parameter list is empty, then do nothing, that is, call the method with no arguments.
- If the method argument list is not empty while the method parameter type list is not empty, the preferred method to convert the parameter type list.
- If the method argument list is not empty while the method parameter type list is empty, the
Method#getParameterTypes()parameter list of the type of conversion method obtained.
The method defines a parameter converter interface MethodArgumentConverter:
public interface MethodArgumentConverter {
ArgumentConvertOutput convert(ArgumentConvertInput input);
}
@Data
public class ArgumentConvertInput {
/**
* 目标方法
*/
private Method method;
/**
* 方法参数类型列表
*/
private List<Class<?>> parameterTypes;
/**
* 方法参数列表
*/
private List<Object> arguments;
}
@Data
public class ArgumentConvertOutput {
private Object[] arguments;
}The default method parameters of the converter to achieve the following:
@Slf4j
@Component
public class DefaultMethodArgumentConverter implements MethodArgumentConverter {
private final Serializer serializer = FastJsonSerializer.X;
@Override
public ArgumentConvertOutput convert(ArgumentConvertInput input) {
ArgumentConvertOutput output = new ArgumentConvertOutput();
try {
if (null == input.getArguments() || input.getArguments().isEmpty()) {
output.setArguments(new Object[0]);
return output;
}
List<Class<?>> inputParameterTypes = input.getParameterTypes();
int size = inputParameterTypes.size();
if (size > 0) {
Object[] arguments = new Object[size];
for (int i = 0; i < size; i++) {
ByteBuf byteBuf = (ByteBuf) input.getArguments().get(i);
int readableBytes = byteBuf.readableBytes();
byte[] bytes = new byte[readableBytes];
byteBuf.readBytes(bytes);
arguments[i] = serializer.decode(bytes, inputParameterTypes.get(i));
byteBuf.release();
}
output.setArguments(arguments);
return output;
}
Class<?>[] parameterTypes = input.getMethod().getParameterTypes();
int len = parameterTypes.length;
Object[] arguments = new Object[len];
for (int i = 0; i < len; i++) {
ByteBuf byteBuf = (ByteBuf) input.getArguments().get(i);
int readableBytes = byteBuf.readableBytes();
byte[] bytes = new byte[readableBytes];
byteBuf.readBytes(bytes);
arguments[i] = serializer.decode(bytes, parameterTypes[i]);
byteBuf.release();
}
output.setArguments(arguments);
return output;
} catch (Exception e) {
throw new ArgumentConvertException(e);
}
}
} All work is done preamble, a write current Serverinbound processor end ServerHandler, not a temporary code logic optimization, to achieve only, reflecting the called modules written directly in this class:
@Component
@Slf4j
public class ServerHandler extends SimpleChannelInboundHandler<RequestMessagePacket> {
@Autowired
private MethodMatcher methodMatcher;
@Autowired
private MethodArgumentConverter methodArgumentConverter;
@Override
protected void channelRead0(ChannelHandlerContext ctx, RequestMessagePacket packet) throws Exception {
log.info("服务端接收到:{}", packet);
MethodMatchInput input = new MethodMatchInput();
input.setInterfaceName(packet.getInterfaceName());
input.setMethodArgumentSignatures(Optional.ofNullable(packet.getMethodArgumentSignatures())
.map(Lists::newArrayList).orElse(Lists.newArrayList()));
input.setMethodName(packet.getMethodName());
Object[] methodArguments = packet.getMethodArguments();
input.setMethodArgumentArraySize(null != methodArguments ? methodArguments.length : 0);
MethodMatchOutput output = methodMatcher.selectOneBestMatchMethod(input);
log.info("查找目标实现方法成功,目标类:{},宿主类:{},宿主方法:{}",
output.getTargetClass().getCanonicalName(),
output.getTargetUserClass().getCanonicalName(),
output.getTargetMethod().getName()
);
Method targetMethod = output.getTargetMethod();
ArgumentConvertInput convertInput = new ArgumentConvertInput();
convertInput.setArguments(input.getMethodArgumentArraySize() > 0 ? Lists.newArrayList(methodArguments) : Lists.newArrayList());
convertInput.setMethod(output.getTargetMethod());
convertInput.setParameterTypes(output.getParameterTypes());
ArgumentConvertOutput convertOutput = methodArgumentConverter.convert(convertInput);
ReflectionUtils.makeAccessible(targetMethod);
// 反射调用
Object result = targetMethod.invoke(output.getTarget(), convertOutput.getArguments());
ResponseMessagePacket response = new ResponseMessagePacket();
response.setMagicNumber(packet.getMagicNumber());
response.setVersion(packet.getVersion());
response.setSerialNumber(packet.getSerialNumber());
response.setAttachments(packet.getAttachments());
response.setMessageType(MessageType.RESPONSE);
response.setErrorCode(200L);
response.setMessage("Success");
response.setPayload(JSON.toJSONString(result));
log.info("服务端输出:{}", JSON.toJSONString(response));
ctx.writeAndFlush(response);
}
}Write a Serverstart classes ServerApplicationin the Springcontainer after startup, start Nettythe service:
@SpringBootApplication(scanBasePackages = "club.throwable.server")
@Slf4j
public class ServerApplication implements CommandLineRunner {
@Value("${netty.port:9092}")
private Integer nettyPort;
@Autowired
private ServerHandler serverHandler;
public static void main(String[] args) throws Exception {
SpringApplication.run(ServerApplication.class, args);
}
@Override
public void run(String... args) throws Exception {
int port = nettyPort;
ServerBootstrap bootstrap = new ServerBootstrap();
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024, 0, 4, 0, 4));
ch.pipeline().addLast(new LengthFieldPrepender(4));
ch.pipeline().addLast(new RequestMessagePacketDecoder());
ch.pipeline().addLast(new ResponseMessagePacketEncoder(FastJsonSerializer.X));
ch.pipeline().addLast(serverHandler);
}
});
ChannelFuture future = bootstrap.bind(port).sync();
log.info("启动NettyServer[{}]成功...", port);
future.channel().closeFuture().sync();
} finally {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
}
}
}Finally, the preparation of contracts and contract packages to achieve:
- ch0-custom-rpc-protocol 项目根目录
- club.throwable
- utils 工具类
- protocol 协议
- exception 异常
- contract 契约
- HelloService 契约接口
- server 服务端
- contract
- DefaultHelloService 契约接口实现public interface HelloService {
String sayHello(String name);
}
// 实现
@Service
public class DefaultHelloService implements HelloService {
@Override
public String sayHello(String name) {
return String.format("%s say hello!", name);
}
}First start the server ServerApplication, and then start the one mentioned TestProtocolClient, output:
// 服务端日志
2020-01-15 00:05:57.898 INFO 14420 --- [ main] club.throwable.server.ServerApplication : 启动NettyServer[9092]成功...
2020-01-15 00:06:05.980 INFO 14420 --- [ntLoopGroup-3-1] club.throwable.server.ServerHandler : 服务端接收到:RequestMessagePacket(interfaceName=club.throwable.contract.HelloService, methodName=sayHello, methodArgumentSignatures=[java.lang.String], methodArguments=[PooledUnsafeDirectByteBuf(ridx: 0, widx: 6, cap: 6/139)])
2020-01-15 00:06:07.448 INFO 14420 --- [ntLoopGroup-3-1] club.throwable.server.ServerHandler : 查找目标实现方法成功,目标类:club.throwable.server.contract.DefaultHelloService,宿主类:club.throwable.server.contract.DefaultHelloService,宿主方法:sayHello
2020-01-15 00:06:07.521 INFO 14420 --- [ntLoopGroup-3-1] club.throwable.server.ServerHandler : 服务端输出:{"attachments":{},"errorCode":200,"magicNumber":10086,"message":"Success","messageType":"RESPONSE","payload":"\"doge say hello!\"","serialNumber":"65f01b8e89bb479b8a36a60bd6519617","version":1}
// 客户端日志
00:06:05.891 [main] INFO club.throwable.protocol.TestProtocolClient - 启动NettyClient[9092]成功...
...省略...
00:06:13.197 [nioEventLoopGroup-2-1] INFO club.throwable.protocol.TestProtocolClient - 接收到来自服务端的响应消息,消息内容:{"attachments":{},"errorCode":200,"magicNumber":10086,"message":"Success","messageType":"RESPONSE","payload":"\"doge say hello!\"","serialNumber":"65f01b8e89bb479b8a36a60bd6519617","version":1}Visible RPCcall was successful.
summary
Write RPCthe Serverend of the trick is to find the target processing methods and host class, when conversion method parameters need to be considered to simplify the process and improve efficiency, the rest is good and exception handling module package. Due to space limitations, the latter will first analyze the Clienthandle end, and then analyze the heartbeat processing, server optimization, even a docking registry and so on, in Netty, SpringBootthe preparation of a wait for the next blessing excellent framework RPCframework is not difficult, difficult is performance optimization and ecological support circle.
Demoproject address:
(Herein End c-1-d ea-20200115)