The J2EE platform often adopts a multi-layer distributed architecture system. Distributed service nodes need to communicate and interact (between business nodes and resource nodes), and the server and client need to interact (the terminal client needs to call the remote service of the server, the client is implemented in C, and there are other languages such as Java) realized). Therefore, the basic platform needs to provide a stable, efficient and scalable RPC service component.
Requirements for RPC service components:
1. Stable and high performance; as a basic skeleton component, high availability and high performance are necessary; the transport layer is expected to be connection-oriented TCP communication.
2. Lightweight, non-invasive, logically layered, easy to maintain and expand;
3. The data serialization method can support cross-language (C, C++, Java, etc.). Because we have client versions in multiple languages, we can support multiple serialization methods to facilitate extended serialization;
4. It is hoped that the protocol layer and transport layer logic are separated. This is convenient for expansion and reuse; it is convenient for secondary development, and when the application scale becomes larger, it is convenient to develop "service governance";
5. The serialized data should be as small as possible. Our application requires saving the user's traffic as much as possible, so this is also more critical; (do not consider compression, because compression will take up more CPU resources)
RPC open source solutions in the Java field:
1. Avro http://avro.apache.org/ A sub-project of the famous Hadoop. It itself is a serialization framework, and it also implements the functions of RPC;
Avro serialization features: It supports cross-language implementation. Compared with Apache Thrift and Google's Protocol Buffers, Avro's advantage is that it supports dynamic mode, that is, it does not generate code and avoids intrusiveness. DTO (data transfer object) as POJO is Not suitable for code generation. In addition, since Avro does not need field identifiers for labeling during serialization, the data generated by serialization using it is small (it should be the most streamlined in the existing serialization system), and finally its performance is also very good. Avro's own RPC implementation has its own highlights, such as support for synchronous and asynchronous calls, a handshake protocol function, and complete status monitoring. However, it is bound to Avro serialization, does not support the extension of the protocol, nor does it support the extended serialization method, and needs to be improved in terms of extensibility.
2. nfs-rpc http://code.google.com/p/nfs-rpc/ Taobao Niuren is an open source RPC framework. The transport layer uses mina/netty, the protocol layer is flexible and extensible, and supports multiple serialization methods. Currently, Hessian, Java, and Protocol Buffers are supported, and the serialization method is extensible. The framework design is excellent, the code quality is high, the readability is strong, the structure is clear, and the layers are clear. nfs-rpc is a very good RPC basic framework, and it is also very convenient to expand and develop on its basis. From the perspective of serialization, although Java serialization is simple to use, its performance is not high and cannot be cross-language. Hessian is cross-language and easy to use, but its performance is not dominant.
3. Dubbo http://code.alibabatech.com/wiki/display/dubbo/Home-zh Ali's open source distributed service framework, dedicated to providing high-performance and transparent RPC remote service invocation solutions, as well as SOA service governance plan. The function is very powerful, especially it provides a complete service governance function, which is very suitable for large-scale distributed applications. But it seems a bit too heavy for small to medium applications.
Summary:
1) From the perspective of serialization, Apache Thrift and Google's Protocol Buffers and Avro should belong to the same level of framework, which can be cross-language, excellent performance, and data reduction, but Avro's dynamic mode (without generating code, And the performance is very good) This feature is very popular, and it is more suitable for RPC data exchange.
2) RPC is logically divided into two layers, one is the transport layer, which is responsible for network communication; the other is the protocol layer, which packs and unpacks data according to a certain protocol format. Decided to refer to the characteristics of the previous three open source frameworks to implement a lightweight RPC framework.
avro-rpc is an RPC service framework that supports cross-language implementation. It is very lightweight, simple to implement, easy to use, and also convenient for users to carry out secondary development. Logically, the framework is divided into two layers:
1: Network transmission layer. Use Netty's Nio implementation.
2: Protocol layer. Extensible currently supports data serialization methods including Avro, Json, Hessian, and Java serialization. Users can register their own protocol formats and serialization methods.
main feature:
1: The client transport layer is logically separated from the application layer. The main responsibilities of the transport layer include connection creation, connection search and multiplexing, data transmission, and callback to the application layer after receiving the reply from the server;
2: The client supports synchronous and asynchronous calls. Asynchronous service can greatly improve system throughput. It is recommended to use asynchronous calls. To prevent asynchronously sending requests too quickly, the client adds the "request traffic limit" function, which calls timeout management.
3: The server has a protocol registration factory and a serialization registration factory. This makes it easy to customize service methods for different application scenarios. RPC should be just one way of serving. In a distributed system architecture, there are many ways to communicate between distributed nodes, such as MQ's TOP message, a message can have multiple subscribers. Therefore, avro-rpc is not only an RPC service framework, but also a basic skeleton of distributed communication, providing good scalability;
4: Very lightweight, good performance, suitable for secondary development.
Run the performance test instance
Start the server code.google.dsf.test.StartServerTest
Run the client code.google.dsf.test.performance.RPSTest
test object, the client sends a POJO (10 attribute fields) object
two sets in an asynchronous way , one runs the client, the other runs the server, the configuration is as follows:
• CPU: E5645 @ 2.40GHz 2 core
• Memory: 2G
• Network: 1000Mb
JVM Options: -server -XX:+UseParallelGC -XX:+AggressiveOpts -XX :+UseFastAccessorMethods -Xms512M -Xmx1024M
program executes for 6 minutes, RPS: 70308
Supplement: After optimization, under the same environment, the latest test RPS is: 100937
