Overview of Communication Protocols

In the distributed service framework, one of the most basic problems is how to communicate with remote services. There are many technologies that can realize remote communication in the Java field, such as: RMI, MINA, ESB, Burlap, Hessian, SOAP, EJB and JMS Wait, what is the relationship between these nouns, and what are the principles behind them? Understanding these is the basic knowledge of implementing a distributed service framework, and if there are high performance requirements, then in-depth understanding The mechanism behind these technologies is necessary. In this blog, we will explore the details in the future, and we will invite you to provide more introductions to the technologies and principles of remote communication.

Basic Principles
To realize the communication between network machines, we must first look at the basic principles of network communication in computer systems. At the bottom level, what network communication needs to do is to transmit streams from one computer to another computer. Based on the transmission Protocol and network IO to achieve, among which the famous transmission protocols are http, tcp, udp, etc. http, tcp, udp are all transmission protocols extended for certain application scenarios based on the concept of Socket, network IO, mainly There are three methods: bio, nio, and aio. All distributed application communications are implemented based on this principle. Just for the ease of use of applications, various languages ​​usually provide some application layer protocols that are closer to the ease of use of applications.

Application-level protocol For
remote service communication, the goal that needs to be achieved is to initiate a request on one computer, and another machine will process the request and return the result to the requester after receiving the request. There will be such as one way request, Synchronous request, asynchronous request and other request methods, according to the principle of network communication, what needs to be done is to convert the request into a stream and transmit it to the remote end through a transmission protocol. The remote computer processes the requested stream after receiving it. After completion, the result is converted into a stream and returned to the caller through the transmission protocol.
The principle is like this, but for the convenience of application, the industry has launched many application-level protocols based on this principle, so that you don’t need to directly operate such low-level things. Usually, the application-level remote communication protocol will provide:
1. In order to avoid the trouble of doing streaming operations directly, a standard transmission format that is easier to use or fits the language is provided;
2. The realization of the network communication mechanism is to complete the conversion of the transmission format into a stream for you, through some kind of transmission The protocol is transmitted to the remote computer, and the remote computer, after receiving the stream, converts it into a transmission format, and stores it or notifies the remote computer in some way.
Therefore, when learning the application-level remote communication protocol, we can learn with these questions:
1. What is the standard format of transmission?
2. How to convert the request into a transmitted stream?
3. How to receive and process streams?
4. What is the transmission protocol?
However, the application-level remote communication protocol does not make much improvement in the transmission protocol, mainly in terms of stream operations, so that the process of generating streams and processing streams by the application layer is more suitable for the language or standard used. Transmission protocols are usually optional, well-known in the java field are: RMI, XML-RPC, Binary-RPC, SOAP, CORBA, JMS, let's take a look at these application-level protocols for remote communication:
--- -------------------------------------------------- -------------------------------------------------- -------------------------------------------
RMI
RMI is a typical java Customized remote communication protocol, we all know that in single vm, we can achieve communication by directly calling java object instance, then in remote communication, it is of course the best if we can follow this method. This kind of remote communication The mechanism becomes RPC (Remote Procedure Call), and RMI was born with this goal in mind.
Let's take a look at the principle of a complete remote communication process based on RMI:
1. The client initiates a request, and the request is forwarded to the stub class of the RMI client;
2. The stub class serializes the requested interface, methods, parameters and other information;
3 、Transfer the serialized stream to the server based on socket;
4. The server receives the stream and forwards it to the corresponding skelton class;
5. The skelton class deserializes the requested information and calls the actual processing class;
6. Processing After the class is processed, the result is returned to the skelton class;
7. The Skelton class serializes the result, and sends the stream to the client's stub through the socket;
8. The stub deserializes the deserialized Java Object after receiving the stream. returned to the caller.
Let's see a better illustration of this process by jboss-remoting:

Hessian <wbr>Principle Analysis

According to the principle, we will answer several questions brought by learning the application-level protocol before:
1. What is the standard format of transmission?
is a Java ObjectStream.
2. How to convert the request into a transmitted stream?
Converts the requested java object information into a stream based on the Java serialization mechanism.
3. How to receive and process streams?
Start the corresponding listening port according to the adopted protocol, deserialize the stream based on the Java serialization mechanism when a stream enters, and obtain the corresponding processing object information according to the RMI protocol, call and process it, and the result after processing It is also returned based on the java serialization mechanism.
4. What is the transmission protocol?
socket.
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
XML-RPC
XML - RPC is also a remote calling protocol similar to RMI. The difference between it and RMI is that it defines the requested information (requested objects, methods, parameters, etc.) in standard xml format. What are the benefits of this? , that is, it can also be used when communicating across languages.
Let's take a look at a remote communication process of the XML-RPC protocol:
1. The client initiates a request and fills in the request information according to the XML-RPC protocol;
2. After filling, the xml is converted into a stream and transmitted through the transmission protocol;
3. After receiving the stream, it is converted into xml, and the requested information is obtained and processed according to the XML-RPC protocol;
4. After processing, the result is written into xml according to the XML-RPC protocol and returned.
The above process is shown in the figure:

Hessian <wbr>Principle Analysis

Also to answer the question:
1. What is the standard format of transmission?
Standard format XML.
2. How to convert the request into a transmitted stream?
Convert XML to stream.
3. How to receive and process streams?
Obtain the requested stream through the listening port, convert it into XML, obtain the requested information according to the protocol, process it, and write the result into XML for return.
4. What is the transmission protocol?
Http.
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
Binary-RPC
Binary - RPC is similar to XML-RPC from the name, the only difference is that the standard format of transmission is changed from XML to binary format.
To answer the same questions:
1. What is the standard format of transmission?
Standard format binary file.
2. How to convert the request into a transmitted stream?
Convert a binary format file to a stream.
3. How to receive and process streams?
Obtain the requested stream through the listening port, convert it into a binary file, obtain the requested information according to the protocol, process it, and write the result into XML for return.
4. What is the transmission protocol?
Http.
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
SOAP
SOAP originally means Simple Object Access Protocol. It is a lightweight, XML-based communication protocol for information exchange in a distributed environment. It can be considered that SOAP is an advanced version of XML RPC. The principles of the two are exactly the same. http+XML, the only difference is that the XML specifications defined by the two are different. SOAP is also a service invocation protocol standard adopted by Webservice, so it will not be elaborated here.
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
CORBA
Common Object Request Broker Architecture (Common Object Request Broker [Scheduler] Program Architecture) is a set of standards used to define "distributed object systems", and is initiated and standardized by OMG (Object Menagement Group). The purpose of CORBA is to define a set of protocols, objects conforming to this protocol can interact with each other, no matter what language they are written in, no matter what machine and operating system they run on.
In my opinion, CORBA is an architecture similar to SOA, covering optional remote communication protocols, but it cannot be included in the communication protocol itself, and CORBA is basically eliminated, and I don’t know much about CORBA. will not be elaborated.
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
JMS
JMS is a means and method to realize remote communication in the java field. It is different from RPC when implementing remote communication based on JMS. Although it can achieve the effect of RPC, it is not defined from the protocol level, so we do not think that JMS It is an RPC protocol, but it is indeed a long-distance communication protocol. There are also things similar to JMS in other language systems. This kind of mechanism can be called a message mechanism in a unified way, and the message mechanism is usually high concurrency and distributed. A communication mechanism recommended by the domain, a major problem here is fault tolerance (see ErLang paper for details).
Let's look at the process of a remote communication in JMS:
1. The client converts the request into a message that conforms to the JMS regulations;
2. The message is put into the JMS Queue or Topic through the JMS API;
3. If it is a JMS Queue, it is being sent In the corresponding target Queue, if it is a Topic, it will be sent to the JMS Queue subscribed to this Topic.
4. The processing end obtains the message by training the JMS Queue in turn. After receiving the message, it parses and processes the message according to the JMS protocol.
Answer the questions:
1. What is the standard format of transmission?
Message specified by JMS.
2. How to convert the request into a transmitted stream?
Put the parameter information into the Message.
3. How to receive and process streams?
The JMS Queue is trained in rotation to receive the Message, and it is processed after receiving it. After processing, it is still sent or Multicast in the Queue in the form of a Message.
4. What is the transmission protocol?
Unlimited.
Based on JMS is also one of the commonly used methods to realize remote asynchronous invocation.

Optional Implementation Technology
Of course, the above principle does not introduce all the optional remote communication protocols in the java field, such as ORMI used by EJB, a simple Http Invoker defined by Spring, and so on.
After reading the principle, let's take a look at the current framework or library that can be used for remote communication in the java field. Well-known ones are: JBoss-Remoting, Spring-Remoting, Hessian, Burlap, XFire(Axis), ActiveMQ, Mina, Mule, EJB3, etc. Wait, let's make a brief introduction and evaluation of each. In fact, to do a distributed service framework, you must have a very deep understanding of these things, because the distributed service framework actually includes solutions to the distributed field and There are two aspects of the application-level field.
Of course, you can also implement your own communication framework or library based on the principle of remote network communication (transport protocol+Net IO).
So what questions do you take to learn about these telecommunication frameworks or libraries?
1. What protocol is it based on?
2. How to initiate a request?
3. How to convert the request into a format that conforms to the protocol?
4. What transmission protocol is used for transmission?
5. What mechanism does the responder use to receive the request?
6. How to restore the stream to the transmission format?
7. How to respond after processing?
-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
JBoss-Remoting
Jboss-remoting is a remote communication framework in the java field written by jboss. Based on this framework, RPC of java objects based on various transmission protocols can be easily realized.
To answer the questions directly:
1. What protocol is it based on?
JBoss-Remoting is a communication framework, so it supports communication in a variety of protocols, such as pure socket+io, rmi, http+io, and so on.
2. How to initiate a request?
In JBoss-Remoting, you only need to pass the request parameter object that needs to be initiated into the InvocationRequest object of jboss-remoting, or you can encapsulate the InvocationRequest object that meets the requirements based on the InvocationRequest according to the protocol.
3. How to convert the request into a format that conforms to the protocol?
JBoss-Remoting converts requests into object byte streams based on the Java serialization mechanism or JBoss' own serialization implementation.
4. What transmission protocol is used for transmission?
Support a variety of transport protocols, such as socket, http, etc.
5. What mechanism does the responder use to receive the request?
The responder only needs to register its own processing object in the Connector object on the server side provided by JBoss-Remoting.
6. How to restore the stream to the transmission format?
JBoss-Remoting restores request information to java objects based on the java serialization mechanism or jboss's own serialization implementation.
7. How to respond after processing?
After processing, the result object can be returned directly. jboss-remoting will serialize the object according to the protocol and return it to the calling end.
In addition, jboss-remoting supports a variety of communication methods, such as synchronous/asynchronous/one-way communication.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
Spring-Remoting
Spring - Remoting is a remote communication framework provided by Spring in the java field. Based on this framework, it is also very simple to publish ordinary spring beans in a certain remote protocol, and it is also possible to configure spring beans as bean for remote invocation.
1. What protocol is it based on?
Like JBoss-Remoting, as a remote communication framework, Spring supports multiple protocols by integrating multiple remote communication libraries, such as rmi, http+io, xml-rpc, binary-rpc, etc.
2. How to initiate a request?
In Spring, since it uses proxy implementation for remotely invoked beans, the request is completely invoked through the service interface.
3. How to convert the request into a format that conforms to the protocol?
Spring converts the requested object information into a stream according to the protocol. For example, Spring Http Invoker is implemented based on a protocol defined by Spring itself. The transmission protocol is http, and the request information is converted into a stream based on the java serialization mechanism. transmission.
4. What transmission protocol is used for transmission?
Support a variety of transport protocols, such as rmi, http and so on.
5. What mechanism does the responder use to receive the request?
The responder follows the protocol to receive the request. For the user, it is only necessary to configure the common spring bean as the responder or provide the server through the spring configuration method.
6. How to restore the stream to the transmission format?
Restore according to the protocol.
7. How to respond after processing?
After processing, you can directly return, and spring-remoting will do the corresponding serialization according to the protocol.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
Hessian
The Hessian was created by A remote communication library based on binary-RPC implemented by caucho.
1. What protocol is it based on?
Based on Binary-RPC protocol implementation.
2. How to initiate a request?
The request needs to be initiated through the API provided by Hessian itself.
3. How to convert the request into a format that conforms to the protocol?
Hessian serializes the request information through its custom serialization mechanism to generate a binary stream.
4. What transmission protocol is used for transmission?
Hessian is transmitted based on the Http protocol.
5. What mechanism does the responder use to receive the request?
The responder receives the request according to the API provided by Hessian.
6. How to restore the stream to the transmission format?
Hessian deserializes the request information according to its private serialization mechanism, and it is the corresponding request information object when it is passed to the user.
7. How to respond after processing?
Return directly after processing. Hessian serializes the result object and transmits it to the calling end.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
Burlap
Burlap also has Provided by caucho, it differs from hessian in that it is based on the XML-RPC protocol.
1. What protocol is it based on?
Implementation based on XML-RPC protocol.
2. How to initiate a request?
According to the API provided by Burlap.
3. How to convert the request into a format that conforms to the protocol?
Convert the request information into a protocol-compliant XML format, and convert it into a stream for transmission.
4. What transmission protocol is used for transmission?
Http protocol.
5. What mechanism does the responder use to receive the request?
Listen to Http requests.
6. How to restore the stream to the transmission format?
Restore according to the XML-RPC protocol.
7. How to respond after processing?
The returned result is written into XML and returned to the calling end by Burlap.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
XFire, Axis
XFire , Axis is the implementation framework of Webservice. WebService can be regarded as a complete SOA architecture implementation standard. Therefore, the use of XFire and Axis means the use of webservice.
1. What protocol is it based on?
Based on SOAP protocol.
2. How to initiate a request?
Call it directly after obtaining the proxy of the remote service.
3. How to convert the request into a format that conforms to the protocol?
The request information is converted into XML format that follows the SOAP protocol, and the frame is converted into a stream for transmission.
4. What transmission protocol is used for transmission?
Http protocol.
5. What mechanism is the responder based on to receive the request?
Listen to Http requests.
6. How to restore the stream to the transmission format?
Restore according to the SOAP protocol.
7. How to respond after processing?
The returned result is written into XML and returned by the framework to the calling end.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
ActiveMQ
ActiveMQ is JMS It is a good choice to realize remote communication based on a message mechanism such as JMS. After all, the function of the message mechanism itself makes it easy to implement synchronous/asynchronous/one-way calls based on it, and the message mechanism is from a fault-tolerant perspective. It is also a good choice, which is an important basis for Erlang to be fault-tolerant.
1. What protocol is it based on?
Based on the JMS protocol.
2. How to initiate a request?
Follow the JMS API to initiate a request.
3. How to convert the request into a format that conforms to the protocol?
Not very clear, guess should be binary stream.
4. What transmission protocol is used for transmission?
Support a variety of transport protocols, such as socket, http and so on.
5. What mechanism is the responder based on to receive the request?
Listen on a port that conforms to the protocol.
6. How to restore the stream to the transmission format?
Same as question 3.
7. How to respond after processing?
Follow the JMS API to generate messages and write them to the JMS Queue.
Examples of remote communication based on such mechanisms as JMS include Spring-Integration, Mule, Lingo, and so on.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
Mina
Mina is Apache The communication framework provided has not mentioned the network IO before. The framework or library mentioned before is basically based on BIO, while Mina uses NIO. When the concurrency increases, NIO will be more obvious than BIO. The performance improvement of Java, and the improvement of java performance has a lot to do with the close integration of NIO and OS.
1. What protocol is it based on?
Based on pure Socket+NIO.
2. How to initiate a request?
Through the Client API provided by Mina.
3. How to convert the request into a format that conforms to the protocol?
Mina follows the java serialization mechanism to serialize the request object.
4. What transmission protocol is used for transmission?
Support a variety of transport protocols, such as socket, http and so on.
5. What mechanism does the responder use to receive the request?
Listen to the protocol port in the NIO way.
6. How to restore the stream to the transmission format?
Deserialize the request object following the java serialization mechanism.
7. How to respond after processing?
Follow the Mina API to return.
MINA is NIO way, so there is no suspense to support asynchronous calls.

-------------------------------------------------- -------------------------------------------------- ----------------------------------------------
EJB
EJB most prominent The difference lies in its distribution. EJB adopts the ORMI protocol, which is similar to the RMI protocol. However, EJB's outstanding performance in distributed communication security control, transport pool, smart proxy, etc. makes it a force that cannot be ignored in the distributed field. .
1. What protocol is it based on?
Based on ORMI protocol.
2. How to initiate a request?
EJB calls.
3. How to convert the request into a format that conforms to the protocol?
Follow the java serialization mechanism to serialize the request object.
4. What transmission protocol is used for transmission?
socket.
5. What mechanism does the responder use to receive the request?
Listening protocol port.
6. How to restore the stream to the transmission format?
Deserialize the request object following the java serialization mechanism.
7. How to respond after processing?
Just return the processing object directly.

In the previous series of distributed service framework articles, there is a misleading suspicion of jndi. In this blog, the mechanism of jndi is also mentioned incidentally. Since JNDI depends on the specific implementation, I can only explain jboss's jndi here. has been realized.
After the object instance is bound to the jboss jnp server, when the remote end uses the context.lookup() method to obtain the remote object instance and starts to call, the implementation method of jboss jndi is to obtain the object instance from the jnp server and serialize it back to Locally, then deserialize locally, then do the class call locally.
Through this mechanism, you can know that the local must have a class bound to the object instance on jboss, otherwise the deserialization will definitely fail, and what remote communication needs to do is to perform an action remotely , and get the corresponding results, it can be seen that remote communication cannot be achieved purely based on JNDI.
But JNDI is also a key technical point for implementing a distributed service framework, because it can implement transparent remote and local calls, just like ejb, and it is also a good mechanism to hide the actual deployment (like datasource) etc. plan.
Summary
From the previous series of analysis, we can see that in the field of remote communication, there are quite a lot of knowledge points involved, such as: communication protocols (Socket/tcp/http/udp /rmi/xml-rpc etc.), message mechanism , Network IO (BIO/NIO/AIO), MultiThread, transparent solutions for local calls and remote calls (involving java classloader, Dynamic Proxy, Unit Test etc.), asynchronous and synchronous calls, network communication processing mechanisms (automatic reconnection, Broadcast, exception, pool processing, etc.), Java Serialization (private serialization mechanism of various protocols, etc.), implementation principles of various frameworks (transmission format, how to convert transmission format into stream, how to convert request information into transmission) format, how to receive the stream, how to restore the stream to the transmission format, etc.), to be proficient in which of them, you have to decide according to the actual needs, only when you understand the principle can you make a choice easily , you can even make a private remote communication protocol according to your needs. For those who are engaged in distributed service platforms or develop larger distributed applications, I think at least the knowledge points mentioned above need to be understood.
Reprinted from http://blog.sina.com.cn/s/blog_56fd58ab0100mrl6.html