In recent years micro-services fire, surely it is more or less the Friends of Bo contact. Micro-service concept,
calling each other among the service is an integral part. Do you know the service is between micro calling each other by what means it?
You might say, this is not simple, with socket chant. Service points between the caller and callee, we will establish a TCP or UDP connection to communicate just fine.
Talking, you might find that this thing is not so simple.
We took the simplest scenario:
A client invokes a function adder, the two integers together, and their return.
If you placed a local call, it is simply no longer simple, but once into a long-distance call, the threshold suddenly go up.
First of all, you have to be socket programming, at least we must first understand all agreements of this series, and then look at this book N-thick brick socket programming, learn to understand the model we had several socket programming.
This makes the already graduated from college competent job, became a five-year work experience are not necessarily doing a good job, but also, to get the first step socket programming, is the Long March, and there are many problems yet.
There is a problem
One problem: how to provide remote call syntax?
How to tell the client service side, I was an addition, and another subtraction. Is the string "add" it to you, or you pass an integer, such as 1 represents an addition, subtraction 2 shows?
If the server tells the client what should I this is an addition, can only add integer, decimal and string can not be added. And another addition "add1", it can achieve decimal integer addition and mixing, the return value of that? What is the correct time to return, what is wrong when they return?
Second problem: how to pass parameters?
Is to pass two integers, after a pass operand "add", or the first pass operator, re-transmission of two integers?
In addition, if we use UDP transport, the parameters on a message in okay, but if it is TCP, is a stream, the stream in which two calls around how to distinguish?
Question three: how to represent data?
In addition our example, the transfer is a fixed length int value, which has been good, if the variable length type, is a structure, or even a class, how should I do it? Even int, on different platforms have different lengths, how to do it?
Question 4: How do we know a server which implements a remote call? Port from which you can access the remote call?
Assume that the server implements multiple remote calls, each implementation is not possible in a process, not the same listening port, and because of his own server to achieve, can not we all use a recognized port, and may deploy multiple processes on a single machine, we need to seize the port, in order to prevent conflict, often using a random port, how to find the port that the client these listeners do?
Question 5: An error has occurred, retransmission, packet loss, performance and other issues how to do?
Local calls do not have this problem, but once on the network, these issues need to be addressed, because the network is not reliable, although in the same connection, we can guarantee packet loss, retransmission by the TCP protocol, but if the server crashes and restarts, the current connection is disconnected, TCP can not guarantee it, and need to apply themselves recall, re-transmission will do the same operation twice, remote call performance will not be affected it?
Solve the problem
To see so many problems, is not a headache? Remember when we understand http understanding of the three elements of the agreement do?
Local call a function in a lot of problems, such as lexical analysis, syntax analysis, semantic analysis of waiting, the compiler of these issues are basically to help us solve, but in the long-distance call, these questions we have to consider their own.
Agreement issues
For many companies this problem is to get a core communications group, which is socket programming Daniel, to achieve a unified repository, so that people other business groups to call, business people do not need to know the details of the intermediate transmission.
Communication between the two sides syntax, semantics, format, ports, error handling, etc., require the caller and the called party meeting to discuss the two sides reach an agreement. Once one party to change, to promptly notify the other party, otherwise it will be a problem.
However, not every company can solve this problem by Daniel team, but the use of already achieved good framework.
There is a large cattle (Bruce Jay Nelson) through a paper defines the standard RPC call of. All RPC rear frame are in accordance with this standard pattern.
The whole process is as follows:
- When a client application wants to initiate a remote call, it is actually Stub by local caller. It is responsible for the interface, invokes the methods and parameters, encoded by the agreed protocol specification, and transmitted via the local RPCRuntime, calls the network packet to the server;
- RPCRuntime the server receives the request, to the provider Stub encoded and then call the method server, the acquisition result, and the result of encoding, sent to the client;
- RPCRuntime clients receive the results, issued caller Stub decoded result is returned to the client.
The above process carve three levels: client, Stub layer, the server.
For the client and server are like as a local call, deal with focus on business logic on it. For Stub layer, both good agreement processing the syntax, semantics, encapsulation, decapsulation. For RPCRuntime, the main transmission processing performance, as well as network errors and exceptions.
The earliest an implementation of RPC called Sun RPC or ONC RPC . Sun was the first company to provide commercial RPC and RPC libraries compiler of the company. The RPC framework is used in the NFS protocol.
NFS (Network File System) is a network file system. For NFS run successfully, it is necessary to start two server, a mountd, to mount the file path. Another is nfsd, used to read and write files. NFS can go to a local directory on the local mount a remote directory, enabling users to make local directory in the local file read and write when the operation is a file on another machine remotely.
Remote operation and remote calls the idea is the same, just as local operation, so the NFS protocol is RPC-based implementation. Of course, no matter what RPC, it is the underlying socket programming.
XDR (External Data Representation, external data representation) is a standard data compression format, may represent a basic data types, said structure.
There are several basic data types.
The RPC call, all data types to be packaged into a similar format, and the RPC calls and returns results have strict format.
- XID uniquely identifies requests and replies. The request is 0, 1 replies;
- RPC version number, the version number to be matched at both ends of the RPC protocol. If not, it will return Deny, because RPC_MISMATCH;
- Programs are numbered. If the server can not find this program, it will return PROG_UNAVAIL;
- Program has a version number. If the program does not match the version number, will return PROG_MISMATCH;
- A program can have multiple methods, the method also numbered, if not find a method, will return PROG_UNAVAIL;
- Call requires certification authorization, if not passed, return to Deny;
- Finally, the list of parameters, if the argument can not be resolved, return GABAGE_ARGS;
In order to be successful RPC call, when the client and server-side implementation of RPC, we must first define a mutually recognized program, version, method and parameters.
For the above addition, the two sides agreed to a protocol definition file, the same token, if a NFS, mount and read and write, will have a similar definition.
With the protocol definition file, ONC RPC provides a tool to generate Stub client and server-side program based on this document.
XDR is lowermost document, for encoding and decoding parameters. This file is the client and server share, because only the two sides agreed to successful communication.
The client will call clnt_create create a connection, and then call add_1, this is a Stub function, the feeling is the same as calling a local function. This function is actually launched an RPC call to call ONC RPC library by calling clnt_call, to really send the request. Procedure Call is more complex, then follow specific instructions.
Of course, the server also has a Stub program, monitor the client's request, when the call arrives, determine if it is add, then call the real server-side logic, that is, the two numbers add up.
The server returns the result to the service side of Stub, Stub program sends the results to the client Stub, after receipt of the client Stub results are returned to the client application, thus completing the call too.
With this RPC framework, the first five issues of "how to provide remote call grammar?", "How to pass parameters?" And "how to represent data?" Basically resolved, these three issues we referred to as the agreement issue .
Transmission problems
The first three problems are solved, but the error, retransmission, packet loss and performance issues yet to be resolved, these issues we referred to as transmission problems . The Stub layer can not do anything, but to be implemented by the ONC RPC library.
In this library, in order to solve the transport problem, for each client, creates a transmission management, and each time an RPC call, will be a task in transmission management, you can see the familiar queuing, congestion window mechanism.
Since the network transmission time, often we need to wait while a synchronized manner is often relatively low efficiency and, therefore, the asynchronous model socket.
To enable asynchronous processing, the processing for remote calls, often through the state machine to achieve. Only when a state meet before the next step, if not the state, is not there waiting, but will stay out of resources to handle other RPC calls.
Above, can also be seen from FIG, this state transition diagram is very complex.
First, an initial state to enter, view the transport layer in the RPC queue has no free location, new RPC can handle the task, if not, indicating too busy or retry is ended. If the application is successful, you can allocate memory, access to the server port number, and then connect to the server.
Process connections have for some time, and thus wait for a connection to a result, if the connection fails, retry or directly ends. If the connection is successful, then please start sending RPC, RPC and then wait to obtain results. Similarly, this process takes time, if the transmission error, it is re-transmitted, if the connection is disconnected, to reconnect, if the timeout to retransmit. After getting the results, it can decode, normal end.
Here the connection process fails, retry, fail to send, timeouts, retry and other scenes, thus achieving a RPC framework, in fact, very difficult.
Service discovery issue
Transmission problems are solved, we also left behind a "how to find a random port that RPC service side," the problem we call service discovery problems in ONC RPC, the service discovery is implemented through the portmapper.
portmapper will be started on a well-known port, RPC program because it is the user's own writing, listens on a random port, but the RPC program startup, will be registered with the portmapper.
When the client wants to access the RPC service side of this program, first query the portmapper, RPC server program to obtain a random port, and then establish a connection to this random port, start the RPC call.
As can be seen from the figure, RPC mount command is implemented as call.
summary
- Remote calls look with socket programming can, in fact, is very complex, the agreement to solve the problems, transmission problems and service discovery issues;
- ONC RPC framework and NFS implementation, given three problems to solve the above exemplary implementation, a common protocol that is described in the file, and the file generated by the program Stub. RPC transmission generally requires a state machine requires another process specialize in service discovery.
Welcome to add a personal Micro Signal: Like if thoughts.
I welcome the attention of the public numbers, not only recommend the latest blog for you, there are more surprises waiting for you and resources! Learn together and common progress!
