Introduction : GDP (Go Develop Platform) is an RPC framework used in Baidu. It has complete RPC Client and RPC Server capabilities and can be used to develop various applications such as API, Web and back-end services. GDP Streaming RPC is a streaming RPC framework developed based on GDP RPC capabilities. Based on the realization of functions, a set of transmission frameworks oriented to streaming transmission scenarios are designed to provide solutions for streaming transmission application scenarios. The preferred streaming RPC solution in Baidu is baidu-rpc (open source project is brpc) streaming. GDP streaming is the Go version of brpc streaming, a streaming interface solution for Go developers.
The full text is 4700 words, the estimated reading time is 12 minutes
1. Introduction to streaming
1.1 Problem solved
In some data transfer scenarios, the client/server needs to send & receive a large amount of ordered data to each other, which is too large or continuously generated to fit in an RPC message body. For example: replicas or voice data passed between different nodes in a distributed system. An order export interface has 100,000 records. If traditional RPC is used, it needs to receive 100,000 records at one time before proceeding to the next step.
If we use streaming rpc then we can receive a record and process a record until all the data is transmitted. Although the client / server can split the data and transmit it through multiple RPCs, there are the following problems:
-
If the data is sent in parallel, it cannot be guaranteed that the receiving end receives the data in an orderly manner, and the logic of splicing the data is quite complicated.
-
If sent serially, each transfer has to wait for a network RTT + the delay of processing the data, especially the delay of the latter may be difficult to predict.
Common communication methods of RPC framework include simple RPC, server-side streaming RPC, client-side streaming RPC, and bidirectional streaming RPC. Their main features are:
Simple RPC : Pass in a request, return a response.
Server-side streaming RPC : The client sends a request, and the server can continuously return multiple responses. A typical example is that the client sends a stock code to the server, and the server sends the real-time data of the stock continuously. returned to the client.
Client-side streaming RPC : The client passes in multiple requests, and the server returns a final response. A typical example is to receive and process log files.
Two-way streaming RPC combines the characteristics of the first two RPCs, and both ends can pass in multiple requests and return multiple responses.
Figure 1. Four common communication modes of rpc
The streaming interaction model is designed to allow large chunks of data to be passed between client/server in a pipelined fashion. The implementation is bidirectional streaming RPC.
1.2 Design goals
-
Protocol compatible with brpc
-
Stream status can be queried
-
The order in which messages are received is the same as the order in which messages are sent, and messages from different streams are parallelized
-
custom serialization
-
A stocket supports establishing multiple streams
Because the streaming framework of brpc has been widely used inside and outside the company, the first choice for implementing GDP is to align brpc with protocols and functions, and the goal is to implement the Go version of brpc. A transmission task may not be completed by a stream, and parallel processing makes it more efficient. All frameworks need to guarantee the order of sending and receiving messages, as well as consistency and parallelism. In many scenarios, the data sent and received may be passed in or forwarded to services of other protocols. In this way, we consider implementing custom serialization and deserialization methods to reduce usage costs.
For IO-intensive scenarios, the connection overhead of tcp is very large, and we often encounter a situation where a stocket connection fills the network port. It is designed that a stocket can create multiple streams. This solution complicates the management logic of streams, but can save connection overhead. That is, the cost of establishing and destroying the flow is very small.
2. Frame design
2.1 Basic Concepts
The stream stream allows users to establish a user-mode connection between client/service, which is called stream. The transmission data of stream is based on messages. The input end can continuously write messages into the Stream, and the receiving end will press the input end. When the message is received in the writing order, the user can create or close a stream through the stream client. The client side and the server side of the stream are a symmetric structure.
The message body, the streaming data takes the message as the basic unit.
Stream connector Stream connection, the entity for data interaction, maintains a stocket connection, and multiple streamcs can exist on a stream connection at the same time.
event triggers events, including handshake success/failure, data processing/processing failure, processing timeout, stream closing, etc.
The handler callback method, different events correspond to the executed callbacks.
request request, including stream information and message body, because stream transmission is symmetric, all response returns are also reqeust.
2.2 Framework functions
Functionally, streaming RPC can be divided into the following functions:
callback extension
The data processing on the server side uses the handler method to call back the processing. The request is processed every time the request is made, and the handler is registered when the server is established. You can also implement custom handlers of other types. After the handler is registered, the corresponding callback handler is executed through event triggering.
stream component
Streaming rpc enables users to establish userland connections between clients/services, called streams. The transmission data of a stream is a message as the basic unit. This layer is the entity of stream processing logic, including event registration and distribution, stream status query, triggered callback, judgment timeout, and protocol encapsulation are all completed at this layer. The common part of the stream component is the resource component, which is responsible for the management of streams and connections, and is mainly responsible for the idle stream connections allocated to the stream when the stream is created, and the release of resources when it is closed.
stream base module
The common part of the stream component is the resource component, which is responsible for the management of streams and connections, and is mainly responsible for the idle stream connections allocated to the stream when the stream is created, and the release of resources when it is closed. A tcp stocket can correspond to multiple streams, one is connected to a read-write channel, and the boundary characteristics of the message body allow multiple streams to share one stocket, multiple streams process data in parallel, and serial message transmission. Stream management is mainly responsible for selecting an idle stream connection when creating a stream, releasing resources when closing it, recording an erroneous connection and closing the corresponding stream on the connection when a connection read/write timeout or error occurs.
basic components
Provides the basic capabilities and dependent components of rpc, including service discovery, load balancing, network dial-up, and provides maintenance of connections under stream.
Common components include basic functions such as logs, configuration, and environment information, so that stream-layer applications can integrate with the infrastructure without any sense. Because the architecture of the stream is a symmetric structure, the architecture of the stream component layer and the basic component layer is the architecture shared by the server and the client.
2.3 Implementation Details
stream interaction
A complete interaction includes three stages : handshake, communication, and shutdown .
The stream client first creates a stream locally, and then establishes a stream with the specified service through an RPC. If the service chooses to accept the stream after receiving the request, the stream will be established successfully after the response returns to the client. Any error in the process marks the RPC as failed, which also means the stream creation failed.
Taking the process of establishing a connection under Linux as an example, the client first creates a socket (creates a stream), and then tries to establish a connection with the remote end (establishes a stream through RPC), and the connection is established after the remote end accepts (the service is successfully created after accepting it). After the establishment, the client and the server generate a stream id respectively.
After the stream is established, it enters the communication state and uses RPC (strm protocol) for duplex interaction. stream provides a method for streaming sending and receiving messages. The framework is responsible for creating streams and TCP stocket. After the server handshakes, a stream is generated, and the stream id and the end stream id Consistently share a stocket connection. After establishment, execute the accept(context.Context) error method to interact with the client Stream.
When establishing or accepting a stream, the handler registered by the user handles the write data of the peer end, the connection closing and the idle timeout. The closing operation is initiated by any segment, and the peer is notified through an RPC to destroy the stream and close the local resources. When there is no stream on the stocket, the connection is closed at the same time.
The following figure represents the stream state diagram established by the client and closed by the client:
Figure 3. The streaming communication model
2.4 Problems solved
How to ensure that multiple streams are established on a tcp connection
The structure of client and server is symmetrical, and both include stream manager, stream connect, and stream structure, which correspond to the functions of stream allocation (manager), saving connection (connect), and data interaction (stream).
In the process of establishing a stream, a tcp connection is saved in a stream connect, and a stream connect saves multiple streams. When the client/server establishes a stream, the manager will check which connection is in an unsaturated state, that is, the stream can be stored. If no stream is generated A connect, store the stream in this connect and apply for resources.
connect ensures that each stream sends and receives data in parallel, and uses the read channel to send asynchronously. The message body ensures the data boundary. Each time a message body is sent, it is mutually exclusive. Reading stocket uses asynchronous threads to block reading, and allocates data to the corresponding data through streamid. stream.
stream handles exceptions
In most cases, the stream is automatically closed by the client after the transmission is completed, or the server is closed after the idle timeout, but when an error or network abnormality occurs, it is necessary to ensure the exit mechanism of the double-ended stream. When either end processes an exception, the stream will enter an abnormal state, enter the shutdown process, send a shutdown request to the peer, and the peer will also close after receiving the request.
As shown below, the flow interaction process initiated by the server abnormally:
Figure 4. Handling of streaming exceptions in the streaming framework
stream handles network exceptions
For example, when a stream finds that reading and writing fails, it will enter the closing process and notify the stream manager at the same time. All streams that share a connection with this stream will receive the notification from the manager and close all of them. Due to the symmetric structure of the flow, the peer end will enter the same flow when it perceives network anomalies. Exception handling ensures that the error stream will not undertake subsequent work and ensures the exit of the abnormal connection.
Figure 5. Handling of streaming network exceptions
When the reading and writing of the stream fails, the local stream will perceive it, so that the interaction between the two-end streams fails and the stream is automatically closed, the resources are released, and the stream manager is notified at the same time. The stream sharing a stocket in this stream is notified by the stream manager to close. Free up resources.
How to register callbacks
The framework executes the corresponding callback when the event is triggered. The server-side custom callback method is defined as the handler interface in the framework. Implementing the corresponding interface implements the callback. The advantage of this is that it avoids the redundancy caused by the registration of multiple callback methods. Code and scalability issues.
The basic handler is the callback method for the server to process the client request, and the definition is similar to:
Basic handler definition
type Handler interface {
Handle(context.Context, Stream, Request) error
}
The server structure of the server provides a registration method to register the handler:
Handler is the basic callback method and must be implemented. Other types of implementation are optional, including: handshake response, handshake success, handshake failure, timeout callback, error handling, stream closing, etc.
Different callback methods correspond to different interface{}, and the user only needs to implement the corresponding interface{}, that is, the custom callback is completed.
===
3. Access case
We selected a language access module. The original architecture uses brpc to realize the upstream and downstream voice transmission and processing. As shown in the figure, the Go module is used to replace the client and the server side for small traffic verification, and finally completely replaced, as shown in the figure:
Figure 6. Alternative to brpc streaming using GDP streaming
The differences between the new and old modules of the new modules are compared through performance tests. Among them, the success rate and delay are basically the same, and a smooth upgrade is achieved.
On the performance indicators of GDP's Straming RPC:
The forwarding of single-instance small packets (within 1k) reaches 2.5w/s, which reaches the level of brpc under the same environment, which can meet the requirements of high performance. Compared with brpc, the CPU increases by about 20% under the same throughput rate.
4. Summary
Go has the advantages of concurrency and performance that dynamic languages do not have, and occupies less system threads. Compared with C++, Go has a simple syntax and good support for public protocols, which allows it to support efficient business development and debugging on the basis of taking into account performance. and operations.
GDP Streaming provides the Go version of brpc streaming, which is based on SOLID software design principles, making the framework highly scalable, and providing a practical solution for streaming applications for Gopher in Baidu.
Recommended reading:
Decoding optimization in Baidu APP video playback
Baidu Aifanfan real-time CDP construction practice
Technology revealed! The exploration and practice of low-code in Baidu search
Baidu Smart Cloud Practice - Static File CDN Acceleration
Simplify complexity--A summary of the main data architecture of Baidu's smart applet
---------- END ----------
Baidu Geek says
Baidu's official technical public account is online!
Technical dry goods·Industry information·Online salon·Industry conference
Recruitment information · Internal push information · Technical books · Baidu peripherals