In-depth thinking about rpc framework face-to-face series three

6 The process of request invocation of an rpc framework (Xiaohongshu interview)

6.1 Talk about the principles of rpc calls, such as how services are discovered, how they are called, and how providers respond. How to ask and how to come back

The core purpose of an RPC (Remote Procedure Call) framework is to allow a program to call a program or service in another address space (usually another machine on a shared network), just like calling a local method or function. Here is an overview of how a typical RPC framework handles requests:

1. Service definition and interface :

   • Typically, developers start by defining services and their interfaces. These interfaces define methods that can be called remotely.
   • Use IDL (Interface Definition Language) to describe the service and generate client and server stubs.

2. Service registration :

   • After the service provider is started, it will register its own address and the service information it provides to the registration center.

3. Service Discovery :

   • When a service consumer (or client) starts, it subscribes to the services it wants to invoke from the registry.
   • The registry returns the address of the service provider to the consumer. For efficiency, these addresses are usually cached locally at the consumer.

4. remote call :

   • When a client wants to call a remote method, it initiates the call through a local proxy or stub.
   • Stubs are responsible for converting method calls into network requests. This usually involves serializing method names, parameters, etc., and sending them over the network.
   • Load balancing strategies such as round robin, random or least active calls can be applied in this step to select the most suitable service provider instance.

5. Request processing :

   • After receiving the network request, the service provider deserializes the request data to obtain the original method name and parameters.
   • The service provider then executes the method locally and obtains the result.

6. The response returns :

   • After the service provider serializes the execution result, it returns to the service consumer through the network.
   • The service consumer's stub deserializes the returned data, converts it back to the original method call result, and returns it to the caller.

7. Error handling :

   • If an error occurs during the RPC process (such as a network outage, a service provider crash, etc.), the RPC framework may provide fault tolerance mechanisms.
   • Fault tolerance strategies may include rerouting, retrying, returning default results, etc.

8. Monitoring and logging :

   • To track and monitor the performance and reliability of remote calls, RPC frameworks typically provide logging and monitoring capabilities.

This process describes a typical synchronous RPC call. But modern RPC frameworks also support asynchronous calls, where the client can continue with other operations without waiting for a response.

6.2 Use IDL (Interface Definition Language) to describe the service and generate stubs for the client and server. what does stub mean here

In the context of RPC (Remote Procedure Call), a "stub" is a proxy or interface that hides the complexity of the underlying remote call. Through stubs, developers can call remote methods like local methods without caring about details such as underlying network communication, data serialization/deserialization, etc.

When describing a service using IDL (Interface Definition Language), you are basically describing what functions or methods a remote service should have. Then, use a specific tool to generate client and server stubs based on this IDL.

• Client stub (Client Stub) : When you call a remote method on the client, you are actually calling the client stub. This stub will be responsible for:

  • Parameters for serializing method calls.
  • Send these serialized data to the server over the network.
  • Wait for a response from the server.
  • Receives the server's response, deserializes it, and returns it to the caller.

• Server Stub : There is a matching stub on the server side that is responsible for:

  • Receive the request sent by the client and deserialize it.
  • Call the local implementation method.
  • Serialize the response result.
  • Send the serialized response result back to the client.

This design allows the client and server to communicate on different platforms and/or using different programming languages. Because as long as they follow the same IDL definition and use the same data serialization/deserialization mechanism, they can understand each other.

6.3 How is it routed? (routing strategy (such as round robin, random, least active call, etc.)

Service routing:
In the case of multiple service providers providing the same service, the client needs to decide which service provider to call. At this time, routing strategies (such as round robin, random, least active call, etc.) will be applied. The load balancing strategy will select one according to the list of service provider addresses cached locally.

Explained how I designed the load balancing algorithm and the applicable scenarios of each strategy: the
purpose of load balancing is to distribute network traffic to multiple servers to ensure that each server will not go down due to overload and can maximize throughput, minimize response time and avoid any single point of failure. The following are some commonly used load balancing strategies and their applicable scenarios:

1. Round Robin

   • Strategy : This is the simplest load balancing algorithm, and requests are assigned to servers in order. If the server list reaches the end, start over.
   • When to use it : Round robin is a good choice when all servers have similar specifications and expected request processing times are similar.

2. Weighted Round Robin

   • Strategy : Similar to polling, but each server is given a weight, and servers with higher weights will receive more requests.
   • Applicable scenarios : When you have servers of different capabilities and want each server to receive traffic commensurate with its capabilities.

3. Least Connections

   • Policy : Route requests to the server with the fewest connections.
   • Applicable scenarios : Applicable to scenarios where the server processing speed is roughly the same, but the time to process requests can vary. For example, if you have a long polling or Websockets service.

4. Weighted Least Connections

   • Policy : Similar to Least Connections, but takes into account the weight of each server.
   • Applicable scenarios : When the server specifications and processing speeds are different, and the time to process requests is variable.

5. IP hash (IP Hash)

   • Policy : Determine which server should be routed to based on the IP address of the requester. It is usually obtained by taking the hash value of the IP and then modulo the number of servers.
   • Useful scenario : When you want clients from a specific IP to always connect to the same server, this is useful when you need to keep sessions or some cascading data caching.

6. URL Hash

   • Policy : Determine which server to route to based on the hash value of the request URL.
   • Applicable scenarios : Especially suitable for HTTP caching servers, because the same URL requested can be guaranteed to be routed to the same server containing its cache.

7. Least Latency

   • Strategy : The load balancer continuously detects the latency or response time of each server and routes requests to the fastest responding server.
   • Applicable scenarios : For applications that require real-time or fast response, such as online games or voice communications.

8. health examination

   • Policy : Periodically check the health of the server, if the server does not respond or returns an error, it is removed from the pool of active servers until it is determined to be healthy again.
   • Applicable scenarios : Suitable for any application that requires high availability.

Choosing the right strategy based on your application type, server specifications, and expected traffic patterns is key. Many modern load balancers support these policies and allow you to dynamically switch policies based on real-time traffic patterns.

6.4 Also determine the protocol, encoding, how to compile it, and from what point of view to consider such encoding, it refers to the core implementation process, the call chain you have already said

6.4.1 In the HTTP protocol, request packets and response packets can be distinguished in the following ways:

.The format of the starting line :

• The initial line of the request message is a request line , and its format is: <HTTP方法> <请求URI> <HTTP版本>. For example:

  GET /index.html HTTP/1.1
  

• The initial line of the response message is a status line , and its format is: <HTTP版本> <状态码> <状态描述>. For example:

  HTTP/1.1 200 OK
  

In summary, the most direct and reliable way is to look at the start line. The format of the request line and status line is unique, and they can tell you exactly whether the message is a request or a response.

6.4.2 Select the HTTP protocol as the upper layer application protocol of netty

1 Version 1: The HTTP protocol is selected as our application layer protocol. For sending requests, there are request line, request header, request line and request body. The request header contains a content-type field, which contains a specific sequence Protocols, such as the commonly used web server serialization method application/json, there is also a content-length field, in order to distinguish the packet type, that is, to let the server know whether a call request or a call response, we can use the http request line and status line to do distinguish. In addition, using the blank line and content-length field of the http request can prevent sticky packets and unpacking problems.

6.4.3 Customize the MRF protocol (can brag with the interviewer)

The functions of these fields are shown in the figure. Compared with the HTTP protocol, many similarities are found.

6.4.3 Do internal rpc calls of general services need to go through the api-gateway?

Generally speaking, the internal calls of services (that is, direct communication between microservices) do not need to go through the API gateway (so it is generally better to be concise when explaining the call chain). The main purpose of an API gateway is to serve as an access point between the system and external consumers, processing requests into the system. This helps to focus on certain cross-cutting concerns such as request routing, API composition, throttling, authentication and authorization, etc.

When internal services need to communicate with each other, they typically make direct service-to-service calls, possibly through service discovery mechanisms to find and locate other services.

However, in some architectures or specific scenarios, there may be several reasons for internal service calls to pass through the API gateway:

1. Unified ingress and egress policies : Some organizations may want all inbound and outbound communications to go through the API Gateway to facilitate traffic monitoring, logging, or other unified policies.

2. Enhanced security : API gateways may provide additional layers of security, such as additional validation or filtering for certain sensitive operations.

3. Composite APIs : If a service needs to aggregate data from multiple other services, an API gateway may provide API composition capabilities so that services can fetch all necessary data in a single request.

4. Transformation or adaptation : API gateways may provide transformation capabilities, such as converting an old version of an API call to a new version.

But this is not the norm. Relying too much on an API gateway can cause the gateway to become a single point of failure or a performance bottleneck. For communication between internal services, it is more common to use a service mesh (such as Istio or Linkerd) to handle communication between services, providing functions such as load balancing, failure recovery, metrics, and security, rather than relying on API gateways.

6.5 For example, if I want to call the methodA method of the server, how does the server know that the client wants to call this method?

6.5.1 Specific process

The client will define a class called RpcRequest, which defines all the attributes that need to call the remote method, including the interface name, method name, parameters and types required by the method, and also includes a request number (the request number here You can not say, because it is a bypass, which will affect the interviewer's hearing of the main answer), and then through the client's local stub (that is, the proxy), it will automatically create the corresponding RpcRequest object for us and fill in the attributes. The attributes here are Filled according to the specific method we call and the parameters provided by the consumer,

After the filling is completed, the proxy class will also help us serialize the actual request body according to the selected serializer. The serialized rpcRequest data will be placed in the request body, and then sent to the service provider through the network.

After getting the request, the production side will deserialize the request body first, and after getting the actual RpcRequest data, it will reflect according to the interface name, interface method and parameter list to get an executable method, and then use it.

6.5.2 Involves some specific details

In the RPC (Remote Procedure Call) system, when the client wants to call a method on the server, it needs to tell the server which method should be called in some way. To achieve this goal, the request sent by the client will usually contain some metadata to describe the required method and parameters. Here are some common ways to achieve this:

1. Method identifier :

   When a client sends a request, it may include a string or numeric ID in the request as an identifier for the method. For example, it might send a message like this:

   { "method": "methodA", "params": [...] }
   

   The corresponding java objects are as follows:

/**
 * 消费者向提供者发送的请求对象
 *
 * @author ziyang
 */
@Data
@NoArgsConstructor
@AllArgsConstructor
public class RpcRequest implements Serializable {
    
    

    /**
     * 请求号
     */
    private String requestId;
    /**
     * 待调用接口名称
     */
    private String interfaceName;
    /**
     * 待调用方法名称
     */
    private String methodName;
    /**
     * 调用方法的参数
     */
    private Object[] parameters;
    /**
     * 调用方法的参数类型
     */
    private Class<?>[] paramTypes;

    /**
     * 是否是心跳包
     */
    private Boolean heartBeat;

}

2. Serialization and deserialization (related to specific protocols) :

   In addition to the method identifier, the client also needs to serialize the parameters of the method into a format that can be transmitted over the network (such as JSON, protobuf, etc.). After receiving the request, the server will deserialize these parameters and call the corresponding method according to the provided method identifier.

3. reflection :

   After receiving the request and deserializing it, the server may use reflection to find and call the corresponding method, because the client has sent all the parameters and their types required to call the method before, so the method can be called successfully. That's why in some RPC frameworks, you'll see the Java reflection API used to find and call methods based on their name.
   The following is a reflection process through the RpcRequest parameter on the server side:

  private Object invokeTargetMethod(RpcRequest rpcRequest, Object service) {
    
    
        Object result;
        try {
    
    
            Method method = service.getClass().getMethod(rpcRequest.getMethodName(), rpcRequest.getParamTypes());
            result = method.invoke(service, rpcRequest.getParameters());
            logger.info("服务:{} 成功调用方法:{}", rpcRequest.getInterfaceName(), rpcRequest.getMethodName());
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
    
    
            return RpcResponse.fail(ResponseCode.METHOD_NOT_FOUND, rpcRequest.getRequestId());
        }
        return result;
    }

In summary, in order to call a specific method on the server, the client needs to provide enough information to clearly specify which method and the required parameters. The server uses this information to determine which method should be called and which parameters to pass.

6.5.3 In the following piece of code, HelloService helloService = rpcClientProxy.getProxy(HelloService.class); the function of passing in HelloService.class

package top.guoziyang.rpc.transport;

/**
 * RPC客户端动态代理
 *
 * @author ziyang
 */
public class RpcClientProxy implements InvocationHandler {
    
    

    private static final Logger logger = LoggerFactory.getLogger(RpcClientProxy.class);

    private final RpcClient client;

    public RpcClientProxy(RpcClient client) {
    
    
        this.client = client;
    }

    @SuppressWarnings("unchecked")
    public <T> T getProxy(Class<T> clazz) {
    
    
        return (T) Proxy.newProxyInstance(clazz.getClassLoader(), new Class<?>[]{
    
    clazz}, this);
    }

    @SuppressWarnings("unchecked")
    @Override
    public Object invoke(Object proxy, Method method, Object[] args) {
    
    
        logger.info("调用方法: {}#{}", method.getDeclaringClass().getName(), method.getName());
        RpcRequest rpcRequest = new RpcRequest(UUID.randomUUID().toString(), method.getDeclaringClass().getName(),
                method.getName(), args, method.getParameterTypes(), false);
        RpcResponse rpcResponse = null;
        if (client instanceof NettyClient) {
    
    
            try {
    
    
                CompletableFuture<RpcResponse> completableFuture = (CompletableFuture<RpcResponse>) client.sendRequest(rpcRequest);
                rpcResponse = completableFuture.get();
            } catch (Exception e) {
    
    
                logger.error("方法调用请求发送失败", e);
                return null;
            }
        }
        if (client instanceof SocketClient) {
    
    
            rpcResponse = (RpcResponse) client.sendRequest(rpcRequest);
        }
        RpcMessageChecker.check(rpcRequest, rpcResponse);
        return rpcResponse.getData();
    }
}

package top.guoziyang.test;

/**
 * 测试用Netty消费者
 *
 * @author ziyang
 */
public class NettyTestClient {
    
    

    public static void main(String[] args) {
    
    
        RpcClient client = new NettyClient(CommonSerializer.PROTOBUF_SERIALIZER);
        RpcClientProxy rpcClientProxy = new RpcClientProxy(client);
        HelloService helloService = rpcClientProxy.getProxy(HelloService.class);
        HelloObject object = new HelloObject(12, "This is a message");
        String res = helloService.hello(object);
        System.out.println(res);
        ByeService byeService = rpcClientProxy.getProxy(ByeService.class);
        System.out.println(byeService.bye("Netty"));
    }

}

Analysis:
Yes, your understanding is correct.

In this code, RpcClientProxythe class provides a getProxymethod to create a dynamic proxy object for the specified interface. RpcClientProxyThis dynamic proxy object will be proxied to the method of when the method is called invoke.

When you call:

HelloService helloService = rpcClientProxy.getProxy(HelloService.class);

Here HelloService.classis passed into getProxythe method, thus creating a HelloServiceproxy object using Java's dynamic proxy mechanism. This means that when you call any method on this proxy object (such as helloService.hello(object)), it will actually be proxied to the method RpcClientProxyof invoke.

In invokethe method, you can see that it actually creates an RpcRequestobject, which contains the method name, parameters and other information, and then sends this request to the server through the RPC client. This is a typical RPC client implementation, which converts local method calls into remote method calls through proxy mode.

So, what you said " HelloService.classthe role of passing in should only be to create proxy objects" is correct. The purpose of this is to enable the client's code to make remote method calls in a way that looks like a local call, providing a good abstraction.

6.6 Assuming that the provider of the service wants to go offline now, how to make the service caller aware?

When the service provider (Service Provider) wants to go offline, in order to ensure the availability of the service and avoid service call failure, the following strategies are usually adopted to let the service consumer (Service Consumer) or the caller perceive:

1. Use the service registration and discovery mechanism (involving the interaction mechanism of the registration center, consumers and providers) :

   • In many RPC and microservice frameworks, service registration and discovery mechanisms are adopted, such as Zookeeper, Consul, Eureka, etc.
   • When a service provider intends to go offline, it first revokes or goes offline its service from the service registry.
   • The service consumer regularly pulls the service list from the service registry, so that when the provider goes offline, the consumer will perceive this change and no longer send requests to the provider.

2. Graceful shutdown (service provider) :

   • Before the service provider shuts down, it first stops receiving new requests, but continues to process already received requests.
   • Once all accepted requests have been processed, completely shut down the service. This ensures that no requests are interrupted midway.

3. Use a load balancer or API gateway (use a third-party gateway to block traffic) :

   • If you use a load balancer or API gateway to route requests, when a service provider wants to go offline, first remove the provider from the load balancer or API gateway.
   • In this way, even if the service provider is still running, the consumer's request will not be routed to the provider.

4. Health check (mechanism of registry) :

   • A service provider usually provides a health check interface to report its health status.
   • When a service provider is scheduled to go offline, the health check response can be modified to indicate that it is no longer healthy or is about to go offline.
   • The service consumer or load balancer periodically checks the health status of the service, and when it senses the change, it stops sending requests to the provider.

5. Notifications and alerts (expected maintenance or downtime) :

   • When a service provider is scheduled to go offline, all known service consumers can be notified manually.
   • This typically applies to projected maintenance or downtime, ensuring consumers have sufficient time to respond or backup plans.

6. Use fuse and retry mechanism (fuse retry will cause redundant requests) :

   • Circuit breakers can detect consecutive request failures and automatically cut off requests to providers, avoiding sending requests to services that have gone offline.
   • The retry mechanism tries other available service providers when a request fails.

Combined with the above strategies, it can be ensured that when the service provider goes offline, the service consumer can perceive and deal with it in time to avoid service interruption or a large number of request failures.

6.7 Assuming that it takes one second for the server to process a request, and I turn off my machine in less than a second, then the user will report an error. How to deal with this so that it does not report an error, because the error will negatively impact business

Ensuring the user doesn't see an error if the client (the user's device or app) suddenly shuts down or disconnects while waiting for a response from the server can be challenging. Here are some suggested ways to deal with it:

1. Server redundancy and failover :

   • Deploy multiple instances on the server side. When an instance fails, the traffic can be automatically switched to another healthy instance.
   • Use a load balancer to distribute requests while checking the health status of each service instance.

2. Client strategy :

   • Delayed error display : The client can add a longer timeout period. During this time, if it does not receive a response from the server, it can be considered that there is a problem with the server. But during this time, the user interface can display a prompt of "processing" or "wait a moment".
   • Retry mechanism : After receiving an error response or timeout, the client can try to send the request again, but be careful not to retry infinitely to avoid resource exhaustion or unnecessary network traffic.
   • Local caching and backup : For certain predictable requests, the client can cache previous results and return these cached data when the server is unresponsive.

3. Use an offline strategy :

   • For some applications, you can consider using an offline strategy, that is, if the request cannot be processed immediately, save the request locally and wait until the network is stable or the server is available before processing.

4. Friendly user interface :

   • Even when an error occurs, a friendly prompt should be displayed, such as: "The network is unstable, please try again later" or "We are processing your request, please wait", etc., and avoid directly displaying technical error messages.

5. Background processing and notifications :

   • For requests that do not require an immediate response, the client can return immediately after submitting the request, while the actual processing is performed in the background of the server. When finished, tell the client via notification or other means.

6. Data Consistency :

   • Ensure that the data processing logic of the server is idempotent, so that even if the client sends multiple requests, it will not affect the consistency of the data.

7. Use a message queue or event-driven model :

   • The request sent by the client is first stored in the message queue and processed asynchronously by the server. Clients can poll or wait for notifications for results.

When designing the system, we should start from the perspective of user experience to minimize the display of errors, but at the same time ensure the stability of the system and the accuracy of the data.