Microservice Architecture - Graceful Downtime Solution

1 Introduction

        The graceful application shutdown in the microservice architecture mainly refers to the planned and smooth exit of application instances (that is, no accidents that need to be dealt with). Application server shutdowns are mainly divided into two categories: active shutdowns and passive shutdowns, among which active shutdowns and most passive shutdowns can achieve graceful shutdowns. If the application does not do graceful shutdown, it will bring the following situations:

• Data loss: In-memory data has not been persisted to disk

• File corruption: The file being written is not updated, resulting in file corruption

• Request Lost: A queued request was lost waiting to be processed

• Missing Response: Successful transaction has not had time to respond

• Transaction Interruption: A transaction that is being processed to an intermediate state is forcibly interrupted

• The service is not offline: the upstream service will continue to send consumption requests to the downstream service

        The goal of the elegant upgrade of our microservices is to avoid the above situations, so as to avoid the workload of manual intervention and improve the service reliability of the microservice architecture.

2 usage scenarios

Graceful shutdown can solve the following scenarios:

• KILL PID

• Application unexpectedly exits automatically (System.exit(n))

• Stop the application using a script command

Graceful shutdown cannot solve the following scenarios:

• Sudden power failure

• physical destruction of the machine

• KILL-9 PID or taskkill /f /pid

3 ShutdownHook

        The graceful shutdown of Java is usually realized by registering the ShutdownHook (hook) of the JDK. When the system receives the exit command, it first marks the system as being in the exit state and no longer receives new messages, then processes the backlog of messages, and finally calls resource recovery. The interface destroys the resource, and finally each thread exits execution. A simple demo case is as follows (simple version):

/**
 * 优雅停机处理方式
 * 
 * @author lry
 **/
public class Main{

    /**
     * 启动应用
     **/
    public void start(){
        // 第一步：启动应用服务……

        // 第二步：注册JDK钩子
        Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
            @Override
            public void run() {
                System.out.println("The hook running...");
                //第三步：调用停机处理
                stop();
            }
        }));
    }

    /**
     * 停止应用
     **/
    public void stop(){
        // 停止应用前停机处理(如：注销服务、标记不接受请求等)
    }

}

Timeout control
        usually requires a timeout control mechanism for graceful exit. If the resource recovery and other operations before the exit are still not completed after the timeout time is reached, the shutdown script will directly call KILL -9 PID to force the exit, otherwise it may wait for a long time.

4 Graceful shutdown of microservices

        There is no unified solution for the graceful shutdown of microservices, as long as the core idea is grasped for design:
        drainage → baffle → waiting for shutdown

        But in a microservice architecture, we can design graceful downtime mechanisms for microservices by adhering to the following suggested rules:

• All microservice applications should support graceful downtime

• Priority to cancel the service instance registered in the registry

• Access Point Flag Denial of Service for Service to Be Down

• Upstream services support failover of services rejected due to graceful downtime

• Provide appropriate shutdown interface according to specific business

    The graceful shutdown of microservice applications is mainly divided into two types according to the role of its users:

• Microservice business application graceful shutdown design:

  

  Microservice business application graceful shutdown design

• Microservice gateway application graceful shutdown design:

  

  Microservice gateway application graceful shutdown design

    The graceful downtime of the remaining layers of equipment can be derived from the above two types of solutions, such as:

• When the entire backend architecture is upgraded, you can switch directly from DNS or Nginx

• Nginx layer upgrade, you can switch directly from DNS

5 Use Cases

        Among the open source products in the industry, many products use the JDK hook method to achieve graceful shutdown, such as the following products:

• Netty

• DUBBO

Welcome to pay attention to the personal technical public account to view the original text:

ID：i-micro-tech

QQ microservice infrastructure exchange group (191958521):

Thank you sweet potato for your support!