How to achieve elegant publishing in the microservice distributed architecture?

This article mainly explains how to ensure that the API does not report errors during the service version upgrade and deployment process in the spring cloud microservice architecture.

background:

1. The current service is unavailable due to the release of the version, and a fuse message is returned, which affects the user experience, such as "The system is busy, please try again later"

2. The request link is interrupted halfway through, and the data in the first half of a request is correct, but the data in the second half is incorrect, resulting in data confusion

3. For problems caused by situation 2, R&D colleagues need to spend a lot of time locating the problem and repairing the data

Implementation idea:

1. Remove the release service from the eureka registration center, and release the service after the service has no traffic

2. Trigger other services to refresh the service list, stop the request for the release service, and let the release service have no traffic

3. After the release service is started, trigger other services to refresh the service list, request the release service, and let the release service share the traffic

The following is a simple demo example, we want to release the version of B1

1. Eureka's service discovery mechanism

Let's first understand eureka's service discovery mechanism

The component that finally initiates the request in the microservice is the Ribbon (with its own service list). If the version service is released according to eureka's default service discovery mechanism, it will take 0 to 270 seconds, and A1 or A2 will also send the request To B1, B1 is unable to process the request during the period from shutdown to restart, so it will trigger the internal fuse mechanism of A1 or A2 (such as "the system is busy, please try again later")

For this kind of fuse caused by the release, we can actually avoid it. Let's see how I do it

2. Elegant release

1. Remove B1 from eureka, and release B1 after the service has no traffic

Although B1 here has been removed from eureka, B1 is still running normally. If you use the ip+port method to directly request the API of B1, it can be processed normally.

Because B1 and B2 are the nodes of the B service cluster, they will automatically register to eureka after B1 is started, and distribute the traffic, and then do the same release process for B2 (the release of service A is also the same)

During the whole process, we are restarting the service without traffic, which has no effect on the entire microservice request link

2. How to log off the service

1) Disable readOnlyCacheMap

eureka.server.use-read-only-response-cache: false

If it is not disabled, normal active offline will be directly synchronized to readWriteCacheMap, but will not be synchronized to readOnlyCacheMap, there may be a difference of 30 seconds, there is still B1 in the service list pulled by the eureka client, and B1 cannot be removed immediately

2) Execute DiscoveryClient.shutdown()

Define an API to call DiscoveryClient.shutdown() to actively log off from eureka

And use MQ (I use rabbitmq here) to send a broadcast message to all other services, telling other services that this service is offline, refresh the Ribbon service list quickly, and don't send requests to this service anymore

Why refresh the service list of the eureka client here? Because Ribbon's service list data is the service list from the eureka client (see the previous figure)

3) Other offline (currently only MQ consumers need to be dealt with, if you have other needs to be dealt with in your project, you can also refer to it)

Although B1 no longer processes API requests, B1's MQ consumer is still consuming data. If the consumer stops processing data, B1 will cause problems. What should I do?

All MQ consumers of offline B1

4) After the service is started, notify other services to refresh the service list and quickly obtain traffic

This step is not necessary. You can also wait for the service to automatically synchronize the service list. In this case, there may be a certain delay. You need to wait to deploy the next node.

3. Application

Mainly call the /offline interface above (try to keep synchronous blocking, if it is changed to asynchronous non-blocking, the process may be killed before the service is offline, causing an exception)

Usage 1: Directly request http://localhost:8881/offline , you can kill the process and restart after the response is successful

Usage 2: Combined with jenkins deployment, add http://localhost:8881/offline to the shell script, execute the shell script during the jenkins build process, kill the process and restart after the build is complete

How about it? If you find it useful, don't hesitate to start it! ! !

Attachment: the code directory involved

github：https://github.com/897665787/springcloud-template

gitee：https://gitee.com/jq_di/springcloud-template

springcloud-template
└── template-eureka
     eureka.server.use-read-only-response-cache: false -- 禁用readOnlyCacheMap
└── template-framework
     └── deploy
          └── DeployController -- 部署相关接口（用于优雅发版）
          └── MQAutoRefresh -- MQ自动刷新注册列表(如果使用了MQ，可以利用MQ广播消息自动刷新)
          └── RefreshHandler -- 刷新处理器
          └── StartupApplicationRunner -- 服务启动成功执行