Internet products need to be developed and launched quickly and iteratively, but also to ensure quality, to ensure that the newly launched system can quickly control the impact once a problem occurs, and it is necessary to design a grayscale release system.
The role of the gray release system is to direct the user's traffic to the newly launched system according to the configuration to quickly verify the new function, and once a problem occurs, it can be repaired immediately. Simply put, it is a set of A/ B Test system.
Gray release is allowed to go online with bugs, as long as the bug is not fatal, of course, if the bug is unknown, if you know it, you will quickly change it
(the article has super benefits)
Layered architecture model
Design of Simple Grayscale Publishing System
The gray simple architecture is shown in the figure above, and the necessary components are as follows:
1. The configuration platform of the strategy, storing the gray strategy
2. The execution program of the gray function
3. The registration center, the registered service carries ip/Port/name/ version
With the above three components, it can be regarded as a complete grayscale platform
Grayscale strategy
Grayscale must have a grayscale strategy. The common methods of grayscale strategy are as follows:
1. Traffic segmentation based on Request Header 2. Traffic segmentation
based on Cookie 3. Traffic segmentation
based on request parameters
For example: take the modulus according to the user uid carried in the request, the gray scale is one percent, then the uid modulus range is 100, the modulus is 0 to access the new version of the service, and the modulus is 1 to 99 to access the old version of the service.
Gray release strategies are divided into two categories, single strategy and combined strategy
Single strategy: For example, take the modulus according to the user's uid, token, and ip
Combination strategy: multiple services are grayed out at the same time. For example, I have three services A/B/C, and I need to gray level A and C at the same time, but B does not need gray level. At this time, a tag field is needed. The specific implementation is Detailed below
Grayscale release specific execution control
In the above simple grayscale publishing system architecture, we learned that grayscale publishing services are divided into upstream and downstream services. Upstream services are specific programs that execute grayscale strategies. This service can be nginx or microservice architecture. The gateway layer/business logic layer, let’s analyze how different upstream services are implemented.
Nginx
If the upstream service is nginx, then nginx needs to extend nginx through Lua to implement the configuration and forwarding of the grayscale strategy, because nginx itself does not have the implementation of the grayscale strategy
The implementation of the gray-scale strategy is realized through the lua extension, but the problem is coming again. Nginx itself does not have the gray-scale strategy for receiving the configuration management platform. What should I do at this time?
solution:
Deploy Agent locally (you need to develop it yourself), receive the gray policy issued by the service configuration management platform, update the nginx configuration, and gracefully restart the Nginx service
Gateway layer/business logic layer/data access layer
It only needs to integrate the client SDK of the configuration management platform, receive the gray policy issued by the service configuration management platform, and execute the gray policy through the integrated SDK.
Publish complex scenes in grayscale
Let’s take an example of two slightly more complicated grayscale publishing scenarios. The grayscale strategy assumes that the uid modulates one percent of the grayscale users. Let’s see how it can be implemented.
Scenario 1: Invoke multiple services on the chain at the same time
The function upgrade involves multiple service changes. The gateway layer and data access layer are gray-scale, and the business logic layer remains unchanged. How should the gray-scale be performed at this time?
solution:
After the request of the new version of the gateway layer, all are tagged with tag T, and forwarded according to tag T in the business logic layer, all requests to tag T are forwarded to the new version of the data access layer service, and requests without tag T are all forwarded to the old version of data access Layer up.
Scenario 2: Grayscale service involving data
When it comes to the gray-scale service of data, you will definitely use the database. When you use the database, the table fields before and after you use the database are inconsistent. My old version has three fields A/B/C, and the new version is A/B/ C/D four fields. At this time, the new version of gray level cannot be modified to the old version of the database. At this time, you need to copy the data to do this. The
database does not actually have the concept of gray level. At this time, we can only copy the data again. Read and write one copy, because at this time your writing must be full (double writing). It cannot be said that 90% of the data is written to the old version and 10% of the data is written to the new version, because at this time you will It is found that the data in the two databases is not full.
There will be data loss during the offline full copy of data. At this time, the business logic layer needs to write a copy of data to MQ. After the data synchronization is completed, the new version of the data access layer will write the MQ data to the new version. In DB, to achieve data consistency, this is also the main purpose of introducing MQ.
During the gray-scale process, it is necessary to compare the data of the two databases to observe whether the data is consistent. In this way, whether it is gray failure, abandoning the new version of DB, or gray successfully switching to the new version of DB, data will not be lost.
Reader benefits
Thank you for seeing here!
I have compiled a lot of 2021 latest Java interview questions (including answers) and Java study notes here, as shown below
The answers to the above interview questions are organized into document notes. As well as interviews also compiled some information on some of the manufacturers & interview Zhenti latest 2021 collection (both documenting a small portion of the screenshot) free for everyone to share, in need can click to enter signal: CSDN! Free to share~
If you like this article, please forward it and like it.
Remember to follow me!
