Microservices Evolution
1. The evolution process of microservices
version 1.0
A few years ago, Xiao Ming and Xiao Piao started an online supermarket together. Xiaoming is responsible for program development, and Xiaobiao is responsible for other matters. At that time, the Internet was not yet developed, and online supermarkets were still a blue ocean. As long as the function is implemented, you can make money at will. So their needs are very simple. They only need a website on the public network, and users can browse and purchase products on this website; in addition, they need a management background to manage products, users, and order data. .
Let’s sort out the feature list:
- website
- User registration and login functions
- Product showcase
- Place an order
- Management background
- user management
- Product management
- Order management
Due to the simple requirements, Xiao Ming moved his left and right hands in slow motion, and the website was ready. For security reasons, the management background is not integrated with the website. Xiao Ming's right and left hands are replayed in slow motion, and the management website is also ready. The overall architecture diagram is as follows:
Xiao Ming waved his hand, found a cloud service, deployed it, and the website was online. After being launched online, it received rave reviews and was loved by all kinds of fat houses. Xiao Mingxiao happily began to lie down and collect the money.
version 2.0
The good times did not last long, and within a few days, various online supermarkets sprang up, causing a strong impact on Xiao Ming Xiaobiao.
Under the pressure of competition, Xiaoming Xiaobiao decided to carry out some marketing methods:
-
Run promotions. For example, New Year’s Day discounts across the site, Spring Festival buy two get one free, Valentine’s Day dog food coupons, etc.
-
Expand channels and add mobile marketing. In addition to the website, mobile APPs, WeChat mini programs, etc. also need to be developed.
-
Precision marketing. Use historical data to analyze users and provide personalized services.
-
……
These activities require the support of program development. Xiao Ming recruited his classmate Xiao Hong to join the team. Xiaohong is responsible for data analysis and mobile terminal related development. Xiao Ming is responsible for the development of functions related to promotional activities.
Because of the urgent development tasks, Xiaoming and Xiaohong did not plan the structure of the entire system. They patted their heads casually and decided to put promotion management and data analysis in the management background, and build WeChat and mobile APP separately. After working all night for several days, the new functions and applications were basically completed. At this time, the architecture diagram is as follows:
There are many unreasonable aspects at this stage:
-
Websites and mobile applications have a lot of duplicate code with the same business logic.
-
Data is sometimes shared through the database, and sometimes transmitted through interface calls. The interface calling relationship is messy.
-
In order to provide interfaces for other applications, a single application gradually changes larger and larger, including a lot of logic that does not belong to it in the first place. Application boundaries are blurred and function attribution is confusing.
-
The management backend was initially designed with a low level of security. After adding functions related to data analysis and promotion management, performance bottlenecks appeared, affecting other applications.
-
The database table structure is relied upon by multiple applications and cannot be reconstructed and optimized.
-
All applications operate on a database, and a performance bottleneck occurs in the database. Especially when data analysis is running, database performance drops sharply.
-
Development, testing, deployment, and maintenance are becoming increasingly difficult. Even if only a small function is changed, the entire application needs to be released together. Sometimes some untested code is accidentally brought into the press conference, or after modifying a function, another unexpected error occurs. In order to reduce the impact of possible problems caused by release and the impact of online business suspension, all applications must be released at three or four in the morning. After release, in order to verify that the application is running normally, we have to keep an eye on the user peak period on the second day...
-
There is a phenomenon of blame-shifting in the team. There is often a long debate about which application some public functions should be built on. In the end, they either simply do their own thing, or put it anywhere but don't maintain it.
Although there are many problems, the results of this stage cannot be denied: the system was quickly built according to business changes. However, urgent and heavy tasks can easily cause people to fall into partial and short-term thinking and make compromised decisions . In this kind of structure, everyone only focuses on their own little piece of land, lacking an overall and long-term design. If things go on like this, system construction will become increasingly difficult, and may even fall into a cycle of constant overthrow and reconstruction.
version 3.0
Fortunately, Xiao Ming and Xiao Hong are good teenagers with pursuits and ideals. After realizing the problem, Xiao Ming and Xiao Hong freed up some energy from trivial business requirements, began to sort out the overall structure, and prepared to start reforming the problem.
To make a transformation, you first need to have enough energy and resources. If your demand side (business people, project managers, bosses, etc.) is so aggressively focused on the demand schedule that you can't allocate extra energy and resources, then you may not be able to do anything... …
In the world of programming, the most important thing is the ability to abstract . The process of microservice transformation is actually an abstract process. Xiao Ming and Xiao Hong organized the business logic of the online supermarket, abstracted common business capabilities, and created several public services:
-
user service
-
Goods and Services
-
Promotional services
-
order service
-
Data analysis service
Each application background only needs to obtain the required data from these services, thus eliminating a large amount of redundant code, leaving only a thin control layer and front end. The structure of this stage is as follows:
At this stage, the services are only separated, and the database is still shared, so some shortcomings of the chimney system still exist:
- The database becomes a performance bottleneck and is at risk of a single point of failure.
- Data management tends to be chaotic. Even if there is a good modular design at the beginning, as time goes by, there will always be a phenomenon where one service directly takes the data of another service from the database.
- The database table structure may be dependent on multiple services, which affects the whole system and is difficult to adjust.
If the shared database model is always maintained, the entire architecture will become more and more rigid and the meaning of the microservice architecture will be lost. Therefore, Xiao Ming and Xiao Hongyi split the database. All persistence layers are isolated from each other and are responsible for each service itself. In addition, in order to improve the real-time performance of the system, a message queue mechanism is added. The structure is as follows:
After complete separation, each service can use heterogeneous technologies. For example, the data analysis service can use the data warehouse as the persistence layer to efficiently do some statistical calculations; commodity services and promotional services are frequently accessed, so a caching mechanism is added.
Another way to abstract public logic is to make these public logic into a public framework library. This method can reduce the performance loss of service calls. However, the management cost of this method is very high, and it is difficult to ensure the consistency of all application versions.
Database splitting also has some problems and challenges: such as the need for cross-database cascading, the granularity of data query through services, etc. But these problems can be solved through reasonable design. Overall, database splitting has more pros than cons.
The microservice architecture also has a non-technical benefit. It makes the division of labor and responsibilities of the entire system clearer, and each person is dedicated to providing better services for others. In the era of monolithic applications, public business functions often do not have clear ownership. In the end, either they each did their own thing, and everyone re-implemented it again; or a random person (usually someone with stronger ability or more enthusiasm) made it into the application he was responsible for. In the latter case, in addition to being responsible for his own application, this person is also responsible for providing these public functions to others - and this function is originally no one is responsible for, just because he is more capable/ If you are more enthusiastic, you will take the blame for no reason (this situation is also praised as saying that the capable person has more trouble). As a result, no one was willing to provide public functions. As time goes on, people in the team gradually become independent and no longer care about the overall architecture design.
version 4.0
2. Microservice splitting
Principle of division
When we split microservices, we need to do so according to certain principles.
-
Based on business logic
- Businesses in the system are identified according to their scope of responsibilities , and those with the same responsibilities are divided into separate services.
-
Based on stability
- Sort the business modules in the system according to their stability . Divide the stable and infrequently modified services into one; divide the unstable and frequently modified services into an independent service. For example, log service and monitoring service are relatively stable services and can be grouped together.
-
Based on reliability
-
Similarly, sort the business modules in the system according to reliability. Core modules with higher reliability requirements are grouped together, and non-core modules with lower reliability requirements are grouped together.
The cleverness of this kind of splitting can well avoid the single disadvantage of a single piece of porridge being damaged by a single piece of poop. At the same time, high-availability solutions in the future can also save the cost of machines or bandwidth.
-
-
Based on high performance
- As above, prioritize the business modules in the system according to performance requirements. Separate modules with high performance requirements into a service, and put modules with low performance requirements together. For example, full-text search, product query and classification, and flash sales are high-performance core modules.
Based on the split principle of appeal, we can split the microservices for the Mulowowo project:
- Travel writing service
- Destination management
- travelling guideline
- Travel diary
- …
- Comment service
- Travel feature reviews
- Travel diary reviews
- Attraction reviews
- Travel agency reviews
- …
- User service
- User Personal Center
- User points related
- Blacklist/Whitelist
- Fans follow
- …
- Messaging service
- SMS notification
- E-mail notification
- Station message
- …
- Search service
- Strategy search
- Travel notes soso
- User search
- Attraction search
- …
Staffing
If we divide by business, according to the size of the granularity, there may be the following two types:
-
The first type is divided into three services: commodity, transaction and user;
-
The second type is divided into six services: commodity, order, payment, logistics, buyer, and seller.
3 VS 6 , what to do?
If your team has only 9 people, it makes sense to split into 3, and if you have 18 people, then 6 services make sense. Team members are introduced here to assist in the split.
When splitting encounters disputes, we generally add a splitting condition. Although it is not a necessary and sufficient condition, at least our answer will be closer to the truth.
In addition to business disputes, other divisions will also be controversial, such as how many personnel are required for an independent service?
Why is it said that three people are assigned one service (of course, the members are mainly back-end personnel)?
Assuming that there is only one person, asking for leave or getting sick will not work. One person will encounter a single point of problem, so it is unreasonable.
Assume there are two people, and finally have a backup, but after removing one, the pressure on the remaining one is still very high, which is unreasonable.
Suppose there are 3 people. If one person is removed, there are still 2 people. And the number 3 is a stable and magical number, which can be used to get twice the result with half the effort. Especially when it comes to technical discussions, three people are relatively comprehensive. If there are two people, they may each have their own opinions, with their own biases and blind spots.
So is this 3 a stable number?
Assuming that you are doing a rewrite job that involves changing the engine while starting the machine, then all three people may be stretched in the early stage. But in the later stages of the service, one may be enough for you.