[Microservice] A picture to understand the design of microservice architecture

1 Introduction

At present, the microservice architecture has been implemented in many companies. The following diagram briefly summarizes the commonly used components in the microservice architecture design. It cannot be said that microservices have been used for several years. As a result, there is no overall understanding of the microservice architecture. A programmer who only knows how to move bricks is not a good coder!

2. Traffic entrance Nginx

As can be seen in the figure above, Nginx, as the traffic entrance of the entire architecture, can be understood as an external gateway, which undertakes functions such as routing and forwarding of requests , load balancing , and dynamic and static separation . As a core entry point, Nginx must adopt multi-node deployment, and at the same time keepalivedachieve high availability through to ensure the high availability of the entire platform.

3. Gateway

Gateway is another core component behind Nginx. It undertakes a series of functions such as request authentication , routing forwarding , protocol conversion , and traffic monitoring . The gateway in the above figure uses Spring Cloud Gateway to realize the function of the business gateway. In the gateway selection, we have other options, such as Zuul1, Zuul2, Kong, etc., these solutions have their own advantages and limitations, we can choose which one to choose according to their characteristics.

In the figure above, there are JWT and OAuth2 under Spring Cloud Gateway. In fact, these two are token-based authentication and authentication. In general Internet projects, the login module supports WeChat or QQ login, which is the authorized login using OAuth2.

4. Business components

As can be seen from the above architecture diagram, after the gateway is our business component, which can be understood as the microservice after splitting, such as account service , order service , invoice service , cashier service , etc. common on e-commerce platforms . Service components are called through Feign http, and Feign integrates Ribbon to achieve client-side load balancing. The specific service domain division and the setting of the service boundary context are additional knowledge. If you want to do a good job of service division, you can learn more about DDD domain-driven design.

5. Service Registration Center

Whether it is an SOA based on Dubbo or a microservice architecture based on Spring Cloud splitting, a service registry is necessary. We register all service components to the registry to realize dynamic invocation of services. Common ones that can implement the registration center function are Zookeeper, Eureka, and Nacos. Zookeeper is widely used in Dubbo. At present, the company's service microservice architecture is based on Eureka, and Eureka does not seem to be maintained at present. Generally, new platforms are recommended to directly integrate Nacos. Nacos can be used not only as a registration center, but also as a distributed configuration center, which is better than Sping Cloud Config.

6. Cache and distributed lock

In the lower left corner of the figure, we can see the Redis component. We can use Redis as a cache, and cache some hot data with slow query and high usage rate, which can quickly improve the interface response time. At the same time, one of the major usage scenarios of Redis in microservices is distributed locks. Traditional Sychronized and display Lock locks obviously cannot solve the problem of distributed concurrency.

In order to ensure the high availability of Redis, sentinel deployment can be used instead of three Redis nodes, one master and two slaves, and three sentinel nodes are deployed at the same time to achieve failover and avoid single-point problems. If the amount of data stored in Redis is large, reaching To eliminate the performance bottleneck of single-node Redis, we can also use Redis cluster mode to implement distributed storage.

7. Data persistence layer

Regardless of single service or microservice, the data persistence layer is necessary. We choose MySQL, which is often used in Internet projects, as the DB. In order to ensure the efficiency of reading and writing services and high availability, we use the master-slave separation mode and realize the separation of reading and writing at the same time . , to ensure the read and write performance of MySQL.

With the growth of business volume, after the data volume of a single table reaches the performance bottleneck, we need to use sub-database and sub-table to split the database table horizontally and vertically. How to split reasonably and select technology , These are closely related to the existing table structure design of the project. Subsequent scalability must be considered. It cannot be dismantled for a short period of time. The split server is deployed. Of course, the magnitude of data in general enterprises cannot reach that magnitude.

8. Structured data storage

MySQL is better at storing relational data. There are scenarios where structured data needs to be stored in the project, such as storing JSON strings. It is obviously inappropriate to use MySQL to store such scenarios. Generally, we will use Elasticsearch or MongoDB for storage. If retrieval function is required in the business, Elasticsearch is recommended. Elasticsearch supports DSL, has relatively rich query and retrieval functions, and can even realize GIS spatial retrieval functions.

9. Message middleware

As mentioned earlier, in the microservice architecture, synchronous calls between services are realized through Feign, and the asynchronous decoupling between services must be realized through MQ. Although we can implement asynchronous calls through multiple threads, such asynchronous calls do not support persistence and may cause message loss, so RabbitMQ or RocketMQ is generally integrated.

10. Log Collection

In the microservice architecture, through a component, for example, the order service is deployed in a multi-node distributed manner, and the logs of each node are stored locally on the node. If we want to query the logs, do we have to log in to each node to find the corresponding log information? This kind of viewing log is definitely not acceptable. Therefore, ELK is generally introduced for log collection and visual display query.

• Logstash is used for log collection. Usually, a Filebeat is added before Logstash. Filebeat collects logs, and Logstash does data conversion.
• Elasticsearch does data storage and generates index data for easy retrieval by Kibana.
• Kibana does data display and query retrieval functions. We can quickly query the desired log information by searching keywords.

11. Task scheduling center

The timing function is often used in the project. In the single application, we can use the Schedule that comes with Spring, or use Quartz. In the distributed application, we need to integrate the distributed timer, such as Quartz (Quartz cooperates with the database table It also supports distributed timing tasks), as well as Elastic-Job, XXL-JOB, etc.

• Elastic-Job is an elastic distributed task scheduling system developed by Dangdang.com based on Quartz. It has rich and powerful functions. It uses Zookeeper to realize distributed coordination, high availability and fragmentation of tasks. Elastic-Job is a distributed scheduling solution, which is open sourced by Dangdang.com. It consists of two independent sub-projectsElastic-Job-LiteandElastic-Job-Cloud. Using Elastic-Job can quickly realize distributed task scheduling.

• XXL-JOB is a distributed task scheduling platform (XXL is the initials of the author Xu Xueli's name in pinyin), and its core design goals are rapid development, easy learning, lightweight, and easy expansion. The dispatching behavior is abstracted into a public platform of "dispatching center", and the platform itself does not undertake business logic, and the "dispatching center" is responsible for initiating dispatching requests. The tasks are abstracted into scattered JobHandlers, which are managed by the "executor", which is responsible for receiving scheduling requests and executing the business logic in the corresponding JobHandler. Therefore, the two parts of "scheduling" and "task" can be decoupled from each other to improve the overall stability and scalability of the system.

12. Distributed object storage

Projects often have file upload functions, such as pictures, audio and video. In a distributed architecture, it is obviously impossible for us to store files on node servers. At this time, we need to introduce distributed file storage. Common solutions include MinIO, Ali's OSS (paid), Ali FastDFS, etc.

• MinIO is a high-performance, distributed object storage system developed based on the Go language . The client supports Java, Net, Python, Javacript, Golang languages.

• FastDFS is an open source lightweight distributed file system . It manages files. Its functions include: file storage, file synchronization, file access (file upload, file download), etc., which solves the problem of large-capacity storage and storage. It is especially suitable for online services with files as the carrier , such as photo album websites, video websites and so on.