foreword
At present, the microservice architecture has been implemented in many companies. The following diagram briefly summarizes the commonly used components in the microservice architecture design. It cannot be said that microservices have been used for several years. As a result, there is no overall understanding of the microservice architecture. A programmer who only knows how to move bricks is not a good coder!
Traffic entry Nginx
As can be seen in the figure above, Nginx, as the traffic entrance of the entire architecture, can be understood as an external gateway, which undertakes functions such as routing and forwarding of requests, load balancing, and dynamic and static separation. As a core entry point, Nginx must adopt multi-node deployment, and at the same time achieve high availability through keepalived, so as to ensure the high availability of the entire platform.
gateway
The gateway is another core component behind Nginx. It undertakes a series of functions such as request authentication, route forwarding, protocol conversion, and traffic monitoring. The gateway in the above figure uses spring Cloud Gateway to realize the function of the business gateway. In the gateway selection, we have other options, such as Zuul1, Zuul2, Kong, etc., these solutions have their own advantages and limitations, we can choose which one to choose according to their characteristics. For an in-depth understanding of the gateway, you can refer to the previous series of articles about the gateway , so I won’t go into details here.
In the figure above, there are jwt and OAuth2 under the Spring Cloud Gateway. In fact, these two are token-based authentication and authentication. In general Internet projects, the login module supports WeChat or qq login, which is the authorized login using OAuth2. If you want to learn more about the details of Oauth2, you can refer to the series of articles such as the previous Oauth2.0 client server example .
business component
As can be seen from the above architecture diagram, after the gateway is our business component, which can be understood as the microservice after splitting, such as account service, order service, invoice service, cashier service, etc. common on e-commerce platforms . Feign is used to make http calls between service components, and Feign integrates Ribbon to achieve client-side load balancing. The specific service domain division and the setting of the service boundary context are additional knowledge. If you want to do a good job of service division, you can learn more about DDD domain-driven design.
Service Registry
Regardless of whether it is an SOA based on Dubbo or a microservice architecture based on Spring Cloud splitting, a service registry is necessary. We register all service components to the registry to realize dynamic invocation of services. The common ones that can realize the registration center function are Zookeeper, Eureka, Nacos, and Zookeeper is widely used in Dubbo. At present, the company's service microservice architecture is based on Eureka, and Eureka does not seem to be maintained at present. Generally, new platforms are recommended to directly integrate Nacos. In addition to being used as a registration center, Nacos can also be used as a distributed configuration center, which is better than Sping Cloud Config.
Caching and Distributed Locks
In the lower left corner of the figure, we can see the Redis component. We can use Redis as a cache, and cache some hot data with slow query and high usage rate, which can quickly improve the interface response time. At the same time, a major usage scenario of redis in microservices is distributed locks. Traditional Sychronized and display Lock locks obviously cannot solve the problem of distributed concurrency.
In order to ensure the high availability of Redis, sentinel deployment can be used instead of three redis nodes, one master and two slaves, and three sentinel nodes are deployed at the same time to achieve failover and avoid single-point problems. If the amount of data stored in Redis is large, reaching To eliminate the performance bottleneck of single-node Redis, we can also use Redis cluster mode to implement distributed storage. For detailed information about Redis Sentinel, please refer to the previous series of articles such as Detailed Explanation of Redis Sentinel Architecture Principles (1) .
data persistence layer
Regardless of single service or microservice, the data persistence layer is necessary. We choose mysql, which is often used in Internet projects, as the DB. In order to ensure the read-write efficiency and high availability of the service, we separate the master-slave mode and realize the read-write separation at the same time. , to ensure the read and write performance of mysql.
With the growth of business volume, after the data volume of a single table reaches the performance bottleneck, we need to use sub-database and sub-table to split the database table horizontally and vertically. How to split reasonably and select technology , These are closely related to the existing table structure design of the project. Subsequent scalability must be considered. It cannot be dismantled for a short period of time. The split server is deployed. Of course, the magnitude of data in general enterprises cannot reach that magnitude.
structured data storage
The mysql storage data mentioned above is all unstructured data storage. Our projects often need to store some structured data, such as storing JSON strings. It is obviously inappropriate to store it through mysql in this scenario.
Generally, we will use Elasticsearch or MangoDB for storage. If you need retrieval function in your business, it is more recommended to use Elasticsearch. Elasticsearch supports DSL, has relatively rich query and retrieval functions, and can even realize GIS spatial retrieval functions.
message middleware
As mentioned earlier, in the microservice architecture, synchronous calls between services are realized through Feign, and the asynchronous decoupling between services must be realized through MQ. Although we can implement asynchronous calls through multiple threads, this kind of asynchronous calls does not support persistence and may cause message loss, so RabbitMq or RocketMq is generally integrated.
log collection
In the microservice architecture, through a component, for example, the order service is deployed in a multi-node distributed manner, and the logs of each node are stored locally on the node. If we want to query the logs, do we have to log in to each node to find the corresponding log information? This kind of viewing log is definitely not acceptable. Therefore, ELK is generally introduced for log collection and visual display query.
- Logstash is used for log collection. Usually, a Filebeat is added in front of Logstash. Filebeat collects logs, and Logstash does data conversion.
- Elasticsearch does data storage and generates index data for easy retrieval by Kibana.
- Kibana does data display and query retrieval functions. We can quickly query the desired log information by searching keywords.
task scheduling center
Timing functions are often used in projects. In single applications, we can use the Schedule that comes with sping, or use Quartz. In distributed applications, we need to integrate distributed timers, such as Quartz (Quartz cooperates with database tables It also supports distributed timing tasks), as well as Elastic-Job, XXL-JOB, etc.
Elastic-job Dangdang is an elastic distributed task scheduling system based on quartz secondary development. It has rich and powerful functions. It uses zookeeper to realize distributed coordination, high availability and fragmentation of tasks. Elastic-Job is a distributed scheduling solution, which is open sourced by Dangdang.com. It consists of two independent sub-projects, Elastic-Job-Lite and Elastic-Job-Cloud. Using Elastic-Job can quickly realize distributed task scheduling. .
XXL-JOB is a distributed task scheduling platform (XXL is the initials of the author Xu Xueli's name in pinyin), and its core design goals are rapid development, easy learning, lightweight, and easy expansion. The dispatching behavior is abstracted into a public platform of "dispatching center", and the platform itself does not undertake business logic, and the "dispatching center" is responsible for initiating dispatching requests. The tasks are abstracted into scattered JobHandlers, which are managed by the "executor", and the "executor" is responsible for receiving scheduling requests and executing the business logic in the corresponding JobHandler. Therefore, the two parts of "scheduling" and "task" can be decoupled from each other to improve the overall stability and scalability of the system.
distributed object storage
Projects often have file upload functions, such as pictures, audio and video. In a distributed architecture, it is obviously impossible for us to store files on node servers. At this time, we need to introduce distributed file storage. Common solutions include MinIo, Ali's OSS (paid), Ali FastDFS, etc.
MinIO is a high-performance, distributed object storage system developed based on the Go language. The client supports Java, Net, Python, Javacript, Golang languages.
FastDFS is an open source lightweight distributed file system. It manages files. Its functions include: file storage, file synchronization, file access (file upload, file download), etc., which solves the problem of large-capacity storage and storage. It is especially suitable for online services with files as the carrier, such as photo album websites, video websites and so on.