Source code of this article: GitHub·click here || GitEE·click here
1. Basic concepts
1. Single service
All business services and application components are deployed on one service to save costs. This is a single-service structure, suitable for scenarios with low concurrency and single business.
2. Cluster mode
The business volume is gradually increasing and the concurrency is high. When a service is horizontally expanded to form a service group, the request pressure is distributed to different services for processing. Each service is called a node of the cluster, and this is the cluster service.
3. Distributed architecture
Distributed structure is divided into independent sub-services according to business functions, independent library tables, which can run independently, and communicate and interact between services, which bring benefits to reduce the coupling between businesses, facilitate development and maintenance, and expand horizontally , High reusability and so on.
4. Technical system
Service infrastructure: Dubbo framework, SpringCloud framework;
Containerized operation and maintenance: Docker, Kubernetes;
Data storage: relational MySQL, NoSQL database, OLAP engine;
Common components: Zookeeper coordination, MQ asynchronous, Redis cache;
Two, distributed framework
1. Dubbo framework
There are more and more vertical applications, and the interaction between applications is inevitable. The core business is extracted as an independent service, and a stable service center is gradually formed, so that front-end applications can more quickly respond to changing market demands. At this time, the Distributed Service Framework (RPC) for improving business reuse and integration is the key. The core capabilities of the Dubbo framework: interface-oriented remote method invocation, intelligent fault tolerance and load balancing, and automatic service registration and discovery.
2. SpringCloud framework
The most mature framework under the distributed architecture, SpringCloud is an ordered collection of a series of frameworks. It uses SpringBoot's development convenience to cleverly simplifies the development of distributed system infrastructure, such as service discovery and registration, configuration center, message bus, load balancing, circuit breakers, data monitoring, etc., all can be achieved with SpringBoot development style Key to start and deploy.
https://github.com/cicadasmile/spring-cloud-base
Core components
Registry: It has a mechanism for service discovery, service recording, query, and dynamic management. Commonly used registration centers, Zookeeper, Eureka, Consul, Nacos, etc.
Fuse degradation: Limiting the sudden high concurrency of traffic and overwhelming the system, so that this kind of messages flow and send at a relatively uniform speed, to achieve the purpose of protecting the relative stability of the system. Commonly used algorithm token buckets and funnels; common components Nginx, CDN, Hystrix, Sentinel, and control traffic through different nodes.
Service gateway: It is also a service in the entire architecture system. As the only entrance to the request, it is very similar to the appearance model. It handles all non-business functions at the gateway layer and provides customized APIs for the client. Commonly used components Zuul, Tyk, Kong.
https://github.com/cicadasmile/husky-spring-cloud
3. Business components
Message middleware: RocktMQ, Kafka, RabbitMQ, etc.;
Cache middleware: Redis, Eache, etc.;
Distributed transactions: Seata, Hmily, TCC-transaction, etc.;
3. Architecture details
1. Global ID strategy
The data generated by the business scenario needs a unique ID as the core identifier for process management. For example, common ones: UUID, snowflake algorithm, self-incrementing primary key, ID container, etc.
2. Interface idempotence
The characteristic of idempotent operation is that the impact of any number of executions is the same as that of one execution. In other words, requesting a certain resource once and multiple times will have the same effect. It is especially necessary to ensure the idempotence of operations in the scenarios of interface, retry, and compensation.
3. Cache processing
In the business system, the modules that are most prone to performance problems when querying, the amount of data faced by the query is large, and the filtering conditions are complex. Therefore, it is necessary to introduce a cache layer in the system architecture to cache hot data, archive data, and Homepage query, etc., to achieve the purpose of quick response.
4. Asynchronous processing flow
Asynchrony is a design concept. Asynchronous operation is not equal to multithreading, MQ middleware, or message broadcasting. These are the ways to achieve asynchronous processing. Asynchronous processing does not need to block the current thread to wait for the completion of the processing, but allows subsequent operations until others The thread will finish processing, and call back to notify this thread.
5. High concurrency and resource lock
The core of high-concurrency services is flow control, which controls the sinking speed of the flow, or controls the size of the container that receives the flow, and the excess directly overflows, which is a relatively complicated process. On the one hand, the amount of requests can be solved by traffic shaping, and on the other hand, the problem of concurrent access to resources can be solved by locking.
6, distributed transactions
Different services are composed of multiple detailed operations of different databases. These non-perceived detailed operations are distributed on different services, and even belong to different regions and applications. Transaction participants, transaction-supporting servers, resource servers, and transaction managers are located respectively For different nodes of different distributed systems, how to ensure that these operations are all successful or all fail, that is, to ensure data consistency between different databases, this is the core problem that distributed transactions need to solve.
https://github.com/cicadasmile/data-manage-parent
Four, data source components
1. Relational database
A database that uses a relational model to organize data, which stores data in the form of rows and columns, such as MySQL and Oracle. In a distributed system, in order to ensure the stability of the core process, relational databases are basically used in key services. When the business is completed, if the amount of data is large, the data will be synchronized to other high-performance components.
2. NoSQL database
NoSQL means "not just SQL". A general term for database management systems that are different from traditional relational databases. NoSQL is used for the storage of very large-scale data. These types of data storage do not require a fixed pattern and can be scaled horizontally without unnecessary operations. For example, MongoDB, Cassandra, etc.
3. Data management strategy
Separation of read-write database, query data sub-database sub-table, distributed business sub-database, sub-database based on user traffic.
https://github.com/cicadasmile/data-manage-parent
Five, service monitoring
1. Production failure
In a distributed and complex architecture, problems at any level of application services, software services, and hardware services may cause the request to be incompletely executed, triggering a series of effects, and monitoring the entire link to quickly locate the problem is very critical.
2. Application layer monitoring
The application layer is the developed business logic service, which is also the most prone to unexpected problems. It usually monitors the system from the aspects of request traffic, service link fuse, and system abnormal logs to observe whether the system is stable.
3. Software layer monitoring
This usually refers to the database level, such as Druid's monitoring and analysis; commonly used middleware, such as the RocketMQ console; Redis cache: provides commands to obtain related monitoring data, etc.
4. Hardware layer monitoring
At the hardware level, the three core contents of concern: CPU, memory, and network. The failure of the underlying hardware resources is likely to be triggered by the upper application service or middleware service. Mature monitoring frameworks, such as zabbix, grafana, etc.
Six, source code address
GitHub·地址
https://github.com/cicadasmile
GitEE·地址
https://gitee.com/cicadasmile
Recommended reading: finishing programming system
| Serial number | project name | GitHub address | GitEE address | Recommended |
|---|---|---|---|---|
| 01 | Java describes design patterns, algorithms, and data structures | GitHub·click here | GitEE·Click here | ☆☆☆☆☆ |
| 02 | Java foundation, concurrency, object-oriented, web development | GitHub·click here | GitEE·Click here | ☆☆☆☆ |
| 03 | Detailed explanation of SpringCloud microservice basic component case | GitHub·click here | GitEE·Click here | ☆☆☆ |
| 04 | SpringCloud microservice architecture actual combat comprehensive case | GitHub·click here | GitEE·Click here | ☆☆☆☆☆ |
| 05 | Getting started with SpringBoot framework basic application to advanced | GitHub·click here | GitEE·Click here | ☆☆☆☆ |
| 06 | SpringBoot framework integrates and develops common middleware | GitHub·click here | GitEE·Click here | ☆☆☆☆☆ |
| 07 | Basic case of data management, distribution, architecture design | GitHub·click here | GitEE·Click here | ☆☆☆☆☆ |
| 08 | Big data series, storage, components, computing and other frameworks | GitHub·click here | GitEE·Click here | ☆☆☆☆☆ |