1. What is high concurrency
High concurrency is one of the factors that must be considered in the design of the Internet distributed system architecture. It usually means that the design ensures that the system can process many requests in parallel at the same time .
Some commonly used indicators related to high concurrency include Response Time, Throughput, QPS (Query Per Second), and the number of concurrent users.
- Response time : The time it takes for the system to respond to the request. For example, it takes 200ms for the system to process an HTTP request, and this 200ms is the response time of the system.
- Throughput : The number of requests processed per unit time.
- QPS : The number of response requests per second. In the Internet domain, the distinction between this metric and throughput is not so clear. Number of concurrent users: The number of users who are using the system functions normally at the same time. For example, in an instant messaging system, the number of simultaneous online users represents the number of concurrent users of the system to a certain extent.
2. How to improve the concurrency capability of the system
There are two main methods to improve the concurrency capability of the Internet distributed architecture design : vertical expansion (Scale Up) and horizontal expansion (Scale Out) .
Vertical expansion: Improve the processing capacity of a single machine. There are two ways to scale vertically:
- Enhance stand-alone hardware performance, for example: increase the number of CPU cores such as 32 cores, upgrade a better network card such as 10 Gigabit, upgrade a better hard disk such as SSD, expand the hard disk capacity such as 2T, and expand the system memory such as 128G;
- Improve the performance of stand-alone architecture, for example: use Cache to reduce IO times, use asynchronous to increase single-service throughput, and use lock-free data structures to reduce response time.
In the early days of the rapid development of the Internet business, if budget is not a problem, it is strongly recommended to use the method of "enhancing the performance of single-machine hardware" to improve the concurrency capability of the system, because at this stage, the company's strategy is often to develop business to seize time, and "enhance the performance of single-machine hardware" "It's often the fastest way.
Whether it is to improve the performance of stand-alone hardware or to improve the performance of stand-alone architecture, there is a fatal deficiency: stand-alone performance is always limited. Therefore , the ultimate solution for the design of high concurrency in the distributed architecture of the Internet is horizontal expansion .
Horizontal expansion: As long as the number of servers is increased, the system performance can be linearly expanded. Horizontal expansion requires system architecture design. How to design horizontal scalability at each layer of the architecture, as well as the common horizontal expansion practices at each layer of the Internet company architecture, are the focus of this article.
3. Common Internet Layered Architecture
The common Internet distributed architecture is as above, divided into:
- Client layer: The typical caller is the browser browser or the mobile application APP
- Reverse proxy layer: system entry, reverse proxy
- Site application layer: implement core application logic and return html or json
- Service layer: If the service is realized, there is this layer
- Data-cache layer: cache accelerates access to storage
- Data-database layer: database curing data storage
How to implement the horizontal expansion of the whole system at all levels?
Fourth, the practice of hierarchical horizontal expansion architecture
Horizontal scaling of the reverse proxy layer
The horizontal expansion of the reverse proxy layer is achieved through "DNS polling": dns-server configures multiple resolution IPs for a domain name, and each time a DNS resolution request is made to access dns-server, these IPs will be polled and returned.
When nginx becomes the bottleneck, as long as the number of servers is increased, the deployment of the nginx service is added, and an external network IP is added, the performance of the reverse proxy layer can be expanded, and the theoretically infinitely high concurrency can be achieved.
Horizontal scaling of the site layer
The horizontal expansion of the site layer is achieved through "nginx". By modifying nginx.conf, multiple web backends can be set up.
When the web backend becomes a bottleneck, as long as the number of servers is increased, the deployment of new web services is added, and a new web backend is configured in the nginx configuration, the performance of the site layer can be expanded, achieving theoretically infinitely high concurrency.
Horizontal scaling of the service layer
When the site layer calls the downstream service layer RPC-server through RPC-client, the connection pool in the RPC-client will establish multiple connections with the downstream service. When the service becomes a bottleneck, just increase the number of servers and deploy new services. Establishing a new downstream service connection at the RPC-client can expand the performance of the service layer and achieve theoretically infinitely high concurrency. If you need to perform automatic expansion of the service layer elegantly, you may need to support the automatic service discovery function in the configuration center.
Horizontal scaling of the data layer
In the case of a large amount of data, the data layer (cache, database) involves horizontal expansion of data, and the data (cache, database) originally stored on one server is horizontally split to different servers to expand system performance. the goal of.
There are several common horizontal splitting methods in the Internet data layer. Take the database as an example:
Split horizontally by range
Each data service stores a certain range of data. The above figure is an example:
- user0 library, storage uid range 1-1kw
- user1 library, storage uid range 1kw-2kw
The benefits of this program are:
- The rules are simple, the service can route to the corresponding storage service only by judging the uid range;
- The data is well balanced;
- It is relatively easy to expand, and you can add a uid[2kw,3kw] data service at any time.
The shortcomings are:
- The load of requests is not necessarily balanced. Generally speaking, newly registered users will be more active than old users, and the service request pressure of large ranges will be greater.
Split by hash level
Each database stores part of the data hashed by a certain key value. The above figure is an example:
- user0 library, storing even uid data
- user1 library, storing odd uid data
The benefits of this program are:
- The rules are simple, the service only needs to hash the uid to route to the corresponding storage service;
- The data is well balanced;
- Request uniformity is better.
The shortcomings are:
- It is not easy to expand. To expand a data service, when the hash method changes, data migration may be required.
It should be noted here that the expansion of system performance through horizontal splitting is fundamentally different from that of master-slave synchronous read-write separation to expand database performance.
Scale database performance by splitting horizontally:
- The amount of data stored on each server is 1/n of the total, so the performance of a single machine will also be improved;
- There is no intersection of data on n servers, and the union of data on that server is the complete set of data;
- The data is horizontally split to n servers, theoretically, the read performance is expanded by n times, and the write performance is also expanded by n times (in fact, it is far more than n times, because the data volume of a single machine has become 1/n of the original).
Extend database performance through master-slave synchronous read-write separation:
- The amount of data stored on each server is the same as the total;
- The data on the n servers are all the same, and they are all complete sets;
- Theoretically, the read performance is expanded by n times, the write is still a single point, and the write performance remains unchanged.
The horizontal splitting of the cache layer is similar to the horizontal splitting of the database layer, and most of them are range splitting and hash splitting, so they will not be expanded.
V. Summary
High concurrency is one of the factors that must be considered in the design of the Internet distributed system architecture. It usually means that the design ensures that the system can process many requests in parallel at the same time .
There are two main methods to improve the concurrency capability of the system: vertical expansion (Scale Up) and horizontal expansion (Scale Out) . The former vertical expansion can improve concurrency by improving the performance of single-machine hardware or improving the performance of single-machine architecture, but there is always a limit to single-machine performance. The ultimate solution for high concurrency in Internet distributed architecture design is the latter: horizontal expansion .
In the layered architecture of the Internet, the practice of horizontal expansion at each layer is different:
- The reverse proxy layer can be extended horizontally through "DNS round-robin";
- The site layer can be extended horizontally through nginx;
- The service layer can be extended horizontally through the service connection pool;
- The database can be scaled horizontally according to data range or data hash.
After the horizontal expansion of each layer, the performance of the system can be improved by increasing the number of servers, so that the theoretical performance is unlimited.
At the end, I hope that the idea of the article is clear, and I hope that everyone has a systematic understanding of the concept and practice of high concurrency. Combined with the sharing of the previous article "Technical Practice of Internet High Availability Architecture" , is the slow Internet distributed architecture gradually no longer mysterious?