I. Overview
Highly concurrent systems are different, such as megabits per second concurrent middleware system, gateway system daily ten billion request, spike big promotion system hundreds of thousands of requests per second instantaneous. When they deal with high concurrency, because different systems each with its own characteristics, so deal with architecture is not the same.
In addition, such electronic business platform in order system, merchandise system, inventory system architecture design in high concurrency scenarios are different, because the business behind the scenes of what is different.
Article provides an answer to such questions is thinking, it does not involve any complex architecture design.
Second, one of the most simple system architecture
Assuming that your system is just beginning to be deployed on a machine behind it connected with a database, the database is deployed on a single server. For example, your system deployment machine is a 4-core 8G, the database server is a 16-core 32G.
At this time, assuming your system to a total of 100,000 subscribers, subscribers few, Nikkatsu user differentiated according to the situation of different systems, we take a more objective proportion, 10% of it, every day active users 10000. 28 in accordance with the law, the peak of the every day count four hours, the peak of active users accounted for 80%, that is 8,000 people active in 4 hours. Then everyone on your system initiated the request, we counted 20 times a day he is right. Then the peak of the 8,000-initiated request also only 16 million times per second to average (14,400 seconds) within four hours, also 10 requests per second. Complete with high concurrency take a share, right? Then the system level is 10 times per second request, calls to the database every request several database operations, such as doing the crud and the like. So we take a first request corresponds to three times the database requests it, that this is the case, the database layer also 30 requests per second, right? According to this configuration database server support is absolutely no problem. The system described above, with a view showing, this is the following:
One of the most simple system architecture
Third, the system cluster deployment
Assuming that your users start of rapid growth, such as the amount of registered users increased 50-fold, rising to 500 million. At this point Nikkatsu 500,000 users, the peak of requests per second, the system is 500 / s. Then the database requests per second number is 1500 / s, this time what will happen?
According to the above machine configuration, if the processing within your system is more complex some business logic, business logic is a kind of heavy system, then, is a relatively time-consuming CPU.
At this point, 4-core 8G requests per second when the machine reach 500 / s, it is likely you will find that the higher the CPU load your machine. Then the database level, in terms of the above-described arrangement, in fact, substantially peak 1500 / s requested pressure, then still acceptable.
This is mainly to observe the machine where the database disk load, network load, CPU load, memory load, in accordance with our online experience, the configuration database is no problem under pressure in 1500 / s request.
So now you need to do a thing, the first is to support your system cluster deployment. You can hang in front of a load-balancing layer, the request hit a uniform system level, the system can use multiple clustered machines support higher concurrency pressure.
For example, the system is assumed here that the deployment of a machine, then each machine only 250 requests / s of the. As a result, the two machines will significantly reduce the CPU load, this initial "high concurrency" is not on the first cover live it?
If even this is not done, when that single machine load higher and higher, in extreme cases is possible to deploy the system on the machine can not have enough resources to respond to the request, and the request appears stuck, even of system downtime problem class.
So simple summary, the first thing to do is add a layer load balancing, system will request even hit the floor.
System layer using cluster deployment of multiple machines, Kang Zhu preliminary concurrent pressure. At this architecture diagram becomes like the following:
Adding a load balancing layer
Fourth, the database sub-library separate read and write sub-table +
Assuming that the number of users continues to grow, reaching 10 million registered users, and then every day is one million users daily living. So this time the amount of requests for system level will reach 1000 per second / s, system level, you can continue to cluster expansion by the way, anyway, front load balancing layer evenly dispersed flow past.
However, when the amount of the request will reach an acceptable level database 3000 / s, this little problem. At this point the database level of concurrent requests doubled, you will find online database load is increasing. Every time the peak of the pressure will be very high disk IO, network IO, memory consumption, CPU load, and we are very worried that the database server can Kang Zhu.
In general, the kind of online database for common configuration, it is recommended that concurrent read and write together, according to the configuration of our example above, not more than 3000 / s. Because the database too much pressure, the first problem is the peak system performance may be reduced, because of the impact on the performance of the database load is too high there. Another, the pressure is too great might do hang database.
So in this case the system must have sub-table to do the sub-library separate read and write +, it is to split a library for multiple libraries, deployed on a plurality of database services, when the primary database write request bearer. Each primary library is then mounted at least one from the library, from the library to be requested by the read bearer.
At this time, assuming that the level of concurrent read and write database is 3000 / s, where concurrent writes accounted for 1000 / s, concurrent read accounted for 2000 / s. Once the sub-library then sub-table used to deploy the primary database on the database server to support both a write request, write each server is hosted concurrent 500 / s. Each mount a main library from the library server deployment, the two concurrent read from the library is 1000 / s each supported from the library.
A brief summary, when the amount of concurrency continues to grow, we need to focus at the database level: sub-library sub-table, separate read and write. At this time, the structure shown in FIG follows:
Database level
Sixth, the cache cluster introduced
If the amount of registered users is growing, then you can add non-stop machine, such as system level constantly adding machine, you can carry higher concurrent requests. If the write concurrent database level higher and higher, plus a database server on expansion through sub-library sub-table is to support the expansion of the machine, if the database level concurrent read more and more, the expansion would add more from the library.
But there is a big problem: The database itself is not actually used to carry high concurrent requests, so generally speaking, concurrent database single carrier per second in the order of thousands, and the database used by the machine are relatively high-profile , more expensive machines, the cost is high. If you simply stop adding machines, in fact, it is wrong. So often have high concurrency architecture in this part of the design cache, the cache system is to carry high concurrency born.
Concurrent single amount carried per second in tens of thousands, even hundreds of thousands per second, concurrent high load carrying capacity than the database system to be higher than one to two orders of magnitude. Therefore, according to business characteristics of the system, of the kind of writing less reading and more requests, the introduction of cache cluster.
Specifically, it is at the same time to write a database of data when writing to the cache cluster, and then use the cache cluster to host most of the read request. In this case, by the cache cluster, the machine can use fewer resources carry higher concurrency.
"Database level .jpg", the read request is currently 2000 per second / s, two from the library each carry a 1000 / s read request, but the request which read 1800 times per second are possible can be directly read cache is not how changes in data. Once introduced cache cluster case, this can be an anti-down 1800 / s read request, the read request database level fell to 200 / s. At this time, the structure shown in FIG follows:
Cache clusters
What are the benefits above architecture is it?
The future, the system reads the request to reach tens of thousands of times per second, but probably 80% to 90% are read by the cache cluster, and the cache in the cluster can support tens of thousands of machines may read requests per second, stand-alone, so consuming machine few resources, may be sufficient to two or three machine. If the database is replaced, it may be necessary to keep the library to increase from 10, 20 machines to tens of thousands of concurrent Kang Zhu read per second, the cost is very high.
Briefly summarized, the carrier high concurrent another point to consider:
- Expansion not blindly database, the database server is expensive, and is not itself used to carry high concurrency.
- Read more for less write request, the introduction of cache cluster, Kang Zhu large number of read requests with the cache cluster.
Sixth, the introduction of messaging middleware clusters
If you say that all write requests all the main library floor landing database, of course, is no problem, but if the pressure to write more and more of it? For example, to write tens of thousands of pieces of data per second, this time to do is stop the main library plus machine? Of course you can, but the same token, consuming machine resources is great, this is the feature of the database system of the decision.
Under the same resources, database systems too heavy and complex, complicated by the carrying capacity on the order of a few thousand / s. At this point some of the other technologies need to be introduced. For example, messaging middleware technology, which is MQ cluster, he is very good to do a write request asynchronous processing, to achieve the effect of load shifting.
If you say, now is the second 1000 / s write request times, 500 times in which such a request is a request must come immediately written to the database, but another 500 may allow asynchronous write request is a wait of several tens of seconds or even minutes after falling in the database. At this time, a message can be introduced middleware cluster 500 requests per second to allow asynchronous write of MQ, then made based on a load shifting MQ. For example at the speed smoothly 100 / s out of the consumer can then fall into the database, this time will be significantly reduced pressure to write the database. At this time, the architecture in FIG becomes the following:
Messaging middleware clusters
We see architecture diagram above, the first messaging middleware system itself but also for high concurrency was born, and so are usually single-level support tens or even hundreds of concurrent requests.
So also with his own caching system, able to use very little resources to support high concurrent requests, use it to support part of the high concurrency allows asynchronous write is no problem, than that portion of the database directly support high-concurrency request to reduce the number of machine usage.
After load shaving and messaging middleware, such as a database with write speed and stability of 100 / s, then the received write request database level pressure, does not become 500 / s + 100 / s = 600 / s up Why?
You see, it is not found to reduce the pressure on the database?
So far, the following means, we have to have the system architecture as much as possible with minimal machine resources Kang Zhu maximum pressure request, reducing the burden on the database.
- System Cluster
- Database level sub-library separate read and write sub-table +
- How much for reading written request, the introduction of cache clusters
- For high pressure is written, the introduction of messaging middleware clusters.