I. Introduction
"Oh, serious online server timeout, the request is very slow, as if too many connections newspaper, how do?" Little friends in the feedback. General approach of our technical boss, the number of connections and thread pools transfer large point, restart, and then observe.
This approach is often emergency measures, a temporary solution, because I do not know the cause of the problem.
There is a serious misunderstanding, that the thread pool is set too small, large point the request will adjust faster.
Today with a small partners to communicate about the size of the thread pool should be how to set reasonable size?
Second, the problem
If there are two tasks to process, a job A, a job B
A program: a thread executing task A and B, after the implementation of A, B performs
Scheme II: two threads A and B to perform the task A and B, at the same time
Which program would hurry? It should be a lot of people will answer, certainly Option II ah, multi-threaded parallel processing tasks to A and B, certainly was quick. Is that right? Before answering this question, to bring everyone to look back comb.
Third, thread execution
Execution thread is dispatched by the CPU, a CPU at the same time will only execute one thread, we looked the thread A and thread B executed concurrently.
In order to allow users to feel these tasks are carried out at the same time, the operating system uses a round-robin fashion, CPU for each task service certain amount of time, then the preservation state of the current task, the next task of the state is loaded, continue service next task. Status of the task of saving and reloading, this process is called a context switch.
Context switching process takes time; now we look at the above problems, little friends look at what is which program fast? Is not some little friends would say a program, because there is no thread switch; Option II need to switch back and forth these two threads will be more time-consuming point.
Little friends heart at this time is not there wondering, why there will be multi-threaded? First in no hurry, and then look down.
Fourth, why multiple threads
Think of a small partner in our real business, we are what process?
FIG on the process:
1, first initiate network requests
2, Web server resolution request
3, the data acquisition request backend database
4, the data acquisition, processing
5, the processing results back to the user
This is the time we deal with business, the conventional request process; we look at the whole process involves what computer processing.
1, network requests -----> network IO
2, resolution request -----> CPU
3, the database request -----> network IO
4, MySQL query data -----> Disk IO
5, MySQL returns data -----> Network IO
6, the data processing -----> CPU
7, data is returned to the user -----> network IO
Talking about this, little friends are not feeling without chaos, in a real business that we do not only involve CPU computing, as well as network and disk IO IO processes which are very time-consuming. If a thread on the entire process flow diagram is really involved only two CPU nodes, the other nodes are IO processing, the thread doing IO processing time, CPU on idle out, not CPU utilization high.
Small partners now know multithreaded use it, right, that is, in order to improve CPU utilization.
Fifth, to enhance the QPS / TPS
How to measure system performance, key indicators system (QPS / TPS)
The number of requests per second can be processed / transaction: QPS / TPS
The number of requests processed while the system / transaction: the number of concurrent
When the processing is the average length of a request / transaction requires: Response Time
QPS / TPS = number of concurrent / Response Time
The above formula represents a higher number of concurrent, QPS greater; so many people will turn up that thread pool, a large number of concurrent will also enhance the QPS, thus giving rise to the beginning of the preface said, most people misunderstanding.
In fact QPS kind enough response time is inversely proportional to the response time of the larger, the smaller will be QPS.
Although the number of concurrent turn up, it will enhance the QPS, but the number of threads can also affect response time, because of the above mentioned problems we context switch, then how to set the number of threads it?
Sixth, how to set the number of threads
How do we allocate a thread? We offer a formula:
The optimum number of threads = ((thread thread CPU time latency +) / thread CPU) * Number of CPU
Remarks This formula also share predecessors, of course, before the front desk looked at Taobao system optimization practice articles, and the above formula is very similar, but the number of CPU over there, they are more refined, the above formula only for reference. But no matter what the formula, eventually after running a production environment, and then optimize the adjustment.
We continue the above tasks, our server CPU core number four nuclear, a time-consuming task thread cpu is 20ms, thread waiting (network IO, disk IO) takes 80ms, and that the optimal number of threads :( 80 + 20) / 20 * 4 = 20. That is the best set the number of 20 threads.
From this we draw the above formula, the greater the waiting time for a thread, the greater the number of threads necessary to set up, this fits well with our analysis above, can improve CPU utilization. It says that from another perspective, the number of threads set how much, according to our own business, and to make their own stress tests, setting a reasonable value.
Seven, the basis of conventional criteria
That our little friends will ask, because a lot of business together in one thread pool, unlike the above case is relatively simple, in fact too much business, how to set it? This is going to adjust the stress test. But our predecessors has helped us to summarize the value of a base (ultimately depends on the operation of the self-adjusting)
1, CPU-intensive: the operation memory processing operations, the number of threads is generally set to: the number of CPU cores or 1 + 2 * number of CPU cores. Nuclear number 4, the general set 5 or 8
2, IO-intensive: the file operations, network operations, database operations, the thread is generally set to: cpu Audit / (1 - 0.9), the number of core 4, the general settings 40
Eight, summary
Today introduced the number of threads set the size of some small partners of errors. Here we talked about little friends are not thread has been updated to understand, unlike before rude, you should try to analyze why so slow, system bottlenecks arise in any place, to reduce the time-consuming bottlenecks.
The recommended little friends and then take a look Redis, Nginx; why they would be so fast? In fact, the point of this article and knowledge have in common.