Table of contents
For every server-side development engineer, multi-threading will definitely be used in daily development to improve the efficiency of task execution. When it comes to multi-threading, everyone can skillfully give several ways to use multi-threading.
In several ways, using the thread pool to manage threads is a more standardized way, which can reduce the time spent on creating and destroying threads, make full use of system resources to bring users faster response, and create a thread pool to handle concurrent tasks It looks simple, but it is not. The parameters of the thread pool are very particular. The common mistake made by beginners is that the thread is either blocked and busy or idle. Speaking of this, I can’t help but ask the friends in front of the screen, what do you think? Can we create the optimal thread pool?
1. How to create a thread pool
Before explaining how to create an optimal thread pool, let's first introduce how to create a standard thread pool.
In Java code, we generally use ThreadPoolExecutor to create a thread pool, and inject appropriate parameters at the same time. The combination of the two makes the use of the thread pool optimal; although Java provides methods such as newFixedThreadPool to create a thread pool, but it has certain Limitations, we should use different thread pool creation parameters for different usage scenarios to obtain optimal performance.
What are the configurations of the thread pool
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler)**加粗样式**
maximumPoolSize: The maximum number of threads in the thread pool, indicating how many threads can be created in the thread pool at most. If and when the number in the thread pool reaches this number, the new task will throw an exception.
keepAliveTime: Indicates how long the thread can keep running when there is no task to execute, and then the number of thread pools is maintained at corePoolSize.
unit: The time unit of the parameter keepAliveTime
workQueue: A blocking queue used to store tasks waiting to be executed. If the current demand for threads exceeds the corePoolSize size, it will be placed here.
threadFactory: thread factory, mainly used to create threads, such as specifying the name of the thread.
handler: If the thread pool is full, the new task processing method.
How to create an optimal thread pool
There is no fixed set of thread pool creation parameters that make the thread pool optimal in any environment, but there are best practices. Through the process of first measuring and then optimizing, the optimal creation parameters in different scenarios can be obtained. Usually the process goes like this
1. Based on experience and general consensus, create relatively reasonable parameters in the order of requirements
Taking the thread pool as an example, we need to consider the appropriate number of threads to set. This number mainly depends on the CPU resources of the server and the type of tasks.
If the task is to read and write data, then it depends on the number of connections to the critical value of the database and the load and performance of the database. For example, the maximum number of connections per second for a single database is 100. If the number of threads is set to be greater than 100, no more threads will be used, and the database will be given Bring greater pressure, and even wear down the database.
If the task is to access a third-party service through the network, it depends on the size of the network load and the load and performance of the third-party service (similar to the case of reading and writing data, it simply depends on the network bandwidth and the performance of the third-party service).
Generally speaking, cpu-intensive tasks occupy a relatively long amount of CPU, and the number of threads can be set smaller. IO-type tasks occupy less CPU time, and the number of threads can be set larger. Our purpose is to make full use of the given CPU. Resources, ok, according to the following formula, assuming that our server has 4 cores, the waiting time for sending tasks to the task execution center is 50ms, and the task center takes 5ms to process each task, and we hope that the CPU usage rate is 10% , and finally we can calculate that the number of threads is 4.4, which is 5 threads.
With the number of threads, we can configure and create our thread pool through the following parameters.
2. Make adjustments based on metrics
After the thread pool is created using the estimated value, the work is not over. We need to tune the parameters through some measurement work. We generally use the following grade points as metrics for thread pool performance.
Three steps for the overall measurement process atmosphere
1. Create a thread pool and register various measurement indicators.
We can use the InstrumentedExecutorService in the metrics library of dropwizard to help us make statistics on the above indicators. Types of dots, statistics of various runtime state parameters of the thread pool during operation.
2. Run the thread pool and collect metrics
After the thread pool is started, the library function will automatically record the thread pool running performance indicators and save them in a csv file.
3. Observe the metrics and adjust the parameters accordingly.
The runtime status data recorded in csv is converted into a graph through the python visualization function, which is convenient for us to analyze. In the analysis, we will focus on four points 1. Execution time of batch
tasks
2. Tasks The number and duration of accumulation in the thread pool queue
3. Changes of working threads in the thread pool
4. Tasks discarded by the thread pool due to overload
Let me give you an example to see how to tune by analyzing the embedded log of the number of threads in the thread pool. We first convert the csv file into a graph.
For a thread pool with good performance, the number of threads in the thread pool should be relatively stable, avoid frequent creation and destruction of threads, and increase the burden of thread context switching. From the figure, we can see that the number of threads fluctuates and compares obvious. Our corePoolSize, maxPoolSize, and keepAliveTime can all be adjusted appropriately. The increase in the number of core threads is because the thread pool has peak clipping during the running process. The reason for increasing the thread storage time is that it can be found through observation that when 45-50ms, the thread It is frequently created and destroyed, increasing the thread survival time and helping to improve thread utilization.