In-depth analysis of the implementation principle of Java thread pool

1. Write in front

1.1 What is a thread pool

Thread Pool is a tool for thread management based on the idea of pooling, and it often appears in multi-threaded servers, such as MySQL.

Too many threads will bring additional overhead, including the overhead of creating and destroying threads, the overhead of scheduling threads, etc., and it also reduces the overall performance of the computer. The thread pool maintains multiple threads, waiting for the supervisory manager to allocate tasks that can be executed concurrently. This approach, on the one hand, avoids the cost of creating and destroying threads when processing tasks, on the other hand, it avoids the over-scheduling problem caused by the expansion of the number of threads, and ensures the full utilization of the kernel.

The thread pool described in this article is the ThreadPoolExecutor class provided in the JDK.

Of course, the use of thread pools can bring a series of benefits:

Reduce resource consumption : Reuse created threads through pooling technology to reduce the loss caused by thread creation and destruction.
Improve response speed : When a task arrives, it can be executed immediately without waiting for the thread to be created.
Improve the manageability of threads: Threads are scarce resources. If they are created without restriction, they will not only consume system resources, but also cause resource scheduling imbalance due to unreasonable distribution of threads and reduce system stability. Use the thread pool to perform uniform allocation, tuning and monitoring.
Provide more and more powerful functions : The thread pool is extensible, allowing developers to add more functions to it. For example, the delayed timing thread pool ScheduledThreadPoolExecutor allows tasks to be executed in postponement or regular execution.

1.2 What is the problem solved by the thread pool

The core problem solved by the thread pool is the problem of resource management. In a concurrent environment, the system cannot determine how many tasks need to be executed and how many resources need to be invested at any time. This uncertainty will bring about the following problems:

Frequent application/destroy of resources and scheduling resources will bring additional consumption, which may be very huge.
The lack of restraint methods for unlimited applications can easily lead to the risk of exhaustion of system resources.
The system cannot manage the internal resource distribution reasonably, which will reduce the stability of the system.

To solve the problem of resource allocation, the thread pool adopts the "Pooling" idea. Pooling, as the name suggests, is an idea of unifying resources for management in order to maximize benefits and minimize risks.

Pooling is the grouping together of resources (assets, equipment, personnel, effort, etc.) for the purposes of maximizing advantage or minimizing risk to the users. The term is used in finance, computing and equipment management.——wikipedia

The "pooling" idea can be applied not only in the computer field, but also in the fields of finance, equipment, personnel management, and work management.

The performance in the computer field is: unified management of IT resources, including servers, storage, and network resources. By sharing resources, users can benefit from low investment. Apart from the thread pool, there are several other typical usage strategies including:

Memory Pooling: Pre-apply for memory to increase the speed of memory application and reduce memory fragmentation.
Connection Pooling: Apply for a database connection in advance to increase the speed of applying for a connection and reduce system overhead.
Object Pooling: Recycle objects to reduce the costly loss of resources during initialization and release.

After understanding the "what" and "why", let's dive into the internal implementation principle of the thread pool.

2. Thread pool core design and implementation

In the previous article, we learned that: thread pool is a tool that helps us manage threads and obtain concurrency through the idea of "pooling". The embodiment in Java is the ThreadPoolExecutor class. So what is its detailed design and implementation like? We will introduce them in detail in this chapter.

2.1 Overall design

The core implementation class of thread pool in Java is ThreadPoolExecutor. This chapter analyzes the core design and implementation of Java thread pool based on the source code of JDK 1.8. Let's first look at the UML class diagram of ThreadPoolExecutor to understand the inheritance relationship of ThreadPoolExecutor.

The top-level interface implemented by ThreadPoolExecutor is Executor, and the top-level interface Executor provides an idea: decoupling task submission and task execution. Users do not need to pay attention to how to create threads and how to schedule threads to perform tasks. Users only need to provide Runnable objects and submit the running logic of the tasks to the Executor. The Executor framework completes the deployment of threads and the execution of tasks. The ExecutorService interface adds some capabilities: (1) Expanding the ability to execute tasks, supplementing the method that can generate Futures for one or a batch of asynchronous tasks; (2) Providing methods to control the thread pool, such as stopping the running of the thread pool. AbstractExecutorService is an upper-level abstract class that connects the process of executing tasks in series to ensure that the implementation of the lower layer only needs to focus on one method of executing the task. The lowest-level implementation class ThreadPoolExecutor implements the most complex operation part. ThreadPoolExecutor will maintain its own life cycle on the one hand, and manage threads and tasks at the same time, so that the two can be combined to perform parallel tasks.

How does ThreadPoolExecutor work? How to maintain threads and perform tasks at the same time? Its operating mechanism is shown in the figure below:

The thread pool actually builds a producer-consumer model internally, decoupling threads and tasks, and not directly related, so as to buffer tasks and reuse threads. The operation of the thread pool is mainly divided into two parts: task management and thread management. The task management part acts as a producer. When the task is submitted, the thread pool will determine the subsequent flow of the task: (1) directly apply for the thread to execute the task; (2) buffer in the queue and wait for the thread to execute; (3) reject the task task. The thread management part is the consumer. They are uniformly maintained in the thread pool. The threads are allocated according to the task request. When the thread finishes the task, it will continue to obtain new tasks for execution. Finally, when the thread cannot obtain the task, the thread Will be recycled.

Next, we will explain the thread pool operating mechanism in detail in accordance with the following three parts:

How the thread pool maintains its own state.
How the thread pool manages tasks.
How the thread pool manages threads.

2.2 Life cycle management

The running state of the thread pool is not explicitly set by the user, but is maintained internally along with the running of the thread pool. The thread pool uses a variable to maintain two values: the running state (runState) and the number of threads (workerCount). In the specific implementation, the thread pool puts together the maintenance of the two key parameters of the running state (runState) and the number of threads (workerCount), as shown in the following code:

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));

ctlThis AtomicInteger type is a field that controls the running state of the thread pool and the number of valid threads in the thread pool. It also contains two parts of information: the running state of the thread pool (runState) and the number of valid threads in the thread pool (workerCount ), the upper 3 bits store runState, and the lower 29 bits store workerCount. The two variables do not interfere with each other. Using one variable to store two values can avoid inconsistencies when making relevant decisions. It is not necessary to occupy lock resources in order to maintain the consistency of the two. It can also be found by reading the source code of the thread pool that it is often necessary to judge the running state of the thread pool and the number of threads at the same time. The thread pool also provides several methods for users to obtain the current running status and the number of threads in the thread pool. The bit operation method is used here. Compared with the basic operation, the speed will be much faster.

The calculation method for obtaining the life cycle status and the number of threads in the thread pool of the internal package is shown in the following code:

private static int runStateOf(int c)     { return c & ~CAPACITY; } //计算当前运行状态
private static int workerCountOf(int c)  { return c & CAPACITY; }  //计算当前线程数量
private static int ctlOf(int rs, int wc) { return rs | wc; }   //通过状态和线程数生成ctl

There are 5 operating states of ThreadPoolExecutor, namely:

Its life cycle conversion is as follows:

2.3 Task execution mechanism

2.3.1 Task scheduling

Task scheduling is the main entrance of the thread pool. When a user submits a task, how the task will be executed next is determined by this stage. Understanding this part is equivalent to understanding the core operating mechanism of the thread pool.

First of all, the scheduling of all tasks is completed by the execute method. The work done in this part is to check the current running status of the thread pool, the number of running threads, and the running strategy, and decide the next execution process. It is to directly apply for thread execution, or Is it buffered in the queue for execution, or directly rejected the task. The execution process is as follows:

First, check the running status of the thread pool, if it is not RUNNING, reject it directly, and the thread pool must ensure that the task is executed in the RUNNING state.
If workerCount <corePoolSize, create and start a thread to execute the newly submitted task.
If workerCount >= corePoolSize and the blocking queue in the thread pool is not full, the task is added to the blocking queue.
If workerCount >= corePoolSize && workerCount <maximumPoolSize, and the blocking queue in the thread pool is full, create and start a thread to execute the newly submitted task.
If workerCount >= maximumPoolSize, and the blocking queue in the thread pool is full, the task will be processed according to the rejection policy. The default processing method is to directly throw an exception.

The execution process is shown in the following figure:

2.3.2 Task buffer

The task buffer module is the core part of the thread pool that can manage tasks. The essence of the thread pool is the management of tasks and threads, and the most critical idea to do this is to decouple the tasks and threads, and prevent the two from being directly related, so that subsequent assignments can be done. The thread pool is implemented in a producer-consumer mode through a blocking queue. The blocking queue caches tasks, and the worker threads obtain tasks from the blocking queue.

Blocking Queue (BlockingQueue) is a queue that supports two additional operations. These two additional operations are: when the queue is empty, the thread that gets the element will wait for the queue to become non-empty. When the queue is full, the thread storing the element will wait for the queue to be available. Blocking queues are often used in producer and consumer scenarios. Producers are threads that add elements to the queue, and consumers are threads that take elements from the queue. The blocking queue is the container in which the producer stores the elements, and the consumer only takes the elements from the container.

The following figure shows that thread 1 adds elements to the blocking queue, while thread 2 removes elements from the blocking queue:

Using different queues can achieve different task access strategies. Here, we can introduce the members of the blocking queue:

2.3.3 Task application

As can be seen from the task allocation section above, there are two possibilities for task execution: one is that the task is directly executed by the newly created thread. The other is that the thread obtains the task from the task queue and then executes it. The idle thread that has completed the task will again apply for the task from the queue and execute it. The first situation only occurs when the thread is initially created, and the second situation is when the thread acquires the vast majority of tasks.

The thread needs to continuously fetch tasks from the task cache module for execution, help the thread obtain tasks from the blocking queue, and realize the communication between the thread management module and the task management module. This part of the strategy is implemented by the getTask method, and its execution process is shown in the following figure:

This part of getTask has been judged many times in order to control the number of threads and make it consistent with the state of the thread pool. If the thread pool should not hold so many threads now, it will return a null value. The worker thread Worker will continue to receive new tasks to execute, and when the worker thread Worker fails to receive tasks, it will start to be recycled.

2.3.4 Task rejection

The task rejection module is the protection part of the thread pool. The thread pool has a maximum capacity. When the task buffer queue of the thread pool is full and the number of threads in the thread pool reaches the maximumPoolSize, the task needs to be rejected and the task rejection strategy is adopted. , Protect the thread pool.

The rejection strategy is an interface, and its design is as follows:

public interface RejectedExecutionHandler {
    void rejectedExecution(Runnable r, ThreadPoolExecutor executor);
}

Users can customize the rejection strategy by implementing this interface, or choose the four existing rejection strategies provided by the JDK. The features are as follows:

2.4 Worker thread management

2.4.1 Worker thread

Thread pool In order to grasp the status of threads and maintain the life cycle of threads, a worker thread in the thread pool is designed. Let's take a look at some of its code:

private final class Worker extends AbstractQueuedSynchronizer implements Runnable{
    final Thread thread;//Worker持有的线程
    Runnable firstTask;//初始化的任务，可以为null
}

Worker, a worker thread, implements the Runnable interface and holds a thread thread and an initialized task firstTask. Thread is a thread created by ThreadFactory when the constructor is called. It can be used to perform tasks; firstTask uses it to save the first task passed in, which can be null or null. If this value is non-empty, then the thread will execute the task immediately at the beginning of startup, which corresponds to the situation when the core thread is created; if this value is null, then you need to create a thread to execute the task list (workQueue) Tasks, that is, the creation of non-core threads.

The model of Worker execution tasks is shown in the figure below:

The thread pool needs to manage the life cycle of threads and needs to be recycled when threads are not running for a long time. The thread pool uses a Hash table to hold thread references, so that the life cycle of threads can be controlled by operations such as adding references and removing references. The important thing at this time is how to judge whether the thread is running.

Worker inherits AQS and uses AQS to realize the function of exclusive lock. Instead of using the reentrant lock ReentrantLock, AQS is used in order to realize the non-reentrant feature to reflect the current execution state of the thread.

1. Once the lock method acquires an exclusive lock, it means that the current thread is executing a task. 2. If the task is being executed, the thread should not be interrupted. 3. If the thread is not in an exclusive lock state, that is, it is in an idle state, it means that it is not processing tasks, and the thread can be interrupted at this time. 4. The thread pool will call the interruptIdleWorkers method to interrupt the idle thread when the shutdown method or tryTerminate method is executed. The interruptIdleWorkers method will use the tryLock method to determine whether the thread in the thread pool is idle; if the thread is idle, it can be safely recycled.

This feature is used in the process of thread recycling. The recycling process is shown in the following figure:

2.4.2 Worker thread increase

The thread is added through the addWorker method in the thread pool. The function of this method is to add a thread. This method does not consider the stage in which the thread pool is added. The thread allocation strategy is completed in the previous step. This step Just finish adding the thread, make it run, and finally return the result of success. The addWorker method has two parameters: firstTask and core. The firstTask parameter is used to specify the first task executed by the new thread. This parameter can be empty; the core parameter is true, which means it will judge whether the current active thread number is less than corePoolSize when adding threads, and false means it needs to be added before threads are added. Determine whether the current number of active threads is less than maximumPoolSize, and the execution flow is shown in the following figure:

2.4.3 Worker thread recycling

The destruction of threads in the thread pool depends on the automatic recycling of the JVM. The work of the thread pool is to maintain a certain number of thread references based on the current thread pool state to prevent these threads from being recycled by the JVM. When the thread pool determines which threads need to be recycled, only You need to eliminate its references. After the Worker is created, it will continuously poll and obtain tasks to execute. Core threads can wait indefinitely for tasks, and non-core threads have to obtain tasks within a limited time. When the Worker cannot obtain the task, that is, when the obtained task is empty, the loop will end, and the Worker will actively eliminate its own references in the thread pool.

try {
  while (task != null || (task = getTask()) != null) {
    //执行任务
  }
} finally {
  processWorkerExit(w, completedAbruptly);//获取不到任务时，主动回收自己
}

The work of thread recycling is done in the processWorkerExit method.

In fact, in this method, removing the thread reference from the thread pool has ended the thread destruction part. However, since there are many possibilities to cause thread destruction, the thread pool must determine what caused this destruction, whether to change the current state of the thread pool, and whether to reallocate threads according to the new state.

2.4.4 Worker thread executes tasks

The run method in the Worker class calls the runWorker method to perform tasks. The execution process of the runWorker method is as follows:

1. The while loop continuously gets tasks through the getTask() method. 2. The getTask() method takes tasks from the blocking queue. 3. If the thread pool is stopping, ensure that the current thread is in an interrupted state, otherwise, ensure that the current thread is not in an interrupted state. 4. Perform tasks. 5. If the result of getTask is null, jump out of the loop and execute the processWorkerExit() method to destroy the thread.

The execution process is shown in the following figure: