The use of thread pools as a Java development is an unavoidable hurdle. How to use thread pools correctly is a problem faced by every development. Today we will look at how thread pools are used in top open source projects from open source projects. . Let's take the example of RocketMQ, an open source project that I recently participated in, and combine some bad habits I encountered in my work to talk about the use of thread pools. From the following aspects:
1. Creation of thread pool
First, let's take a look at the thread creation method of RocketMQ, taking the BrokerController#initializeResourcesmethod as an example. As shown below:
At first glance, it seems to be a custom thread pool. If you look at the source code, you will find that it is BrokerFixedThreadPoolExecutoractually implemented. In ThreadPoolExecutoressence, it is ThreadPoolExecutorjust semantic. Analysis of the above thread pool creation will find the following characteristics:
-
The thread pool will be set with a name (recommended setting)
Why do I need to set the name? The main reason is to know which code fragment executed by which thread pool has the problem when troubleshooting the problem. If you set the name to use the default name of ThreadPoolExecutor, you will not know which thread pool is when troubleshooting.
-
Set the number of threads in the thread pool
Here, it will be judged whether the threads of this thread pool need to be expanded according to the number of core threads and the maximum number of threads. For example, the number of core threads is 10, and the maximum number of thread pools is 100. Then the thread pool will continue after it exceeds the number of core threads during operation. Create threads to satisfy task execution. It can be seen that the core thread pool and the maximum thread pool in RocketMQ are generally set to be the same size, that is, a fixed-size thread pool.
-
Customize the set capacity of the task queue (recommended setting)
Many are used in RocketMQ
LinkedBlockingQueue, and then set the capacity for the queue. Important: Queue selection is based on individual needs, but be sure to set a reasonable capacity for the queue. If you don't set the capacity then the default capacity isInteger.MAX_VALUE, which may largely lead to the service downtime due to running out of memory before triggering the task rejection policy. -
Task Rejection Policy Selection
Task rejection strategy selection In general, the default strategy of JDK ThreadPoolExecutor can be used. If there are special requirements, users can customize the strategy.
The above is the creation of the thread pool as seen from the RocketMQ open source project. In fact, many people will find that Executorscreate . If you want to learn more, you can read the article "Why is it not recommended to use Executors to create thread pool analysis" . The author gives an analysis in this article.
2. Use of thread pool
People who have studied the RocketMQ source code will find that RocketMQ has many thread pools, and many people will definitely wonder why not use one thread pool to complete all tasks. The author's task is mainly due to the following reasons.
2.1 The principle of unity
How to understand the principle of unity? That is, a thread pool should do a class of task processing. BrokerController#initializeResourcesFrom the code in , it can be seen that different tasks are processed using different thread pools. The benefits of doing this:
- It is convenient to trace back the thread execution problem. When an error occurs in the execution of the thread pool, it is necessary to trace the singleness according to the error stack to facilitate the traceback. If all tasks use a thread pool to execute, you won't know which task has the problem by the thread pool name.
- Ability to better assess queue size
- If a problem occurs during the execution of the thread pool, the thread pool shutdown or other problems will only affect one type of task and will not affect all tasks.
2.2 Proper packaging
Proper encapsulation of the thread pool can make the code look more elegant while satisfying security, which is the semantics mentioned above
3. Incorrect use of thread pool
In the author's previous article, "What Happens When the Java Thread Pool Is Used Improperly - Production Case", I gave a company production case, which is to create a thread pool in the method called by the Controller and then perform asynchronous tasks. What happens in this case is that a thread pool will be created every time and then the thread pool will not be destroyed, so every time the method is called, at least one thread will be created. If the method is called 1000 times, 1000 threads will be created.
@Service
@Slf4j
public class AccountAuthServiceImpl implements AccountAuthService {
//省略了部分代码
//功能：将两个系统账号进行绑定
@Override
public boolean bindUser(Long hrsAccountId, Long hmcAccountId){
ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()*2);
Future<Boolean> hrsExt = executorService.submit(new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
//查询hrsAccountId是否存在
return true;
}
});
Future<Boolean> hmcExt = executorService.submit(new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
//查询hmcAccountId是否存在
return true;
}
});
try {
return hrsExt.get(3, TimeUnit.SECONDS) & hmcExt.get(3, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
return false;
}
}
4. Summary
- Thread pools and threads must be set with names familiar to developers to facilitate troubleshooting when problems occur. This can be found through commands in other projects and Java. Thread naming is a very important step. Although Java provides default names, custom names are especially important in troubleshooting
- The task queue capacity of the thread pool must be set, otherwise it may cause memory consumption and service downtime
- The creation of the thread pool cannot be created in the business method