I. Introduction
First of all, if you are busy and drop in, you can bookmark it first, and then read it when you have time or use it;
I believe many people will have a confusion, which is the same as mine before, which is the thread pool. It is very tall, very fashionable to use, and there is no problem in debugging in the local environment test environment, but problems occur as soon as it goes online.
Then Baidu has a lot of information and found that they are talking about the need to customize the thread pool and various configuration parameters.
In the final analysis, it is caused by ignorance of the configuration parameters of the custom thread pool. This article uses a very simple case to clarify the configuration of the thread pool and how to configure the online environment.
Second, the case
1. Write a case
Customize a thread pool and add initial configuration.
The number of core threads is 10, the maximum number of threads is 50, the queue size is 200, the prefix of the custom thread pool name is my-executor-, and the thread pool rejection policy is AbortPolicy, which is also the default policy, indicating that the task is abandoned directly.
package com.example.executor.config;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;
@Configuration
@EnableAsync
@EnableScheduling
@Slf4j
public class AsyncConfiguration {
/**
* 自定义线程池
*/
@Bean(name = "myExecutor")
public Executor getNetHospitalMsgAsyncExecutor() {
log.info("Creating myExecutor Async Task Executor");
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(200);
executor.setThreadNamePrefix("my-executor-");
// 拒绝策略：直接拒绝抛出异常
executor.setRejectedExecutionHandler(
new ThreadPoolExecutor.AbortPolicy());
return executor;
}
}
Next, we write an asynchronous service, directly use this custom thread pool, and simulate a message sending service that takes 5 seconds.
package com.example.executor.service;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;
/**
* <p>
* 异步服务
* </p>
*
* @author 福隆苑居士，公众号：【Java分享客栈】
* @since 2022/4/30 11:41
*/
@Service
@Slf4j
public class AsyncService {
/**
* 模拟耗时的发消息业务
*/
@Async("myExecutor")
public void sendMsg() throws InterruptedException {
log.info("[AsyncService][sendMsg]>>>> 发消息....");
TimeUnit.SECONDS.sleep(5);
}
}
Then, we write a TestService, use the concurrency tool that comes with Hutools to call the above messaging service, and set the number of concurrency to 200, that is, open 200 threads at the same time to execute the business.
package com.example.executor.service;
import cn.hutool.core.thread.ConcurrencyTester;
import cn.hutool.core.thread.ThreadUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
/**
* <p>
* 测试服务
* </p>
*
* @author 福隆苑居士，公众号：【Java分享客栈】
* @since 2022/4/30 11:45
*/
@Service
@Slf4j
public class TestService {
private final AsyncService asyncService;
public TestService(AsyncService asyncService) {
this.asyncService = asyncService;
}
/**
* 模拟并发
*/
public void test() {
ConcurrencyTester tester = ThreadUtil.concurrencyTest(200, () -> {
// 测试的逻辑内容
try {
asyncService.sendMsg();
} catch (InterruptedException e) {
log.error("[TestService][test]>>>> 发生异常: ", e);
}
});
// 获取总的执行时间，单位毫秒
log.info("总耗时：{}", tester.getInterval() + " ms");
}
}
Finally, write a test interface.
package com.example.executor.controller;
import com.example.executor.service.TestService;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* <p>
* 测试接口
* </p>
*
* @author 福隆苑居士，公众号：【Java分享客栈】
* @since 2022/4/30 11:43
*/
@RestController
@RequestMapping("/api")
public class TestController {
private final TestService testService;
public TestController(TestService testService) {
this.testService = testService;
}
@GetMapping("/test")
public ResponseEntity<Void> test() {
testService.test();
return ResponseEntity.ok().build();
}
}
2. Execution sequence
After the case is finished, we are about to start the test of calling the thread pool, but before that, let me explain to you how the configuration of the custom thread pool is executed during the running process, and in what order, this is clear, You won't be confused by adjusting the parameters later.
Number of core threads (CorePoolSize) —> (if all are occupied) —> put into the queue (QueueCapacity) —> (if all are occupied) —> create a new thread according to the maximum number of threads (MaxPoolSize) —> (if the maximum number of threads is exceeded ) —> Start executing the rejection policy (RejectedExecutionHandler)
Watch it three times in a row, and then you will.
3. How to configure the number of core threads
Let's run the program first, and here we will explain the important clues of the above case again for everyone to hear.
1) The number of core threads in the thread pool is 10, the maximum number of threads is 50, and the queue is 200;
2) It takes 5 seconds to send messages;
3) The number of concurrent tool execution threads is 200.
As you can see in the figure below, all 200 threads have been executed. You can observe the time on the left, and 10 threads will be executed every 5 seconds. I can clearly find that it is very slow to execute all 200 threads in the background operation.
It can be seen that 10 core threads are executed first, and the remaining 190 are placed in the queue, and our queue size is 200, so the maximum number of threads does not work.
Thinking: How to improve the execution efficiency of 200 threads? The answer is already obvious, because our business is a time-consuming business that takes 5 seconds. If the number of core threads is configured less, the execution of all 200 threads will be very slow, so we only need to increase the number of core threads. .
We adjust the number of core threads to 100
@Bean(name = "myExecutor")
public Executor getNetHospitalMsgAsyncExecutor() {
log.info("Creating myExecutor Async Task Executor");
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(100);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(200);
executor.setThreadNamePrefix("my-executor-");
// 拒绝策略：直接拒绝抛出异常
executor.setRejectedExecutionHandler(
new ThreadPoolExecutor.AbortPolicy());
// 拒绝策略：调用者线程执行
// executor.setRejectedExecutionHandler(
// new ThreadPoolExecutor.CallerRunsPolicy());
return executor;
}
Look at the effect: Huh? Reported an error?
Why, just look at the source code.
It turns out that when the thread pool is initialized, internal judgments are made. If the maximum number of threads is less than the number of core threads, this exception will be thrown, so we must pay special attention when setting, at least the number of core threads must be greater than or equal to the maximum number of threads.
Let's modify the configuration: the number of core threads and the maximum number of threads are both set to 100.
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(100);
executor.setMaxPoolSize(100);
executor.setQueueCapacity(200);
executor.setThreadNamePrefix("my-executor-");
Look at the effect: During the background running process, it can be found that the running speed is very fast, at least 10 times higher than before, and 200 threads will finish running in a while.
Reason: We set the time-consuming business for 5 seconds, and the number of core threads is only 10. Then the threads waiting in the queue will execute the time-consuming business in batches, and each batch of 5 seconds will be very slow. When we put the core After the number of threads is increased, it is equivalent to only executing one or two batches to complete the 5-second business, and the speed is naturally doubled.
Here we can draw the first conclusion:
If your business is time-consuming, the number of core threads in the thread pool configuration should be increased.
think for a while:
What kind of business is suitable for configuring a smaller number of core threads and a larger queue?
4. How to configure the maximum number of threads
Next, let's take a look at what is the maximum number of threads. This is interesting. A lot of information on the Internet is wrong.
Still the previous case, for clarity, let’s adjust the configuration parameters: the number of core threads is 4, the maximum number of threads is 8, and the queue is only 1.
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(8);
executor.setQueueCapacity(1);
executor.setThreadNamePrefix("my-executor-");
Then we changed the number of concurrent tests to 10.
ConcurrencyTester tester = ThreadUtil.concurrencyTest(10, () -> {
// 测试的逻辑内容
try {
asyncService.sendMsg();
} catch (InterruptedException e) {
log.error("[TestService][test]>>>> 发生异常: ", e);
}
});
Start, test:
Surprise discovery, huh? There are 10 concurrent numbers, how come only 9 are executed, and where is the other one going?
Let's change the maximum number of threads to 7 and try again
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(7);
executor.setQueueCapacity(1);
executor.setThreadNamePrefix("my-executor-");
Look at it again, and found that only 8 of them were executed. Now that it’s done, 2 of them have disappeared...
Why, I will demonstrate the specific demonstration effect in the following rejection strategy. Here I will directly tell you the conclusion:
What does the maximum number of threads mean in the thread pool? Yes, it means literally. When the number of core threads is full, the queue is also full, and the remaining threads execute tasks through new threads created by the maximum number of threads. This process has been sorted out for everyone at the beginning.
But listen carefully, because it is the maximum number of threads, so the execution thread will not exceed this number, and if it exceeds this number, it will be rejected by the rejection policy.
Now let's sort it out again based on the configuration parameters at the beginning of this section. 10 concurrent numbers, 4 occupy core threads, 1 enters the queue, and the maximum number of threads is configured to be 8. In the current 2-second business time, The total number of active threads is:
the number of core threads (4) + the number of newly created threads (?) = the maximum number of threads (8). It
can be seen that because the maximum number of threads is configured to be 8, after the number of core threads and queues are full, the newly created The number of threads can only be 8-4=4, so the final execution is:
the number of core threads (4) + the number of newly created threads (4) + the number of threads in the queue (1) = 9 There
is no problem at all, and the remaining The next one exceeded the maximum number of threads 8 and was rejected by the reject policy.
Finally, a picture gives you a clear picture. Pay attention to the time on the left, and you will know that the last one is the thread that will be executed after 2 seconds in the queue.
Here, we can also draw the second conclusion:
The maximum number of threads is the literal meaning. The current active thread cannot exceed this limit. If it exceeds this limit, it will be rejected by the rejection policy.
5. How to configure the queue size
The first two are understood, and the queue size can actually be understood with a simple test.
We modify the previous thread pool configuration:
the number of core threads is 50, the maximum number of threads is 50, the queue is 100, and the business time is changed to 1 second to facilitate testing.
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(50);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(100);
executor.setThreadNamePrefix("my-executor-");
The number of concurrency is set to 200
ConcurrencyTester tester = ThreadUtil.concurrencyTest(200, () -> {
// 测试的逻辑内容
try {
asyncService.sendMsg();
} catch (InterruptedException e) {
log.error("[TestService][test]>>>> 发生异常: ", e);
}
});
Test the effect: It can be seen that only 150 of the 200 concurrent numbers are executed in the end. The maximum number of threads in the specific algorithm has been mentioned in the previous section and will not be repeated.
Here we mainly make it clear that after the current number of threads exceeds the queue size, the maximum number of threads will be used to calculate and create new threads to execute the business. Then we might as well think about whether it is enough to set the queue to be larger, so that it will not Will take the maximum number of threads again.
Let's adjust the queue size from 100 to 500 to see
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(50);
executor.setMaxPoolSize(50);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("my-executor-");
Test results: As you can see, all 200 of them have been executed, which shows that our assumption is correct.
A third conclusion can be drawn here:
If the queue size is set reasonably, there is no need to use the maximum number of threads to cause additional overhead, so the best way to configure the thread pool is to match the number of core threads with the queue size.
6. How to match the rejection strategy
In the previous section on how to configure the maximum number of threads, after testing, it can be found that some threads are directly rejected after exceeding the maximum number of threads, because we have configured a rejection policy at the beginning. This policy is the default policy of the thread pool, which means direct rejection.
// 拒绝策略：直接拒绝抛出异常
executor.setRejectedExecutionHandler(
new ThreadPoolExecutor.AbortPolicy());
So how do we know that these threads are indeed rejected, here we restore the parameter configuration in the maximum number of threads section.
Then, change the default policy to another policy: CallerRunsPolicy, which means that the caller thread will continue to execute after rejection.
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(7);
executor.setQueueCapacity(1);
executor.setThreadNamePrefix("my-executor-");
// 拒绝策略：调用者线程执行
executor.setRejectedExecutionHandler(
new ThreadPoolExecutor.CallerRunsPolicy());
return executor;
The number of concurrency is changed to 10
ConcurrencyTester tester = ThreadUtil.concurrencyTest(10, () -> {
// 测试的逻辑内容
try {
asyncService.sendMsg();
} catch (InterruptedException e) {
log.error("[TestService][test]>>>> 发生异常: ", e);
}
});
Test results:
It can be seen that all 10 concurrent numbers have been executed, and in the section on the maximum number of threads, 2 threads were rejected by the default strategy when we tested, because the strategy has now been changed to let the caller thread continue to execute the task, so the 2 threads Although rejected, it is still executed by the caller thread.
You can see that the names of the two threads in the red line in the figure are obviously different from those of the custom thread, which is the caller thread to execute.
So, is this strategy so humane that it must be good?
NO! This strategy is uncontrollable. If it is an Internet project, it is easy to go wrong online. The reason is very simple.
What the thread pool occupies is not the main thread, but an asynchronous operation to perform tasks. This strategy actually re-hands the rejected thread to the main thread for execution, which is equivalent to changing asynchronous to synchronous. Just imagine, in peak traffic stage, if a large number of asynchronous threads leave the main thread because of this strategy, what is the consequence? It is likely to cause the program of your main thread to crash, and then form a service avalanche.
Show the four strategies provided by the thread pool:
1), AbortPolicy: the default policy, which directly rejects and throws a RejectedExecutionException;
2), CallerRunsPolicy: the caller thread continues to execute the task, a simple feedback mechanism policy;
3), DiscardPolicy: directly discards the task without any notification and feedback;
4), DiscardOldestPolicy: Discard an old task, usually the one with the longest survival time.
Many people think that the CallerRunsPolicy strategy is the most complete, but in my personal opinion, the default strategy is actually the lowest risk in the production environment, and our online projects tend to prioritize security.
Speaking of this, based on the case, basically everyone can understand the meaning of these thread pool parameters, so do you still remember a thinking question I sent earlier? I don’t remember it, because everyone is a fish’s memory. The thinking question is:
What kind of business is suitable for configuring a smaller number of core threads and a larger queue?
Answer: Low time consumption and high concurrency scenarios are very suitable, because low time consumption belongs to millisecond-level business, which is more suitable for CPU and memory. Queue buffering is required when high concurrency is high, and because of low time consumption, it will not be in Waiting in the queue for a long time, a large number of core threads will increase the CPU overhead at one time, so configuring a smaller number of core threads and a larger queue is very suitable for this scenario.
As an aside, those who have used cloud products will know that when you buy a cloud server, you will always be asked to choose CPU-intensive or IO-intensive models. If you know more about thread pools, you will know what it means , the server models that need to be matched with different projects are actually considered. In the above scenario, it is obvious to choose a CPU-intensive server. However, the case scenarios in the previous chapters are time-consuming and suitable for IO-intensive servers.
3. Summary
In addition to the summary of this chapter, there are additional points added, which come from my work experience.
1) If your business is a time-consuming business, the number of core threads in the thread pool configuration should be increased, and the queue should be appropriately reduced; 2)
If your business is a low-time-consuming business (millisecond level), at the same time If the traffic is large, the number of core threads in the thread pool configuration should be reduced, and the queue should be increased appropriately;
3), the maximum number of threads is the literal meaning, the current active thread cannot exceed this limit, if it exceeds this limit, it will be rejected by the policy Reject it;
4) If the queue size is set reasonably, there is no need to use the maximum number of threads to cause additional overhead, so the best way to configure the thread pool is to match the number of core threads with the queue size; 5), the thread pool rejection strategy should be based on the default as much as
possible , reduce the risk of the production environment, and do not change unless necessary;
6), the total number of thread pools for projects or microservices deployed in the same server should not exceed 5, otherwise life and death will depend on life and wealth; 7)
, Do not use the thread pool indiscriminately, distinguish business scenarios clearly, and try to use them in scenarios that can be delayed and are not particularly important, such as sending messages here, or sending subscription notifications, or doing log records for a certain scenario, etc. It is easy to use the thread pool in the core business;
8), do not mix thread pools, remember to isolate specific businesses, that is, customize their own thread pools, different names and different parameters, you can imagine that you wrote a thread pool at will, configure I found the appropriate parameters for my own business, but it was used by another colleague in a business with a large amount of concurrency. At that time, I can only have the same difficulties; 9), thread pool
configuration is not a treat for dinner, even if you are familiar with it , please still do a stress test before going online, this is a painful lesson for me;
10), please be sure to clarify the application scenario of the thread pool, and do not confuse it with the high-concurrency processing solution, the two business directions are completely targeted different.
4. Sharing
Finally, I will share with you a formula that I have used in my previous work. It is only for scenarios where the current threads of the specific business of small and medium-sized enterprises are more than thousands of levels. After all, I have never worked in a large factory, and the experience I can share is limited. .
Take our company as an example. We are a small and medium-sized Internet company. We use Huawei Cloud, and the online servers are basically 8 cores. I usually use the thread pool for specific businesses to test with the current number of threads being 2000, because 2000 threads are running at the same time. A concurrent thread is not as easy to appear in small and medium-sized enterprises as everyone thinks. Our company serves hospitals, and we don't meet them a few times a year, except for the surge in the number of nucleic acids due to the epidemic in the past two years.
You can imagine for yourself, 2,000 threads are processing a certain business at the same time, how many users are required, and what kind of scene will appear. The key is that you use the thread pool. Why do you use the thread pool itself in this scene? It is also something to reflect on. Some similar scenarios use caching and MQ to cut peaks. This is why I said in my summary not to be confused with high-concurrency processing solutions. You should use thread pools when delay processing is required. It is most suitable for less important business.
The formula I summarized can be obtained from here:
Link: https://pan.baidu.com/doc/share/TES95Wnsy3ztUp_Al1L~LQ-567189327526315
Extraction code: 2jjy
My original article is purely hand-written, if you think it is useful, please give it a thumbs up .
From time to time, I will share the experience and interesting things in the actual work. If you are interested, please pay attention~