1. Custom thread pool
1. Import pom
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.0.1-jre</version>
</dependency>
2. Create an asynchronous configuration class AsyncTaskConfig
/**
* @author wangli
* @create 2022-10-15 17:08
*/
@EnableAsync// 支持异步操作
@Configuration
public class AsyncTaskConfig {
@Bean("async-executor-spring")
public Executor AsyncExecutor(){
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
// 核心线程数
executor.setCorePoolSize(10);
// 线程池维护线程的最大数量，只有在缓冲队列满了之后才会申请超过核心线程数的线程
executor.setMaxPoolSize(50);
// 缓存队列
executor.setQueueCapacity(20);
// 空闲时间，当超过了核心线程数之外的线程在空闲时间到达之后会被销毁
executor.setKeepAliveSeconds(200);
// 异步方法内部线程名称
executor.setThreadNamePrefix("async-executor-spring");
/**
* 当线程池的任务缓存队列已满并且线程池中的线程数目达到maximumPoolSize，如果还有任务到来就会采取任务拒绝策略
* 通常有以下四种策略：
* ThreadPoolExecutor.AbortPolicy:丢弃任务并抛出RejectedExecutionException异常。
* ThreadPoolExecutor.DiscardPolicy：也是丢弃任务，但是不抛出异常。
* ThreadPoolExecutor.DiscardOldestPolicy：丢弃队列最前面的任务，然后重新尝试执行任务（重复此过程）
* ThreadPoolExecutor.CallerRunsPolicy：重试添加当前的任务，自动重复调用 execute() 方法，直到成功
*/
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
executor.initialize();
return executor;
}
/**
* 1、corePoolSize：核心线程数
* * 核心线程会一直存活，及时没有任务需要执行
* * 当线程数小于核心线程数时，即使有线程空闲，线程池也会优先创建新线程处理
* * 设置allowCoreThreadTimeout=true（默认false）时，核心线程会超时关闭
*
* 2、queueCapacity：任务队列容量（阻塞队列）
* * 当核心线程数达到最大时，新任务会放在队列中排队等待执行
*
* 3、maxPoolSize：最大线程数
* * 当线程数>=corePoolSize，且任务队列已满时。线程池会创建新线程来处理任务
* * 当线程数=maxPoolSize，且任务队列已满时，线程池会拒绝处理任务而抛出异常
*
* 4、 keepAliveTime：线程空闲时间
* * 当线程空闲时间达到keepAliveTime时，线程会退出，直到线程数量=corePoolSize
* * 如果allowCoreThreadTimeout=true，则会直到线程数量=0
*
* 5、allowCoreThreadTimeout：允许核心线程超时
* 6、rejectedExecutionHandler：任务拒绝处理器
* * 两种情况会拒绝处理任务：
* - 当线程数已经达到maxPoolSize，切队列已满，会拒绝新任务
* - 当线程池被调用shutdown()后，会等待线程池里的任务执行完毕，再shutdown。如果在调用shutdown()和线程池真正shutdown之间提交任务，会拒绝新任务
* * 线程池会调用rejectedExecutionHandler来处理这个任务。如果没有设置默认是AbortPolicy，会抛出异常
* * ThreadPoolExecutor类有几个内部实现类来处理这类情况：
* - AbortPolicy 丢弃任务，抛运行时异常
* - CallerRunsPolicy 执行任务
* - DiscardPolicy 忽视，什么都不会发生
* - DiscardOldestPolicy 从队列中踢出最先进入队列（最后一个执行）的任务
* * 实现RejectedExecutionHandler接口，可自定义处理器
*/
/**
* com.google.guava中的线程池
* @return
*/
@Bean("async-executor-guava")
public Executor GuavaAsyncExecutor() {
ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("async-executor-guava").build();
// 当前可用cpu数
//最佳线程数可通过计算得出http://ifeve.com/how-to-calculate-threadpool-size/
int curSystemThreads = Runtime.getRuntime().availableProcessors() * 2;
/**
* int corePoolSize,
* int maximumPoolSize,
* long keepAliveTime,
* TimeUnit unit,
* BlockingQueue<Runnable> workQueue,
* ThreadFactory threadFactory
*/
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(curSystemThreads, 100,
200, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(), threadFactory);
//允许核心线程超时
threadPool.allowsCoreThreadTimeOut();
return threadPool;
}
}
3. Create a controller class
/**
* @author wangli
* @create 2022-10-15 17:48
*/
@Controller
public class AsyncController {
@Autowired
private AsyncService asyncService;
@PostMapping("/AsyncMethond")
public void AsyncMethond(){
asyncService.AsyncMethond();
}
}4. Create a service class
/**
* @author wangli
* @create 2022-10-15 17:49
*/
public interface AsyncService {
public void AsyncMethond();
}
5. Create service impl
/**
* @author wangli
* @create 2022-10-15 17:49
*/
@Service
public class AsyncServiceImpl implements AsyncService {
@Override
@Async("async-executor-guava")
public void AsyncMethond() {
System.out.println("调用异步方法");
}
}Two. @Async annotation
The role of @Async is to process tasks asynchronously.
- Add @Async to the method to indicate that this method is an asynchronous method;
- Add @Async to the class to indicate that all methods in the class are asynchronous methods;
- The class using this annotation must be a class managed by Spring;
- The @EnableAsync annotation needs to be added to the startup class or configuration class for @Async to take effect;
- When using @Async, if you do not specify the name of the thread pool, that is, you do not customize the thread pool, @Async has a default thread pool and uses Spring's default thread pool SimpleAsyncTaskExecutor.
The default configuration of the default thread pool is as follows:
- Default number of core threads: 8;
- Maximum number of threads: Integet.MAX_VALUE;
- The queue uses LinkedBlockingQueue;
- The capacity is: Integet.MAX_VALUE;
- Idle thread retention time: 60s;
- Thread pool rejection policy: AbortPolicy;
As can be seen from the maximum number of threads, in the case of concurrency, unlimited threads will be created
It can also be reconfigured via yml:
spring:
task:
execution:
pool:
max-size: 10
core-size: 5
keep-alive: 3s
queue-capacity: 1000
thread-name-prefix: my-executor3. Why asynchrony fails
- The method annotated @Async is not a public method;
- The return value of annotation @Async can only be void or Future;
- Annotation @Async methods using static modification will also fail;
- No @EnableAsync annotation added;
- The caller and @Async cannot be in the same class;
- It is useless to mark @Transactional on the Async method, but it is valid to mark @Transcational on the method called by the Async method;
4. Thread pool execution process
5. How to reasonably plan the size of the thread pool
Although this question may seem trivial, it is not so easy to answer. If you have a better method, welcome to enlighten me. Let’s start with a naive estimation method: Assume that a system’s TPS (Transaction Per Second or Task Per Second) is required to be at least 20, and then assume that each Transaction is completed by one thread, and continue to assume the average The time for each thread to process a Transaction is 4s. The problem then becomes:
How to design the thread pool size so that 20 Transactions can be processed within 1s?
The calculation process is very simple. The processing capacity of each thread is 0.25TPS, so to achieve 20TPS, obviously 20/0.25=80 threads are needed.
Obviously this estimation method is naive, because it does not take into account the number of CPUs. Generally, the number of CPU cores of a server is 16 or 32. If there are 80 threads, it will definitely bring too much unnecessary thread context switching overhead.
Here is the second simple method but I don't know if it is feasible (N is the total number of CPU cores):
- If it is a CPU-intensive application, the thread pool size is set to N+1
- If it is an IO-intensive application, the thread pool size is set to 2N+1
If only one application and one thread pool is deployed on one server, then this estimate may be reasonable, and you need to test and verify it yourself.
Next, find an estimation formula in this document: Server Performance IO Optimization:
1 |
最佳线程数目 = （（线程等待时间+线程CPU时间）/线程CPU时间 ）* CPU数目 |
For example, the average CPU running time of each thread is 0.5s, while the thread waiting time (non-CPU running time, such as IO) is 1.5s, and the number of CPU cores is 8, then it can be estimated according to the above formula: ((0.5+1.5)/ 0.5)*8=32. This formula further translates into:
1 |
最佳线程数目 = （线程等待时间与线程CPU时间之比 + 1）* CPU数目 |
A conclusion can be drawn:
The higher the percentage of thread wait time, the more threads are required. The higher the percentage of thread CPU time, the fewer threads are required.
The previous estimation method is also consistent with this conclusion.
The fastest part of a system is the CPU, so it is the CPU that determines the upper limit of a system's throughput. Enhanced CPU processing capability can increase the upper limit of system throughput. However, according to the short-board effect, the real system throughput cannot be calculated purely based on the CPU. To improve system throughput, we need to start from the "system short board" (such as network delay, IO):
- Try to increase the parallelization ratio of short board operations, such as multi-threaded download technology
- Enhance short board capabilities, such as replacing IO with NIO
The first one can be related to Amdahl's law, which defines the formula for calculating the speedup ratio of a serial system after parallelization:
1 |
加速比=优化前系统耗时 / 优化后系统耗时 |
The larger the speedup ratio, the better the optimization effect of system parallelization. Addahl's law also gives the relationship between the system parallelism, the number of CPUs, and the speedup ratio. The speedup ratio is Speedup, the system serialization ratio (referring to the proportion of serially executed code) is F, and the number of CPUs is N:
1 |
Speedup <= 1 / (F + (1-F)/N) |
When N is large enough, the smaller the serialization ratio F is, the larger the speedup ratio Speedup will be.
As I write this, I suddenly have a question.
Is using a thread pool necessarily more efficient than using a single thread?
The answer is no. For example, Redis is single-threaded, but it is very efficient, and basic operations can reach 100,000 levels/s. From the perspective of threads, this is partly due to:
- Multi-threading brings thread context switching overhead, and single-threading does not have this overhead
- Lock
Of course, the more essential reason for "Redis is fast" is that Redis is basically a memory operation. In this case, a single thread can utilize the CPU very efficiently. The applicable scenario of multi-threading is generally: there is a considerable proportion of IO and network operations.
So even with the above simple estimation method, it may seem reasonable, but it may not be reasonable in fact. It needs to be combined with the real situation of the system (such as IO-intensive or CPU-intensive or pure memory operation) and hardware environment (CPU, Memory, hard disk read and write speed, network conditions, etc.) to constantly try to reach a realistic and reasonable estimate.