Article directory
- Flyweight mode
- Why do you need a thread pool?
- Custom thread pool
- Thread pool in JDK
Flyweight mode
It is a flyweight pattern that belongs to the 23 design patterns . Please see this blog for the specific definition.
Specifically, the wrapper classes
in the JDK provide the valueOf method such as Boolean, Byte, Short, Integer, Long, and Character. For example, the valueOf of Long will cache Long objects between -128~127, and objects will be reused between this range. , if it is greater than this range, a new Long object will be created.
public static Long valueOf(long l) {
final int offset = 128;
if (l >= -128 && l <= 127) {
// will cache
return LongCache.cache[(int)l + offset];
}
return new Long(l);
}
Note:
Byte, Short, Long cache range is -128~127
Character cache range is 0~127
Integer default range is -128~127
- The minimum value cannot be changed
- But the maximum value can be
-Djava.lang.Integer.IntegerCache.highchanged by adjusting virtual machine parameters
Boolean cached TRUE and FALSE
Handwritten database connection pool
For example: an online shopping mall application, the QPS reaches thousands. If the database connection is re-created and closed every time, the performance will be greatly affected. At this time, a batch of connections are created in advance and put into the connection pool. After a request arrives, the connection is obtained from the connection pool and returned to the connection pool after use. This not only saves the creation and closing time of the connection, but also realizes the reuse of the connection, so that the huge number of connections will not overwhelm the database.
Why do you need a thread pool?
-
Although creating and destroying threads is relatively small (compared to processes), when there are a large number of threads in the system, this overhead is considerable, so the thread pool is introduced.
-
The purpose is to allow certain objects to be reused multiple times and reduce the overhead caused by frequent creation and destruction of objects (these objects must be reusable)
-
Analogy to the database pool, creating and destroying database connections is a relatively time-consuming operation, which means that if the current user no longer uses this connection, it will be recycled to the connection pool (the same connection can be used multiple times by multiple users, reducing the number of System overhead of creating and destroying connections)
-
Similarly, different threads have different contents of the run method, but the general process of the thread is the same. Therefore, in order to avoid the overhead caused by repeatedly creating and destroying threads, threads can be "reused" (the biggest benefit of the thread pool It is to reduce the loss of starting and destroying threads each time and provide high time and space utilization.) Several threads are created inside the thread pool. These threads are all runnable. You only need to remove the task from the system and you can start execution immediately.
There are three main reasons for thread pools:
- Creating/destroying threads consumes system resources, and the thread pool can reuse created threads.
- Control the amount of concurrency. Too many concurrencies may cause excessive resource consumption and cause the server to crash. (main reason)
- Threads can be managed uniformly.
Custom thread pool
- The thread in our thread pool is the corresponding consumer, the main thread is the producer, the consumer takes the task from the blocking queue, and the main thread fills in the task
Custom rejection policy interface
@FunctionalInterface // 拒绝策略
interface RejectPolicy<T> {
void reject(BlockedQueue<T> queue, T task);
}
- What is a rejection policy? If there are many tasks and our thread pool cannot handle them, that is, our blocking queue is full, then we need a corresponding processing method. The method here is the rejection policy.
- Why do we need to define an interface here? We know that different scenarios require different strategies. If we write it in our blocking queue, it will not comply with the opening and closing principle, and there will also be a lot of if-else
- So the strategy pattern is adopted - the interface here is the abstract strategy class that implements specific strategies under specific conditions.
Custom task queue
public class BlockedQueue<T> {
//1任务队列
private Deque<T> queue = new ArrayDeque<>();
//2 锁
private ReentrantLock lock = new ReentrantLock();
//3 生产者条件变量
private Condition fullWaitSet=lock.newCondition();
//4消费者条件变量
private Condition emptyWaitSet=lock.newCondition();
//5容量
private int capcity;
public BlockedQueue(int capcity) {
this.capcity = capcity;
}
//阻塞获取
public T take(){
lock.lock();
try {
while (queue.isEmpty()){
try {
emptyWaitSet.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
T t = queue.removeFirst();
fullWaitSet.signal();
return t;
}finally {
lock.unlock();
}
}
//带时间的阻塞获取
public T poll(long timeOut, TimeUnit unit){
lock.lock();
try {
long nanos = unit.toNanos(timeOut);
while (queue.isEmpty()){
try {
if(nanos<=0){
return null;
}
nanos = emptyWaitSet.awaitNanos(nanos);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
T t = queue.removeFirst();
fullWaitSet.signal();
return t;
}finally {
lock.unlock();
}
}
//阻塞添加
public void put(T task){
lock.lock();
try {
while (queue.size() == capcity){
try {
System.out.println("等待加入任务队列"+task);
fullWaitSet.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("加入了任务队列"+task);
queue.addLast(task);
emptyWaitSet.signal();
}finally {
lock.unlock();
}
}
//带超时时间阻塞添加
public boolean offer(T task, long timeOut, TimeUnit timeUnit){
lock.lock();
try {
long nanos = timeUnit.toNanos(timeOut);
while (queue.size()== capcity){
if (nanos<=0){
return false;
}
System.out.println("等待加入任务队列"+task);
try {
nanos = fullWaitSet.awaitNanos(nanos);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("加入了任务队列"+task);
queue.addLast(task);
emptyWaitSet.signal();
return true;
}finally {
lock.unlock();
}
}
public int size() {
lock.lock();
try {
return queue.size();
} finally {
lock.unlock();
}
}
public void tryPut(RejectPolicy<T> rejectPolicy,T task){
lock.lock();
try {
if(queue.size() == capcity){
rejectPolicy.reject(this,task);
}else {
queue.addLast(task);
emptyWaitSet.signal();
}
}finally {
lock.unlock();
}
}
}
- Because ArrayDeque is not thread-safe, it needs to be locked
Custom thread pool
public class ThreadPool {
//任务阻塞队列
private BlockedQueue<Runnable> taskQueue;
//线程集合 作为线程池的容器
//这里的Worker使用了适配器模式
private HashSet<Worker> workers = new HashSet<>();
//核心线程池
private int coreSize;
//获取任务时的超时时间
private long timeOut;
//进行时间转换
private TimeUnit timeUnit;
//具体的策略实现
private RejectPolicy<Runnable> rejectPolicy;
public ThreadPool(int coreSize, long timeout, TimeUnit timeUnit, int queueCapcity,
RejectPolicy<Runnable> rejectPolicy) {
this.coreSize = coreSize;
this.timeOut = timeout;
this.timeUnit = timeUnit;
this.taskQueue = new BlockedQueue<>(queueCapcity);
this.rejectPolicy = rejectPolicy;
}
public void execute(Runnable task){
// 当任务数没有超过 coreSize 时，直接交给 worker 对象执行
// 如果任务数超过 coreSize 时，加入任务队列暂存
synchronized (workers){
if(workers.size() < coreSize){
Worker worker = new Worker(task);
workers.add(worker);
worker.start();
}else {
//1死等
//2带超时等待
//3让调用这放弃任务执行
//4让调用者抛出异常
//5让调用者自己执行任务
taskQueue.tryPut(rejectPolicy,task);
}
}
}
class Worker extends Thread{
private Runnable task;
public Worker(Runnable task){
this.task=task;
}
@Override
public void run() {
//执行任务
//1当task不为空 执行任务
//2当task执行完毕，再接着从任务队列获取任务并执行
while (task != null || (task = taskQueue.poll(timeOut,timeUnit))!= null){
try {
task.run();
}catch (Exception e){
e.printStackTrace();
}finally {
task=null;
}
}
}
}
}
test
public class PoolTest {
public static void main(String[] args) {
ThreadPool threadPool = new ThreadPool(1,1000, TimeUnit.MILLISECONDS,1,
(queue, task) ->{
//1死等
queue.put(task);
//2超时等待
// queue.poll(1500,TimeUnit.MILLISECONDS);
// 3) 让调用者放弃任务执行
// log.debug("放弃{}", task);
// 4) 让调用者抛出异常
// throw new RuntimeException("任务执行失败 " + task);
// 5) 让调用者自己执行任务
// task.run();
});
for (int i = 0; i < 4; i++) {
int j = i;
threadPool.execute(() -> {
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(j);
});
}
}
}
Thread pool in JDK
The relationship between commonly used thread pool classes and interfaces
Thread pool status
ThreadPoolExecutor uses the high 3 bits of int to represent the thread pool status, and the low 29 bits represent the number of threads.
| status name | High 3 digits | receive new tasks | Handle blocking queue tasks | illustrate |
|---|---|---|---|---|
| RUNNING | 111 | Y | Y | |
| SHUTDOWN | 000 | N | Y | New tasks will not be received, but remaining tasks in the blocking queue will be processed. |
| STOP | 001 | N | N | Will interrupt the task being executed and abandon the blocking queue task |
| TIDYING | 010 | - | - | All tasks have been executed, the active thread is 0 and is about to be terminated. |
| TERMINATED | 011 | - | - | terminal state |
Comparing numerically, TERMINATED > TIDYING > STOP > SHUTDOWN > RUNNING
- Because the highest bit is the sign bit, representing negative numbers
This information is stored in an atomic variable ctl. The purpose is to combine the thread pool status and the number of threads into one, so that a CAS atomic operation can be used for assignment.
// c 为旧值， ctlOf 返回结果为新值
ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))));
// rs 为高 3 位代表线程池状态， wc 为低 29 位代表线程个数，ctl 是合并它们
private static int ctlOf(int rs, int wc) {
return rs | wc; }
Construction method
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler)
- corePoolSize Number of core threads (maximum number of threads reserved)
- maximumPoolSize maximum number of threads
- Number of emergency threads = maximum number of threads - number of core threads
- keepAliveTime survival time - for emergency threads
- unit time unit - for emergency threads
- workQueue blocking queue
- threadFactory thread factory - you can give a good name when creating a thread
- handler rejection strategy - a strategy for when our blocking queue is also full
Thread pool workflow
- There are no threads in the thread pool at the beginning. When a task is submitted to the thread pool, the thread pool will create a new thread to execute the task.
- When the number of threads in the thread pool is less than the number of core threads, a new thread will be opened to handle the task.
- When the number of threads reaches corePoolSize and no thread is idle, a task is added at this time. The newly added task will be added to the workQueue queue until there is an idle thread.
- If the queue selects a bounded queue, when the task exceeds the queue size, maximumPoolSize - corePoolSize number of threads will be created for emergency relief.
- If the thread reaches maximumPoolSize and there are still new tasks, the rejection policy will be executed. Denial strategy jdk provides 4 implementations, and other famous frameworks also provide implementations
- AbortPolicy lets the caller throw RejectedExecutionException, which is the default policy
- CallerRunsPolicy lets the caller run tasks
- DiscardPolicy abandon this task
- DiscardOldestPolicy discards the oldest task in the queue and replaces it with this task
- Dubbo's implementation will record logs and dump thread stack information before throwing RejectedExecutionException to facilitate problem location.
- Netty's implementation is to create a new thread to perform tasks
- Implementation of ActiveMQ, with timeout waiting (60s) to try to put into the queue, similar to our previously customized rejection policy
- The implementation of PinPoint uses a rejection policy chain and will try each rejection policy in the policy chain one by one. After the peak has passed, if the emergency thread that exceeds corePoolSize has no tasks to do for a period of time, it needs to end to save resources. This time is determined by keepAliveTime and unit to control
Deny policy
- CallerRunsPolicy is returned to the caller of the thread for processing
- AbortPolicy directly rejects tasks that exceed the load and throws an exception
- DiscardOldestPolicy: Discard the oldest task in the queue (the task with the longest queue)
- DisCardPolicy: Discard new tasks
executors
This is a tool class that implements the four major thread pools
The difference between Executor and Executors in Java
- Different methods of the Executors tool class create different thread pools according to our needs to meet business needs. The Executor interface object can execute our thread tasks.
- The ExecutorService interface inherits and extends the Executor interface, providing more methods to obtain the status of task execution and the return value of the task. Use ThreadPoolExecutor to create a custom thread pool
newFixedThreadPool
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
Features - fixed size thread count
- Number of core threads == maximum number of threads (no emergency threads are created), so no timeout is required
- The blocking queue is unbounded and can hold any number of tasks.
Evaluation: Suitable for tasks with known workload and relatively time-consuming tasks
newCachedThreadPool
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
Features - dynamically changing thread pool
- The number of core threads is 0, the maximum number of threads is Integer.MAX_VALUE, and the idle survival time of the emergency thread is 60s.
- It means that all are emergency threads (can be recycled after 60s)
- Emergency threads can be created unlimitedly
- The queue uses SynchronousQueue. The characteristic of the implementation is that it has no capacity and cannot be put in without a thread to take it (pay with one hand and deliver with the other).
Evaluating the entire thread pool shows that the number of threads will continue to grow according to the amount of tasks, with no upper limit. When the task is completed, the threads will be released after being idle for 1 minute. Suitable for situations where the number of tasks is relatively dense, but the execution time of each task is short
public class SynchronousQueueTest {
public static void main(String[] args) {
SynchronousQueue<Integer> integers = new SynchronousQueue<>();
new Thread(()->{
try {
System.out.println("putting 1");
integers.put(1);
System.out.println("putted 1");
System.out.println("putting 2");
integers.put(2);
System.out.println("putted 2");
} catch (InterruptedException e) {
e.printStackTrace();
}
},"t1").start();
new Thread(()->{
try {
System.out.println("taking 1");
integers.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
},"t2").start();
new Thread(()->{
try {
System.out.println("taking 2");
integers.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
},"t3").start();
}
}
putting 1
taking 1
putted 1
putting 2
taking 2
putted 2
newSingleThreadExecutor
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
scenes to be used:
- Want multiple tasks to be queued for execution. The number of threads is fixed at 1. When the number of tasks is more than 1, it will be put into an unbounded queue. After the task is executed, the only thread will not be released.
The difference from ordinary threads:
-
Create a single-threaded serial execution task yourself. If the task execution fails and terminates, there is no remedy. The thread pool will also create a new thread to ensure the normal operation of the pool.
-
Executors.newSingleThreadExecutor() The number of threads is always 1 and cannot be modified.
- FinalizableDelegatedExecutorService applies the decorator mode and only exposes the ExecutorService interface to the outside world, so it cannot call the unique methods in ThreadPoolExecutor.
-
Executors.newFixedThreadPool(1) is initially 1,
- In the future, you can also modify the ThreadPoolExecutor object that is exposed to the outside world. You can force it and then call setCorePoolSize and other methods to modify it.
-
Does single-threading make sense? What is the difference between creating a thread when we create a thread? When we create a thread, it can only perform one task, and a task is destroyed after it is executed. However, a single-thread pool can only perform one task at the same time. After the execution of this task is completed, continue to schedule a new task in the work queue to continue execution (still reducing the overhead of destroying and creating threads)
newScheduledThreadPool
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}
Before the "Task Scheduling Thread Pool" function was added, you could use java.util.Timer to implement the timing function. The advantage of Timer is that it is simple and easy to use, but since all tasks are scheduled by the same thread, all tasks are serialized. For execution, only one task can be executed at the same time. Delays or exceptions in the previous task will affect subsequent tasks.
public class TimerTest {
public static void main(String[] args) {
Timer timer = new Timer();
TimerTask task1= new TimerTask(){
@Override
public void run() {
try {
System.out.println("task 1");
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
TimerTask task2=new TimerTask(){
@Override
public void run() {
try {
System.out.println("task 2");
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
// 使用 timer 添加两个任务，希望它们都在 1s 后执行
// 但由于 timer 内只有一个线程来顺序执行队列中的任务，因此『任务1』的延时，影响了『任务2』的执行
timer.schedule(task1, 1000);
timer.schedule(task2, 1000);
}
}
20:46:09.444 c.TestTimer [main] - start...
20:46:10.447 c.TestTimer [Timer-0] - task 1
20:46:12.448 c.TestTimer [Timer-0] - task 2
Rewrite using ScheduledExecutorService:
ScheduledExecutorService executor = Executors.newScheduledThreadPool(2);
// 添加两个任务，希望它们都在 1s 后执行
executor.schedule(() -> {
System.out.println("任务1，执行时间：" + new Date());
try {
Thread.sleep(2000); } catch (InterruptedException e) {
}
}, 1000, TimeUnit.MILLISECONDS);
executor.schedule(() -> {
System.out.println("任务2，执行时间：" + new Date());
}, 1000, TimeUnit.MILLISECONDS);
任务1，执行时间：Thu Jan 03 12:45:17 CST 2019
任务2，执行时间：Thu Jan 03 12:45:17 CST 2019
Common methods of ExecutorService interface
Submit task
// 执行任务
void execute(Runnable command);
// 提交任务 task，用返回值 Future 获得任务执行结果
<T> Future<T> submit(Callable<T> task);
// 提交 tasks 中所有任务
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)throws InterruptedException;
// 提交 tasks 中所有任务，带超时时间
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks,long timeout, TimeUnit unit)throws InterruptedException;
// 提交 tasks 中所有任务，哪个任务先成功执行完毕，返回此任务执行结果，其它任务取消
<T> T invokeAny(Collection<? extends Callable<T>> tasks) throws InterruptedException, ExecutionException;
// 提交 tasks 中所有任务，哪个任务先成功执行完毕，返回此任务执行结果，其它任务取消，带超时时间
<T> T invokeAny(Collection<? extends Callable<T>> tasks,long timeout, TimeUnit unit)throws InterruptedException, ExecutionException, TimeoutException;
What is the difference between submit() and execute() methods in thread pool
? Similarities:
- The same thing is that they can all start threads to execute tasks in the pool.
difference:
- Receive parameters: execute() can only execute Runnable type tasks. submit() can perform Runnable and Callable type tasks.
- Return value: The submit() method can return a Future object holding the calculation result, but execute() does not
- Exception handling: submit() facilitates Exception handling
Close thread pool
/*
线程池状态变为 SHUTDOWN
- 不会接收新任务
- 但已提交任务会执行完
- 此方法不会阻塞调用线程的执行
*/
void shutdown();
public void shutdown() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
// 修改线程池状态
advanceRunState(SHUTDOWN);
// 仅会打断空闲线程
interruptIdleWorkers();
onShutdown(); // 扩展点 ScheduledThreadPoolExecutor
} finally {
mainLock.unlock();
}
// 尝试终结(没有运行的线程可以立刻终结，如果还有运行的线程也不会等)
tryTerminate();
}
/*
线程池状态变为 STOP
- 不会接收新任务
- 会将队列中的任务返回
- 并用 interrupt 的方式中断正在执行的任务
*/
List<Runnable> shutdownNow();
public List<Runnable> shutdownNow() {
List<Runnable> tasks;
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
// 修改线程池状态
advanceRunState(STOP);
// 打断所有线程
interruptWorkers();
// 获取队列中剩余任务
tasks = drainQueue();
} finally {
mainLock.unlock();
}
// 尝试终结
tryTerminate();
return tasks;
}
Other methods
// 不在 RUNNING 状态的线程池，此方法就返回 true
boolean isShutdown();
// 线程池状态是否是 TERMINATED
boolean isTerminated();
// 调用 shutdown 后，由于调用线程并不会等待所有任务运行结束，因此如果它想在线程池 TERMINATED 后做些事情，可以利用此方法等待
boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException;
How to correctly handle task execution exceptions
Method 1: Actively catch exceptions
ExecutorService pool = Executors.newFixedThreadPool(1);
pool.submit(() -> {
try {
log.debug("task1");
int i = 1 / 0;
} catch (Exception e) {
log.error("error:", e);
}
});
Method 2: Use Future
ExecutorService pool = Executors.newFixedThreadPool(1);
Future<Boolean> f = pool.submit(() -> {
log.debug("task1");
int i = 1 / 0;
return true;
});
log.debug("result:{}", f.get());
- If an exception is reported, then what our submit returns is our exception information.
How to use thread pool
Ali coding reduction: Try not to use the built-in thread pool. It is best to customize the thread pool's own new ThreadPoolExeutor object according to actual business needs and pass relevant parameters.
How many thread pools are appropriate to create?
- Too small will cause the program to be unable to fully utilize system resources and easily lead to starvation.
- If it is too large, it will cause more thread context switches and occupy more memory.
CPU-intensive operations
CPU intensive means that the task requires a lot of calculations without blocking and the CPU is always running at full speed. CPU-intensive tasks can only be accelerated (through multi-threading) on a real multi-core CPU, and on a single-core CPU, no matter how many simulated multi-threads you open, the task cannot be accelerated because the total computing power of the CPU The ability is just that.
- Usually, the optimal CPU utilization can be achieved by using the number of CPU cores + 1. +1 ensures that when a thread is suspended due to a page missing fault (operating system) or other reasons, the additional thread can take over to ensure the CPU clock cycle. not wasted
I/O intensive operations
IO intensive, that is, the task requires a lot of IO, that is, a lot of blocking. Running IO-intensive tasks on a single thread will result in a lot of wasted CPU computing power waiting. Therefore, using multi-threading in IO-intensive tasks can greatly speed up program execution. Even on a single-core CPU, this acceleration mainly takes advantage of wasted blocking time.
- When IO-intensive, most threads are blocked, so it is necessary to configure more threads, 2*cpu core number
The empirical formula is as follows
Number of threads = number of cores * expected CPU utilization * total time (CPU computing time + waiting time) / CPU computing time
- For example, the calculation time of 4-core CPU is 50%, and the other waiting time is 50%, and the expected cpu is 100% utilized, apply the formula
- 4 * 100% * 100% / 50% = 8
- For example, the calculation time of 4-core CPU is 10%, and the other waiting time is 90%. It is expected that the cpu is 100% utilized, and the formula is applied
- 4 * 100% * 100% / 10% = 40