content
- Background introduction
- How Thread Pools Work
- Analysis of the problem in the high concurrency scenario of the thread pool
- Thread pool performance optimization in high concurrency scenarios
- Summarize
Background introduction
Hello everyone, today I will tell you about a relatively hard-core technology, which is the high concurrency optimization of Java thread pools in production projects. Many brothers may have heard of the theoretical principles of Java thread pools and know how it works. , but I have never played with the Java thread pool in the project, let alone the optimization of the Java thread pool in a high-concurrency environment, so today let's discuss the high-concurrency optimization of this Java thread pool in production projects !
How Thread Pools Work
Since we want to talk about thread pools, at least everyone needs to know a little about the basic working principles of Java thread pools. If we want to explain the principles of thread pools clearly, or even analyze the source code level of JDK thread pools, we may have to write a separate article. Come to write, this is not our theme this time, so in this article, we will first tell you the simplest principle of thread pool.
The thread pool, in simple terms, is a pool of threads that are generally not destroyed, they will always exist, and then you can keep submitting tasks to the thread pool, and the thread pool will take out the threads for execution. your task. After the task is executed, the thread will not terminate, it will return to the thread pool and continue to stand by, as shown in Figure 1 below:
figure 1
But there will be a key problem at this time, that is, the number of threads in the thread pool is usually limited . Note that this is usually because of the real principle of the Java thread pool, in fact, through customized means, you can make Java thread pools have various performances. We are talking about the most basic situation here, that is, the number of threads in the thread pool is fixed and limited.
So if you submit too many tasks to the thread pool at once, and all the threads are busy running their own tasks at this time, if you want to submit new tasks, what do you think? Can tasks be submitted? **As shown in Figure 2 below:
figure 2
Of course, it can't be submitted. Is it true that the thread pool can only reject you at this time? **That's not true. In order to deal with this situation, the thread pool usually sets up a queue for you to submit tasks, and let your tasks wait in the queue for a period of time. After a thread finishes running its current task and becomes idle again, Then pull and run the tasks in this queue. Note that this is also the usual situation, because the Java thread pool can actually have other performances through customization, but we usually set the thread pool like this, as shown in Figure 3 below:image 3
Analysis of the problem in the high concurrency scenario of the thread pool
Then comes the question. The principle and usage of the most basic Java thread pool are introduced above, but what kind of problems will it encounter after it is actually put into a production project? First of all, the biggest problem is that the tasks submitted to the thread pool may all perform various network io tasks. For example, rpc calls other services, or processes a large amount of data in the db in the background, so it is very likely that It will take a long time for a thread to run a task , ranging from hundreds of milliseconds to several seconds, or even tens of seconds, as shown in Figure 4 below:
Figure 4
The second question , you noticed that the above picture is not there, that is, some tasks are rpc calls, which may only take a few hundred ms, and some tasks are a large number of data operations, which may take tens of seconds. Therefore, in fact, a common thread pool runs various tasks, which leads to uncertainty when a thread in the thread pool can complete a task, because the task may be an rpc call, or It may be a large amount of data processing.
The third problem is that some tasks may be in an http request. Originally, during the processing of an http request, multiple time-consuming tasks will be processed in turn. Now, in order to optimize performance, multiple tasks need to be submitted to the thread. In the pool , multiple threads are used to execute multiple tasks concurrently to improve the performance of this request. This http request needs to wait for the execution of these multiple concurrently running tasks to complete before returning a response to the user, as shown in Figure 5 below:
Figure 5
So, **The ultimate big problem**, this thread pool running in the production project is provided for a variety of different tasks to share, such as timing rpc calls, timing large data processing, foreground http Multiple tasks are requested concurrently, so during the busy period of the production environment, there may be the following scenarios: the thread pool is running multiple **timed rpc calls and timed large data processing tasks**, and these tasks are particularly time-consuming. , causing many threads to be busy and a few threads to be idle.Then at this time, the interface provided by the system for C-end users has high concurrent access scenarios. A large number of HTTP requests come, and each request must submit multiple tasks to the thread pool to run concurrently, resulting in a few idle threads in the thread pool running quickly. is full, and then a large number of tasks enter the queue of the thread pool and start to wait in line, as shown in Figure 6 below:
Image 6
At this time, it will inevitably cause a large number of http requests to hang dead , because many http request tasks are queued in the thread pool, they cannot run, and the http request cannot return a response, giving the user the feeling of clicking on the APP / The front end of a web page, click and click, no response, the system has a freeze problem! As shown in Figure 7 below:
Figure 7
Thread pool performance optimization in high concurrency scenarios
In response to this production environment problem, the first and biggest improvement we need to make is to separate various tasks from a thread pool so that they do not affect each other. That is to say, the timed rpc task is Put it in a thread pool, put a large amount of data processing tasks in the regular DB into another thread pool, and then put the HTTP request multitasking concurrent processing in a separate thread pool, everyone uses their own thread pools and resources, and they do not affect each other. , as shown in Figure 8 below:
Figure 8
如上图所做的话，我们有一个专门处理http请求的线程池，这压力一下子就下来了，因为http请求的任务通常耗时都在几十ms到一百ms级，整体速度很快，线程池里没有定时rpc和定时db访问这种耗时任务进来捣乱了，所以http请求的专有线程池可以轻松+愉快的快速处理所有http请求的任务，即使是在高并发场景下，可以通过线程池增加线程资源来合理抗下高并发压力。
另外就是对线上系统生产环境的线程池任务运行，我们通常会在公司里或者项目内研发统一的线程池监控框架，所有的线程池任务都需要封装到一个线程池监控框架提供的Class里，然后通过这个Class来实现任务的排队等待与运行耗时的两个维度的监控数据统计，如下面的代码所示。
// 线程任务包装类，用了**装饰设计模式 **
// 线程任务包装类，用了装饰设计模式
public class RunnableWrapper implements Runnable {
// 实际要执行的线程任务
private Runnable task;
// 线程任务被创建出来的时间
private long createTime;
// 线程任务被线程池运行的开始时间
private long startTime;
// 线程任务被线程池运行的结束时间
private long endTime;
// 当这个任务被创建出来的时候，就会设置他的创建时间
// 但是接下来有可能这个任务提交到线程池后，会进入线程池的队列排队
public RunnableWrapper(Runnable task) {
this.task = task;
this.createTime = new Date().getTime();
}
// 当任务在线程池排队的时候，这个run方法是不会被运行的
// 但是当任务结束了排队，得到线程池运行机会的时候，这个方法会被调用
// 此时就可以设置线程任务的开始运行时间
public void run() {
this.startTime = new Date().getTime();
// 此处可以通过调用监控系统的API，实现监控指标上报
// 用线程任务的startTime-createTime，其实就是任务排队时间
// monitor.report("threadName", "queueWaitTime", startTime-createTime);
// 接着可以调用包装的实际任务的run方法
task.run();
// 任务运行完毕以后，会设置任务运行结束的时间
this.endTIme = new Date().getTime();
// 此处可以通过调用监控系统的API，实现监控指标上报
// 用线程任务的endTime - startTime，其实就是任务运行时间
// monitor.report("threadName", "taskRunTime", endTime - startTime);
}
}
复制代码大家通过上面的代码可以清晰的看到，只要我们所有提交到线程池的任务，都用一个框架统一封装的RunnableWrapper类，基于装饰模式来进行包装，此时就可以得到线程任务的创建时间、开始时间、结束时间，接着就可以计算出这个任务的排队耗时、运行耗时，通过监控系统进行上报。
此时我们通过在监控系统里配置告警条件，就可以实现不同线程池的每个任务的耗时指标上报，同时如果有某个线程池的某个线程排队耗时或者运行耗时超过了我们配置的阈值，就会自动告警，如下图9所示：
图9
总结
好了，今天这篇文章到这里为止，就给大家把我们的线程池在生产项目里的生产问题和高并发如何优化，以及生产环境下的监控方案，都告诉大家了，希望大家学以致用，以后在项目里用线程池的时候，能够灵活运用咱们文章里学到的知识点。
END
扫码 免费获取 600+页石杉老师原创精品文章汇总PDF
原创技术文章汇总