The interviewer's original question was: How many requests can a SpringBoot project handle at the same time?
I don’t know what your first reaction was after hearing this question.
I roughly know what direction he wants to ask, but for this kind of interview question with only one sentence, my first reaction is: Will there be any pitfalls?
Therefore, I will not answer the question rashly. I will first ask for some information, such as: What is this project specifically about? What parameters have been configured for the project? What web container is used? What is the deployed server configuration? What interfaces are there? What is the approximate average response time of the interface?
In this way, after asking a few questions, you can basically reach an agreement with the interviewer on at least the direction of the interview questions.
For example, after asking the previous interview question several times, the interviewer may change it to:
A SpringBoot project, without any special configuration, all adopt the default settings. How many requests can this project handle at the same time at the same time?
How much can it handle?
I don’t know either, but when the question became like this, I found an angle to explore the answer.
Since "no special configuration has been made", then I can make a demo myself and just do it, right?
Sit tight and brace yourself, ready to drive.
Demo
With a little trembling of hands, I made a demo first.
This Demo is very simple, just create a new SpringBoot project through idea.
My SpringBoot version is 2.7.13.
The entire project only has these two dependencies:
There are only two categories in the entire project, so one must be empty and clear.
The TestController in the project has only one getTest method, which is used for testing. After receiving the request, the method sleeps for one hour.
The purpose is to directly occupy the current request thread, so that we can know how many threads are available in the project:
@Slf4j
@RestController
public class TestController {
@GetMapping("/getTest")
public void getTest(int num) throws Exception {
log.info("{} 接受到请求:num={}", Thread.currentThread().getName(), num);
TimeUnit.HOURS.sleep(1);
}
}
The application.properties file in the project is also empty:
In this way, wouldn't there be a SpringBoot "without any special configuration"?
Based on this Demo, the previous interview question will become: How many times can I call the getTest method of this Demo continuously in a short period of time?
Has the problem become a little simpler again?
So how does the previous "continuous call in a short period of time" be expressed in code?
It's very simple, just continue to call the interface in the loop.
public class MainTest {
public static void main(String[] args) {
for (int i = 0; i < 1000; i++) {
int finalI = i;
new Thread(() -> {
HttpUtil.get("127.0.0.1:8080/getTest?num=" + finalI);
}).start();
}
//阻塞主线程
Thread.yield();
}
}
Of course, if you use some stress testing tools, such as jmeter, it will look more compelling and professional. I'm going to be lazy here and go straight to the code.
Answer
After the previous preparations, the demo and test codes are ready.
The next step is to run the Demo first:
Then run MainTest.
When MainTest runs, Demo will quickly and in large quantities output logs like this:
That is the log I wrote in the previous getTest method:
Okay, now let's get back to this question:
I continuously call the getTest method of this Demo in a short period of time. How many times can it be called at most?
Come on, please tell me how to get the answer to this question?
What I am doing here is a miracle, just count the number of times the "request received" keyword appears in the log:
Obviously, the answer is:
So, when the interviewer asks you: How many requests can a SpringBoot project handle at the same time?
You pretend to think carefully and say firmly: 200 times.
The interviewer nods slightly and waits for you to continue.
You are also secretly happy, luckily you read Master Wai Wai Wai's article and recited an answer. Then wait for the interviewer to continue asking other questions.
The atmosphere suddenly became awkward.
Then, you go home and wait for the notification.
200 times, this answer is correct, but if you only say 200 times, this answer seems a bit awkward.
The important thing is, where does this value come from?
So, the following part, you also have to memorize it.
how come?
Before starting to explore how it came about, let me ask you a question first, whose thread does these 200 threads belong to, or who is managing this thread?
Is it SpringBoot?
Definitely not, SpringBoot is not a web container.
Tomcat should be managing these 200 threads.
We can also verify this through thread Dump:
Through the thread dump file, we can know that a large number of threads are in the sleep state. And click on these threads to view their stack messages, you can see keywords such as Tomcat, threads, ThreadPoolExecutor:
at org.apache.Tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
at org.apache.Tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at org.apache.Tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.Tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.Tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
Based on the phenomenon that "200 requests are processed immediately in a short period of time", combined with your memorized and very solid thread pool knowledge, you first make a bold guess: the default number of core threads in Tomcat is 200.
Next, we need to go to the source code to verify whether this guess is correct.
I have shared the way to read source code before, "I try to teach you a way to read source code through this article. 》 , the most important one is to set a valid breakpoint, and then locate the source code based on the call stack at the breakpoint.
Here I will teach you another way to get the call stack without interrupting points.
As shown before, it is thread dump.
On the right is the complete call stack of a thread:
From this call stack, since we are looking for the source code related to the Tomcat thread pool, the first place where the relevant keywords appear is this line:
org.apache.Tomcat.util.threads.ThreadPoolExecutor.Worker#run
Then we put a breakpoint on this line.
Restart the project and start debugging.
After entering runWorker, this part of the code looks very familiar:
It is exactly the same as the thread pool source code in JDK.
If you are familiar with the JDK thread pool source code, debugging Tomcat's thread pool will feel like going home.
If you are not familiar with it, I suggest you get familiar with it as soon as possible.
As the breakpoint goes down, in the getTask method, you can see several key parameters about the thread pool:
org.apache.Tomcat.util.threads.ThreadPoolExecutor#getTask
corePoolSize, the number of core threads, the value is 10.
maximumPoolSize, the maximum number of threads, the value is 200.
And based on the maximumPoolSize parameter, if you scroll forward in the code, you will find that the default value is 200:
Okay, here you find that your previous guess that "Tomcat's default number of core threads is 200" is wrong.
But you don't panic at all, combined with the very solid thread pool knowledge you have memorized again.
And I recited it silently in my heart: when the thread pool receives the task, first enable the number of core threads, then use the queue length, and finally enable the maximum number of threads.
Because we verified earlier, Tomcat can process 200 requests at the same time, and its thread pool has only 10 core threads, and the maximum number of threads is 200.
This shows that my previous test case filled the queue, causing the Tomcat thread pool to enable the maximum number of threads:
Well, that must be the case!
So, the key question now is: What is the default queue length of the Tomcat thread pool?
In the current Debug mode, the queue length can be viewed through Alt+F8:
wc, this value is Integer.MAX_VALUE, so big?
I only have 1,000 tasks in total, so it's impossible to be full?
A thread pool:
-
Number of core threads, value is 10.
-
Maximum number of threads, value is 200.
-
Queue length, value is Integer.MAX_VALUE.
After 1,000 time-consuming tasks come over, only 10 threads should be working, and then the remaining 990 will be queued, right?
Did I memorize the eight-part essay wrongly?
Don't panic at this time, just calm down.
What is currently known is the number of core threads, which is 10. The workflow of these 10 threads is in line with our knowledge.
But when the 11th task came, it should have entered the queue.
Now it seems that the maximum number of threads is directly enabled.
So, let’s modify the test case first:
Then the question arises: How is the last request submitted to the thread pool?
As mentioned earlier, Tomcat's thread pool source code is basically the same as that of JDK.
When submitting tasks to the thread pool, the execute method will be executed:
org.apache.Tomcat.util.threads.ThreadPoolExecutor#execute(java.lang.Runnable)
For Tomcat it will call executeInternal this method:
org.apache.Tomcat.util.threads.ThreadPoolExecutor#executeInternal
In this method, the place labeled ① is to judge whether the current number of worker threads is less than the number of core threads, and if it is less than that, directly call the addWorker method to create threads.
The place marked ② mainly calls the offer method to see if tasks can continue to be added to the queue.
If you cannot continue to add, it means that the queue is full, then go to the place marked ③ to see if you can execute the addWorker method to create non-core threads, that is, enable the maximum number of threads.
After smoothing out this logic, it will be very clear which part of the code we should look at next.
The main thing is to look at the logic of workQueue.offer(command).
If it returns true, it means joining the queue, and if it returns false, it means enabling the maximum number of threads.
This workQueue is a TaskQueue, and it doesn’t look familiar at all:
Of course it doesn't look familiar, because this is a queue created by Tomcat itself based on LinkedBlockingQueue.
The answer to the question is hidden in the offer method of TaskQueue.
So I will focus on taking you through this offer method:
org.apache.Tomcat.util.threads.TaskQueue#offer
At the place marked ①, it is judged whether the parent is null. If so, the offer method of the parent class is called directly. Note that to enable this logic, our parent cannot be null.
So what is this parent and where did it come from?
The parent is the Tomcat thread pool. Through its set method, we can know that the assignment is made after the thread pool completes its initialization.
In other words, you can understand that in the Tomcat scenario, parent will not be empty.
The place marked ② calls the getPoolSizeNoLock method:
This method is to get multiple threads in the current thread pool.
So if this expression is true:
parent.getPoolSizeNoLock() == parent.getMaximumPoolSize()
It means that the number of threads in the current thread pool is already the configured maximum number of threads, then call the offer method and put the current request into the queue.
The place marked ③ is to determine whether the number of tasks that have been submitted to the thread pool for execution or are being executed is less than the number of threads in the current thread pool.
If it is, it means that there are idle threads in the current thread pool that can execute the task. If the task is put into the queue, it will be taken away by the idle thread for execution.
Then, the key comes, the place marked ④.
If the number of threads in the current thread pool is less than the maximum number of threads configured in the thread pool, false is returned.
As mentioned earlier, what happens if the offer method returns false?
Did you go directly to the place marked ③ in the above figure and try to add non-core threads?
That is, the configuration of the maximum number of threads is enabled.
So, friends, what’s going on?
This situation is indeed different from the eight-part essay on the thread pool we memorized.
The JDK thread pool first uses the core thread number configuration, then uses the queue length, and finally uses the maximum thread configuration.
Tomcat's thread pool first uses the core thread number configuration, then uses the maximum thread configuration, and finally uses the queue length.
So, in the future, when the interviewer tells you: Let’s talk about the working mechanism of the thread pool?
You should ask first: Are you talking about the JDK thread pool or the Tomcat thread pool, because there is a slight difference in the operating mechanism of the two.
Then, you look at his expression.
If there is a trace of hesitation, and then said lightly: Let's compare it.
So congratulations, you have begun to take some initiative on this topic.
Finally, in order to give you a deeper understanding of the difference between the Tomcat thread pool and the JDK thread pool, I will give you a code that can be run directly by copying it.
When you comment out the line of code taskqueue.setParent(executor), its operating mechanism is the thread pool of JDK.
When this line of code exists, its running mechanism becomes Tomcat's thread pool.
Go ahead and play.
import org.apache.tomcat.util.threads.TaskQueue;
import org.apache.tomcat.util.threads.TaskThreadFactory;
import org.apache.tomcat.util.threads.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class TomcatThreadPoolExecutorTest {
public static void main(String[] args) throws InterruptedException {
String namePrefix = "歪歪歪-exec-";
boolean daemon = true;
TaskQueue taskqueue = new TaskQueue(300);
TaskThreadFactory tf = new TaskThreadFactory(namePrefix, daemon, Thread.NORM_PRIORITY);
ThreadPoolExecutor executor = new ThreadPoolExecutor(5,
150, 60000, TimeUnit.MILLISECONDS, taskqueue, tf);
taskqueue.setParent(executor);
for (int i = 0; i < 300; i++) {
try {
executor.execute(() -> {
logStatus(executor, "创建任务");
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
} catch (Exception e) {
e.printStackTrace();
}
}
Thread.currentThread().join();
}
private static void logStatus(ThreadPoolExecutor executor, String name) {
TaskQueue queue = (TaskQueue) executor.getQueue();
System.out.println(Thread.currentThread().getName() + "-" + name + "-:" +
"核心线程数:" + executor.getCorePoolSize() +
"\t活动线程数:" + executor.getActiveCount() +
"\t最大线程数:" + executor.getMaximumPoolSize() +
"\t总任务数:" + executor.getTaskCount() +
"\t当前排队线程数:" + queue.size() +
"\t队列剩余大小:" + queue.remainingCapacity());
}
}
etc.
If you have never understood the working mechanism of Tomcat thread pool before, then you may feel that you have gained a little bit when you see this.
But, notice I said but.
Do you still remember the interviewer’s question at the beginning?
The interviewer's original question was: How many requests can a SpringBoot project handle at the same time?
So, let me ask you, I talked a lot about the operating principle of Tomcat thread pool earlier. Does this answer match this question?
Yes, except for the value of 200 proposed at the beginning, it does not match, and even in the eyes of the interviewer, the answer is completely irrelevant.
Therefore, in order to connect these two "non-matching" things smoothly, you must first answer the interviewer's questions and then start to expand.
For example, the answer is this: A SpringBoot project that does not have any special configuration and uses all default settings. The maximum number of requests this project can handle at the same time depends on the web container we use, and SpringBoot uses Tomcat by default.
Tomcat's default number of core threads is 10, the maximum number of threads is 200, and the queue length is infinite. However, because its operating mechanism is different from the JDK thread pool, after the number of core threads is full, the maximum number of threads will be enabled directly. Therefore, under the default configuration, 200 requests can be processed at the same time.
In actual use, this parameter should be evaluated and set based on the actual service situation and related information such as server configuration.
This answer is almost the same.
However, if unfortunately, if you meet me, in order to verify whether you have really explored it yourself or just read a few articles, I may also ask:
Then nothing else moves. If I just add the configuration server.tomcat.max-connections=10, how many requests can be processed at most at this time?
You might have to guess: 10.
Yes, I resubmitted 1000 tasks, and the console output is indeed 10,
So how can the parameter max-connections control the number of requests?
Why did we not notice this parameter during the previous analysis?
First let's take a look at its default value:
Because its default value is 8192, which is larger than the maximum number of threads, 200, this parameter does not limit us, so we did not pay attention to it.
When we adjust it to 10, which is less than the maximum number of threads of 200, it starts to become a limitation.
So what does the max-connections parameter do?
You should explore it yourself first.
At the same time, there is also such a parameter, the default is 100:
server.tomcat.accept-count=100
What does it do?
"It's related to the number of connections", I can only remind you here, you can explore it yourself.
hold on
Through the previous analysis, we know that to answer the question "the number of tasks a SpringBoot project can handle by default", we must first clarify the web container used.
So here comes the question again: what containers does SpringBoot have built in?
Tomcat、Jetty、Netty、Undertow
Previously, we were all based on Tomcat analysis. What if we change the container?
For example, if I switch to Undertow, I have only heard of this thing but have never actually used it. It is a black box to me.
Never mind it, just change it first.
To change from Tomcat to Undertow, you only need to modify the Maven dependencies, and nothing else needs to be changed:
Start the project again, and you can see from the log that it has been modified into an Undertow container:
At this point, I execute the MainTest method again, or submit 1000 requests:
From the log, it was found that only 48 requests were processed.
It's very confusing, what's the matter with 48, why is it not an integer, this makes obsessive-compulsive disorder uncomfortable.
What are you thinking at this time? Do you want to see where the number 48 comes from?
What do you think?
Didn't I teach you when I was looking for Tomcat's 200 before, just put it on Undertow.
Make a thread dump and look at the stack messages:
Discover the thread pool EnhancedQueueExecutor, and then find the parameters when building the thread pool in this class.
It's easy to find this constructor:
So, put a breakpoint here and restart the project.
Through Debug, we can know that the key parameters come from the builder.
In the builder, coreSize and maxSize are both 48, and the queue length is Integer.MAX_VALUE.
So take a look at how the coreSize in the Builder comes from.
Click here and find that the default value of coreSize is 16:
Don't panic, interrupt again, and restart the project.
Then you will stop at its setCorePoolSize method, and the input parameter of this method is the 48 we are looking for:
Follow the vine, and after repeating the break point and restart several times, you will find that 48 is a variable named WORKER_TASK_CORE_THREADS, which comes from here:
The WORKER_TASK_CORE_THREADS variable is set like this:
io.undertow.Undertow#start
The value of workerThreads here is as follows:
io.undertow.Undertow.Builder#Builder
Take the number of CPUs of the machine multiplied by 8.
So what I have here is 6*8=48.
Oh, the truth is revealed. It turns out that this is how 48 came about.
boring.
It's really boring, but since it has been replaced by Undertow, you can study its NIO ByteBuffer, NIO Channel, BufferPool, XNIO Worker, IO thread pool, Worker thread pool...
Then compare it with Tomcat and learn from it.
It started to get a little interesting.
Wait for the end
This article is based on the interview question "How many requests can a SpringBoot project handle at the same time?"
But after our simple analysis above, you also know that the answer to this question is different if some specific prerequisites are not added.
For example, let me give you another example. It is still our Demo. We just use the @Async annotation and leave everything else unchanged:
Start the project again, initiate access, and the log output becomes like this:
The number of requests that can be processed at the same time has changed from Tomcat's default 200 to 8?
Because the @Async annotation corresponds to the thread pool, the default number of core threads is 8.
So you see, with a slight change, the answer looks different again, and at the same time, the process of this request's internal circulation is also different, which is another point that can be discussed.
The same is true during the interview process. Don't rush to answer the questions. When you feel that the interviewer's question description is unclear, you can ask tentatively first to see if you can dig out some default conditions that he didn't say.
The more "default conditions" you dig into, the more likely your answer will be accepted by the interviewer. This excavation process is also an important performance link in the interview process.
Moreover, sometimes, the interviewer likes to give such "fuzzy" questions, because the more vague the question, the more pits there will be. When the interviewer jumps into the pit he has dug, it is the end of a confrontation; When the interviewer sees the hole he has dug and goes around it, it is also the end of the round of confrontation.
So, don’t rush to answer questions, think more and ask more questions. Whether it is for the interviewer or the interviewer, a good interview experience must not be a question and answer without interaction, but a process of seeing each other.