Tomcat's maxThreads, acceptCount (maximum number of threads, maximum number of queues)
The Connector configuration of tomcat is as follows
<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" maxThreads="800" acceptCount="1000"/>
maxThreads : The maximum number of threads that tomcat starts, that is, the number of tasks processed at the same time, the default value is 200
acceptCount : When the number of threads started by tomcat reaches the maximum, the number of queued requests accepted, the default value is 100
How these two values work, please see the following three cases
Case 1: Accepting a request, at this time, the number of threads started by tomcat does not reach maxThreads, and tomcat will start a thread to process the request.
Case 2: Accept a request. At this time, the number of threads started by tomcat has reached maxThreads, and tomcat will put the request into the waiting queue and wait for idle threads.
Case 3: Accepting a request, the number of threads started by tomcat has reached maxThreads, and the number of requests in the waiting queue has also reached acceptCount. At this time, tomcat will directly reject the request and return connection refused
How to configure maxThreads
General server operations include volume: 1 calculation (mainly consuming cpu), 2 waiting (io, database, etc.)
In the first extreme case, if our operation is purely computing, then the main limitation of the system response time is the computing power of the cpu. At this time, maxThreads should be set as small as possible to reduce the number of threads competing for the cpu at the same time. Improve computing efficiency and improve the overall processing power of the system.
In the second extreme case, if our operation is purely IO or database, then the main limitation of response time becomes waiting for external resources. At this time, maxThreads should be set as large as possible, so as to increase the number of simultaneous requests, thereby improving The overall processing power of the system. In this case, because the amount of requests processed by tomcat at the same time will be relatively large, you need to pay attention to the virtual machine memory settings of tomcat and the open file limit of linux.
I encountered a problem during the test. I set maxThreads to be relatively large, such as 3000. When the number of threads served is large to a certain extent, usually in the early 2000s, the response time of a single request will increase sharply.
I can't figure out why this is, and I can't find the answer. In the end, I concluded that the reason may be that the time consumed by the CPU during thread switching increases with the increase of the number of threads.
The cpu spends most of its time switching directly between these 2,000 or more threads. Of course, the cpu has no time to process our program.
It has always been simple to think that multi-threading = high efficiency. . In fact, multithreading itself does not improve CPU efficiency, but too many threads will reduce CPU efficiency.
When the number of cpu cores < the number of threads, the cpu needs to switch directly back and forth between multiple threads to ensure that each thread will get cpu time, which is usually what we call concurrent execution.
So the configuration of maxThreads is definitely not the bigger the better.
In real applications, our operations will include the above two types (calculation, waiting), so the configuration of maxThreads does not have an optimal value, and it must be configured according to the specific situation.
The best way is: on the basis of continuous testing, continuous adjustment and optimization can get the most reasonable configuration.
For the configuration of acceptCount , I usually set it to be as large as maxThreads. This value should be weighed and configured mainly according to the peak value and average value of the application's access.
If it is set to a small value, it can guarantee that the accepted requests will respond quickly, but the exceeding requests may be rejected directly.
If it is set larger, there may be a large number of request timeouts, because the processing capacity of our system is certain.