(Turn) Detailed explanation of Tomcat's connection number and thread pool

http://www.cnblogs.com/kismetv/p/7806063.html

Preface

When using tomcat, you often encounter configuration problems such as the number of connections and threads. To truly understand these concepts, you must first understand the connection of Tomcat Connector.

In the previous article, I wrote in detail the Tomcat configuration file server.xml: The main function of Connector is to receive connection requests, create Request and Response objects to exchange data with the requester; then assign threads to the Engine (that is, the Servlet container) to process the request and pass the resulting Request and Response objects to the Engine. When the Engine finishes processing the request, it also returns the response to the client through the Connector.

It can be said that when the servlet container processes requests, it needs the Connector for scheduling and control. The Connector is the backbone of Tomcat processing requests. Therefore, the configuration and use of the Connector has an important impact on the performance of Tomcat. This article will start with Connector and discuss some important issues related to Connector, including NIO/BIO mode, thread pool, number of connections, etc.

According to different protocols, Connector can be divided into HTTP Connector, AJP Connector, etc. This article only discusses HTTP Connector.

1. Protocol of Nio, Bio, APR

1, Connector

Connectors use different protocols when processing HTTP requests. The protocols supported by different Tomcat versions are different. The most typical protocols include BIO, NIO and APR (Tomcat7 supports these three types, Tomcat8 adds support for NIO2, and in Tomcat8.5 and Tomcat9.0, the support for NIO2 is removed. BIO support).

BIO is Blocking IO, as the name implies, blocking IO; NIO is Non-blocking IO, which is non-blocking IO. APR is Apache Portable Runtime, an Apache portable runtime library, which can achieve high scalability and high performance by using local libraries; Apr is the preferred mode for running high-concurrency applications on Tomcat, but apr, apr-utils, tomcat need to be installed -native and other packages.

2. How to specify

which protocol is used by the protocol Connector can be specified through the protocol attribute in the <connector> element, or the default value can be used.

The specified protocol values ​​and corresponding protocols are as follows:

HTTP/1.1: Default value, the protocol used is related to the Tomcat version
org.apache.coyote.http11.Http11Protocol: BIO
org.apache.coyote.http11.Http11NioProtocol: NIO
org.apache .coyote.http11.Http11Nio2Protocol:NIO2
org.apache.coyote.http11.Http11AprProtocol:APR
If no protocol is specified, the default value HTTP/1.1 is used, and its meaning is as follows: In Tomcat7, BIO or APR is automatically selected (if the native library required by APR is found, APR is used, otherwise BIO is used); in Tomcat8, automatic Choose to use NIO or APR (use APR if a native library required by APR is found, NIO otherwise).

3. What is the difference between BIO and NIO?

Whether it is BIO or NIO, the general process of Connector processing requests is the same:

receiving connections in the accept queue (when the client sends a request to the server, if the client and the OS complete the three-way handshake establishment If there is a connection, the OS puts the connection into the accept queue); obtains the requested data in the connection, generates the request; calls the servlet container to process the request; returns the response. In order to facilitate the following description, first clarify the relationship between the connection and the request: the connection is at the TCP level (transport layer), corresponding to the socket; the request is at the HTTP level (application layer), which must depend on the TCP connection implementation; a TCP connection Multiple HTTP requests may be transmitted.

In the Connector implemented by BIO, the main entity that handles the request is the JIoEndpoint object. JIoEndpoint maintains the Acceptor and Worker: The Acceptor receives the socket, and then finds the idle thread from the Worker thread pool to process the socket. If there is no idle thread in the Worker thread pool, the Acceptor will block. Among them, Worker is the thread pool that comes with Tomcat. If other thread pools are configured through <Executor>, the principle is similar to that of Worker.

In the Connector implemented by NIO, the main entity that handles the request is the NIoEndpoint object. In addition to Acceptor and Worker, NIoEndpoint also uses Poller. The processing flow is shown in the following figure (image source: http://gearever.iteye.com/blog/1844203).


After the Acceptor receives the socket, it does not directly use the thread in the Worker to process the request, but first sends the request to the Poller, which is the key to implementing NIO. The Acceptor sends requests to the Poller through a queue, using a typical producer-consumer pattern. In the Poller, a Selector object is maintained; when the Poller takes out the socket from the queue, it is registered in the Selector; then by traversing the Selector, find the readable socket in it, and use the thread in the Worker to process the corresponding request. Similar to BIO, Workers can also be replaced by custom thread pools.

It can be seen from the above process that in the process of NIoEndpoint processing the request, whether the Acceptor receives the socket or the thread processes the request, the blocking method is still used; but in the process of "reading the socket and handing it over to the thread in the Worker" In , the non-blocking NIO implementation is used, which is the main difference between the NIO mode and the BIO mode (other differences have less impact on performance, and will not be mentioned for the time being). And this difference can significantly improve the efficiency of Tomcat in the case of a large amount of concurrency:

at present, most HTTP requests use long connections (HTTP/1.1 default keep-alive is true), and long connections mean that a After the current request of the TCP socket ends, if there is no new request, the socket will not be released immediately, but will be released after a timeout. If BIO is used, the process of "reading the socket and handing it over to the thread in the worker" is blocked, which means that while the socket is waiting for the next request or waiting for release, the worker thread processing the socket will always be occupied. It cannot be released; therefore, the number of sockets that Tomcat can process at the same time cannot exceed the maximum number of threads, and the performance is greatly limited. With NIO, the process of "reading the socket and handing it to the thread in the worker" is non-blocking. When the socket is waiting for the next request or waiting for release, it will not occupy the worker thread, so the number of sockets that Tomcat can process at the same time is much larger. Due to the maximum number of threads, the concurrent performance is greatly improved.

2. 3 parameters: acceptCount, maxConnections, maxThreads Let

's review the process of Tomcat processing requests: receiving connections in the accept queue (when the client sends a request to the server, if the client and the OS complete the three-way handshake to establish a connection, then the OS Put the connection into the accept queue); get the requested data in the connection, generate the request; call the servlet container to process the request; return the response.

Correspondingly, the functions of several parameters in Connector are as follows:

1. acceptCount

The length of the accept queue; when the number of connections in the accept queue reaches acceptCount, the queue is full, and incoming requests will all be rejected. The default value is 100.

2. maxConnections

The maximum number of connections that Tomcat receives and processes at any time. When the number of connections received by Tomcat reaches maxConnections, the Acceptor thread will not read the connections in the accept queue; at this time, the threads in the accept queue will be blocked until the number of connections received by Tomcat is less than maxConnections. If set to -1, the number of connections is unlimited.

The default value is related to the protocol used by the connector: the default value of NIO is 10000, the default value of APR/native is 8192, and the default value of BIO is maxThreads (if Executor is configured, the default value is maxThreads of Executor).

Under Windows, the maxConnections value of APR/native will be automatically adjusted to an integer multiple of 1024 below the setting value; if it is set to 2000, the maximum value is actually 1024.

3. maxThreads

The maximum number of request processing threads. The default value is 200 (both Tomcat 7 and 8). If the Connector has an Executor bound, this value will be ignored because the Connector will use the bound Executor instead of the built-in thread pool to execute tasks.

maxThreads specifies the maximum number of threads, not the actual number of running CPUs; in fact, the size of maxThreads is much larger than the number of CPU cores. This is because the thread processing the request may actually spend very little time for computation, and most of the time may be blocked, such as waiting for the database to return data, waiting for the hard disk to read and write data, and so on. Therefore, at a certain moment, only a few threads are actually using the physical CPU, and most threads are waiting; it is therefore reasonable that the number of threads is much larger than the number of physical cores.

In other words, Tomcat can keep the CPU busy by using much more threads than the number of CPU cores, greatly improving CPU utilization.

4. Parameter setting

(1) The setting of maxThreads is not only related to the characteristics of the application, but also related to the number of CPU cores of the server. From the previous introduction, we can know that the number of maxThreads should be much larger than the number of CPU cores; and the larger the number of CPU cores, the larger the maxThreads should be; the less CPU-intensive (IO more intensive) the application is, the larger the maxThreads should be, so that the CPU can be fully utilized. Of course, the value of maxThreads is not as large as possible. If maxThreads is too large, the CPU will spend a lot of time for thread switching, and the overall efficiency will be reduced.

(2) The setting of maxConnections is related to the running mode of Tomcat. If tomcat is using BIO, the value of maxConnections should be consistent with maxThreads; if tomcat is using NIO, then similar to Tomcat's default value, the value of maxConnections should be much larger than maxThreads.

(3) From the previous introduction, we can know that although the number of connections that tomcat can handle at the same time is maxConnections, the number of connections that can be received at the same time in the server is maxConnections+acceptCount. The setting of acceptCount is related to what the application expects to do when the connection is too high. If the setting is too large, the waiting time for incoming requests will be very long; if the setting is too small, the incoming requests will return connection refused immediately.

3. Thread pool Executor

The Executor element represents the thread pool in Tomcat and can be shared by other components; to use the thread pool, the component needs to specify the thread pool through the executor attribute.

Executor is an inline element of the Service element. Generally speaking, it is the Connector component that uses the thread pool; in order for the Connector to use the thread pool, the Executor element should be placed in front of the Connector. The configuration example of Executor and Connector is as follows:

1
2
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-" maxThreads="150" minSpareThreads="4" />
<Connector executor="tomcatThreadPool" port="8080" protocol ="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" acceptCount="1000" />



maxThreads: The maximum number of active threads in the thread pool, the default value is 200 (both Tomcat 7 and 8)
minSpareThreads: The minimum number of threads kept in the thread pool, the minimum value is 25
maxIdleTime: The maximum time that a thread is idle, it will be closed when the idle exceeds this value Thread (unless the number of threads is less than minSpareThreads), the unit is ms, the default value is 60000 (1 minute)
daemon: whether it is a background thread, the default value is true
threadPriority: thread priority, the default value is 5
namePrefix: the prefix of the thread name, the thread name in the thread pool It is: namePrefix + thread number
4. View the current status

The concept of the number of Tomcat connections and threads and how to set them are introduced above. The following describes how to view the number of connections and threads in the server.

Viewing the status of the server can be roughly divided into two solutions: (1) using off-the-shelf tools, (2) directly using Linux commands to view.

Ready-made tools, such as the jconsole tool that comes with JDK, can easily view thread information (in addition to CPU, memory, class, JVM basic information, etc.), the manager that comes with Tomcat, and the charging tool New Relic, etc. The following figure is the interface of jconsole to view thread information:


Let's talk about how to view the number of connections and threads in the server through the Linux command line.

1. Number of connections

Assuming that the port that Tomcat receives http requests is 8083, you can use the following statement to check the connection status:

1
netstat –nat | grep 8083
The results are as follows:


It can be seen that there is one connection in the listen state, listening for requests; in addition, there are 4 established connections (ESTABLISHED) and 2 connections waiting to be closed (CLOSE_WAIT).

2. The thread

ps command can check the process status, such as executing the following command:

1
ps –e | grep java
The result is as follows: As


you can see, only the information of one process is printed; 27989 is the thread id, and java refers to the executed java command . This is because starting a tomcat, all internal work is done in this process, including the main thread, garbage collection thread, Acceptor thread, request processing thread, and so on.

Through the following command, you can see how many threads there are in the process; where nlwp means number of light-weight process.

1
ps –o nlwp 27989

can see that there are 73 threads inside the process; but 73 does not exclude threads in the idle state. To get the actual number of running threads, you can do it with the following statement:

1
ps -eLo pid ,stat | grep 27989 | grep running | wc -l
where ps -eLo pid ,stat can find all threads and print where they are The process number and the current state of the thread; the two grep commands filter the process number and thread state respectively; wc counts the number. Among them, the results output by ps -eLo pid ,stat | grep 27989 are as follows:


only part of the results are screenshots in the figure; Sl indicates that most threads are in an idle state.