Tomcat handles user connection request mode
BIO: blocking model
NIO: non-blocking model
APR: high-performance, scalable mode, the default mode of Tomcat8 version
When using tomcat, you often encounter configuration problems such as the number of connections and the number of threads. To really understand these concepts, you must first understand Tomcat's Connector.
In the previous article, I wrote in detail the Tomcat configuration file server.xml: The main function of Connector is to receive connection requests, create Request and Response objects to exchange data with the requester; then assign threads to the Engine (that is, the Servlet container) to process the request and pass the resulting Request and Response objects to the Engine. When the Engine finishes processing the request, it also returns the response to the client through the Connector.
It can be said that when the servlet container processes requests, it needs the Connector for scheduling and control. The Connector is the backbone of Tomcat processing requests. Therefore, the configuration and use of the Connector has an important impact on the performance of Tomcat. This article will start with Connector and discuss some important issues related to Connector, including NIO/BIO mode, thread pool, number of connections, etc.
According to different protocols, Connector can be divided into HTTP Connector, AJP Connector, etc. This article only discusses HTTP Connector.
1. Protocols of Nio, Bio, APR
1, Connector
When processing HTTP requests, Connector will use different protocols. The protocols supported by different Tomcat versions are different. The most typical protocols include BIO, NIO and APR (Tomcat7 supports these three types, Tomcat8 adds support for NIO2, and in Tomcat8.5 and Tomcat9.0, the support for NIO2 is removed. BIO support).
BIO is Blocking IO, as the name implies, blocking IO; NIO is Non-blocking IO, which is non-blocking IO. APR is Apache Portable Runtime, an Apache portable runtime library, which can achieve high scalability and high performance by using local libraries; Apr is the preferred mode for running high-concurrency applications on Tomcat, but apr, apr-utils, tomcat need to be installed -native and other packages.
2. How to specify
which protocol the protocol Connector uses can be specified through the protocol attribute in the element, or the default value can be used.
The specified protocol value and corresponding protocol are as follows:
HTTP/1.1: Default, protocol used depends on Tomcat version
org.apache.coyote.http11.Http11Protocol: BIO
org.apache.coyote.http11.Http11NioProtocol: NIO
org.apache.coyote.http11.Http11Nio2Protocol: NIO2
org.apache .coyote.http11.Http11AprProtocol: If APR
does not specify a protocol, the default value HTTP/1.1 is used, and its meaning is as follows: In Tomcat7, BIO or APR is automatically selected (if the native library required by APR is found, APR is used, otherwise, use BIO); in Tomcat8, NIO or APR is automatically selected (if a native library required by APR is found, APR is used, otherwise NIO is used).
3. What is the difference between BIO and NIO?
Whether it is BIO or NIO, the general process of Connector processing requests is the same:
Receive the connection in the accept queue (when the client sends a request to the server, if the client and the OS complete the three-way handshake to establish a connection, the OS puts the connection in the accept queue); obtain the requested data in the connection, and generate the request; Call the servlet container to process the request; return the response. In order to facilitate the following description, first clarify the relationship between the connection and the request: the connection is at the TCP level (transport layer), corresponding to the socket; the request is at the HTTP level (application layer), which must depend on the TCP connection implementation; a TCP connection Multiple HTTP requests may be transmitted.
In the Connector implemented by BIO, the main entity that handles the request is the JIoEndpoint object. JIoEndpoint maintains the Acceptor and Worker: The Acceptor receives the socket, and then finds the idle thread from the Worker thread pool to process the socket. If there is no idle thread in the Worker thread pool, the Acceptor will block. Among them, Worker is the thread pool that comes with Tomcat. If other thread pools are configured, the principle is similar to that of Worker.
In the Connector implemented by NIO, the main entity that handles the request is the NIoEndpoint object. In addition to acceptor and worker, NIoEndpoint also uses Poller. The processing flow is shown in the following figure (image source: http://gearever.iteye.com/blog/1844203 ).
After the Acceptor receives the socket, it does not directly use the thread in the Worker to process the request, but first sends the request to the Poller, which is the key to implementing NIO. The Acceptor sends requests to the Poller through a queue, using a typical producer-consumer pattern. In the Poller, a Selector object is maintained; when the Poller takes out the socket from the queue, it is registered in the Selector; then by traversing the Selector, find the readable socket in it, and use the thread in the Worker to process the corresponding request. Similar to BIO, Workers can also be replaced by custom thread pools.
It can be seen from the above process that in the process of NIoEndpoint processing the request, whether the Acceptor receives the socket or the thread processes the request, the blocking method is still used; but in the process of "reading the socket and handing it over to the thread in the Worker" In , the non-blocking NIO implementation is used, which is the main difference between the NIO mode and the BIO mode (other differences have less impact on performance, and will not be mentioned for the time being). And this difference, in the case of a large amount of concurrency, can bring about a significant improvement in the efficiency of Tomcat:
At present, most HTTP requests use long connections (HTTP/1.1 default keep-alive is true), and long connections mean that after a TCP socket ends the current request, if there is no new request coming, the socket will not be released immediately , but wait for timeout and then release. If BIO is used, the process of "reading the socket and handing it over to the thread in the worker" is blocked, which means that while the socket is waiting for the next request or waiting for release, the worker thread processing the socket will always be occupied. It cannot be released; therefore, the number of sockets that Tomcat can process at the same time cannot exceed the maximum number of threads, and the performance is greatly limited. With NIO, the process of "reading the socket and handing it to the thread in the worker" is non-blocking. When the socket is waiting for the next request or waiting for release, it will not occupy the worker thread, so the number of sockets that Tomcat can process at the same time is much larger. Due to the maximum number of threads, the concurrent performance is greatly improved.
Second, 3 parameters: acceptCount, maxConnections, maxThreads Let
's review the process of Tomcat processing requests: receiving connections in the accept queue (when the client sends a request to the server, if the client and the OS complete the three-way handshake to establish a connection, then the OS Put the connection into the accept queue); get the requested data in the connection, generate the request; call the servlet container to process the request; return the response.
Correspondingly, several parameters in Connector function as follows:
1. acceptCount
The length of the accept queue; when the number of connections in the accept queue reaches acceptCount, the queue is full, and incoming requests will all be rejected. The default value is 100.
2. maxConnections
The maximum number of connections that Tomcat receives and processes at any time. When the number of connections received by Tomcat reaches maxConnections, the Acceptor thread will not read the connections in the accept queue; at this time, the threads in the accept queue will be blocked until the number of connections received by Tomcat is less than maxConnections. If set to -1, the number of connections is unlimited.
The default value is related to the protocol used by the connector: the default value of NIO is 10000, the default value of APR/native is 8192, and the default value of BIO is maxThreads (if Executor is configured, the default value is maxThreads of Executor).
Under Windows, the maxConnections value of APR/native will be automatically adjusted to an integer multiple of 1024 below the setting value; if it is set to 2000, the maximum value is actually 1024.
3. maxThreads
The maximum number of request processing threads. The default value is 200 (both Tomcat 7 and 8). If the Connector has an Executor bound, this value will be ignored because the Connector will use the bound Executor instead of the built-in thread pool to execute tasks.
maxThreads specifies the maximum number of threads, not the actual number of running CPUs; in fact, the size of maxThreads is much larger than the number of CPU cores. This is because the thread processing the request may actually spend very little time for computation, and most of the time may be blocked, such as waiting for the database to return data, waiting for the hard disk to read and write data, and so on. Therefore, at a certain moment, only a few threads are actually using the physical CPU, and most threads are waiting; it is therefore reasonable that the number of threads is much larger than the number of physical cores.
In other words, Tomcat can keep the CPU busy by using much more threads than the number of CPU cores, greatly improving CPU utilization.
4. Parameter setting
(1) The setting of maxThreads is not only related to the characteristics of the application, but also related to the number of CPU cores of the server. From the previous introduction, we can know that the number of maxThreads should be much larger than the number of CPU cores; and the larger the number of CPU cores, the larger the maxThreads should be; the less CPU-intensive (IO more intensive) the application is, the larger the maxThreads should be, so that the CPU can be fully utilized. Of course, the value of maxThreads is not as large as possible. If maxThreads is too large, the CPU will spend a lot of time for thread switching, and the overall efficiency will be reduced.
(2) The setting of maxConnections is related to the running mode of Tomcat. If tomcat is using BIO, the value of maxConnections should be consistent with maxThreads; if tomcat is using NIO, then similar to Tomcat's default value, the value of maxConnections should be much larger than maxThreads.
(3) From the previous introduction, we can know that although the number of connections that tomcat can handle at the same time is maxConnections, the number of connections that can be received at the same time in the server is maxConnections+acceptCount. The setting of acceptCount is related to what the application expects to do when the connection is too high. If the setting is too large, the waiting time for incoming requests will be very long; if the setting is too small, the incoming requests will return connection refused immediately.
3. Thread pool Executor
The Executor element represents the thread pool in Tomcat and can be shared by other components; to use the thread pool, the component needs to specify the thread pool through the executor attribute.
Executor is an inline element of the Service element. Generally speaking, it is the Connector component that uses the thread pool; in order for the Connector to use the thread pool, the Executor element should be placed in front of the Connector. The configuration examples of Executor and Connector are as follows:
<Executor name="tomcatThreadPool" namePrefix ="catalina-exec-" maxThreads="150" minSpareThreads="4" />
<Connector executor="tomcatThreadPool" port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" acceptCount="1000" />- 1
- 2
- 3
The main properties of Executor include:
name: the mark of the
thread pool maxThreads: the maximum number of active threads in the thread pool, the default value is 200 (both Tomcat 7 and 8)
minSpareThreads: the minimum number of threads kept in the thread pool, the minimum value is 25
maxIdleTime: the maximum time that a thread is idle, When the idle exceeds this value, close the thread (unless the number of threads is less than minSpareThreads), the unit is ms, the default value is 60000 (1 minute)
daemon: whether the background thread, the default value is true
threadPriority: the thread priority, the default value is 5
namePrefix: the name of the thread Prefix, the thread name in the thread pool is: namePrefix + thread number
4. Viewing the current status
The concepts of the number of Tomcat connections and threads and how to set them are introduced above. The following describes how to view the number of connections and threads in the server.
Viewing the status of the server can be roughly divided into two solutions: (1) using off-the-shelf tools, (2) directly using Linux commands to view.
Ready-made tools, such as the jconsole tool that comes with JDK, can easily view thread information (in addition to CPU, memory, class, JVM basic information, etc.), the manager that comes with Tomcat, and the charging tool New Relic, etc. The following figure is the interface of jconsole to view thread information:
Let's talk about how to check the number of connections and threads in the server through the Linux command line.
1. Number of connections
Assuming that the port that Tomcat receives http requests is 8083, you can use the following statement to check the connection status:
1
netstat –nat | grep 8083
results in the following:
It can be seen that there is one connection in the listen state, listening for requests; in addition, there are 4 established connections (ESTABLISHED) and 2 connections waiting to be closed (CLOSE_WAIT).
2. The thread
ps command can view the process status, such as executing the following command:
The result of ps -e | grep java
is as follows:
As you can see, only one process information is printed; 27989 is the thread id, and java refers to the executed java command. This is because starting a tomcat, all internal work is done in this process, including the main thread, garbage collection thread, Acceptor thread, request processing thread, and so on.
Through the following command, you can see how many threads there are in the process; where nlwp means number of light-weight process.
ps –o nlwp 27989
It can be seen that there are 73 threads inside the process; but 73 does not exclude threads in the idle state. To get the actual number of threads running, you can do it with the following statement:
1
ps -eLo pid ,stat | grep 27989 | grep running | wc -l
where ps -eLo pid ,stat can find all threads, and print the process number and the current status of the thread; two grep commands filter processes respectively number and thread status; wc counts the number. Among them, the results of ps -eLo pid ,stat | grep 27989 output are as follows:
Only a partial screenshot of the results is shown in the figure; Sl indicates that most threads are idle.
#############################
Reprinted from: https://blog.csdn.net/qq_28740207/article/details/80030743