In a high-concurrency environment, I know to optimize the configuration of tomcat and modify the number of connections and thread pool. The most important thing is that the protocol of the connector Http Connector uses NIO instead of the default AJP Connector. At that time, I did not study its principle carefully. Now let's analyze the above settings.
To understand these, one of the most important functions of tomcat is the connector.
Its role is the core of tomcat, written in tomcat's configuration file server.xml: The main function of Connector is to receive connection requests, create request and response objects for exchanging data with the requesting end; Container) to process the request, and pass the request and response objects to the engine engine. After the engine processes the request, it will also return the response to the client through the Connector.
Now I finally know the principle, and I know who created the request and response objects.
It can be said that the servlet container needs to be dispatched and controlled by the Connector connector. The Connector connector is the backbone of the tomcat processing request. Therefore, the configuration and use of the Connector have a decisive effect on the performance of tomcat. Let ’s talk Connector configuration options, connection protocol, number of connections, thread pool.
Let's talk about connection protocols first. There are two main types of connection protocols for tomcat: HTTP Connector and AJP Connector (default), we mainly discuss HTTP Connector
一, BIO, NIO, APR
1. Connector's protocol
The Connector uses different protocols when processing HTTP requests. Different Tomcat versions support different protocols. The most typical protocols include BIO, NIO, and APR (these three types are supported in Tomcat7, Tomcat8 adds support for NIO2, and Tomcat8.5 and Tomcat9.0 remove the right BIO support).
BIO is Blocking IO, as the name suggests is blocking IO; NIO is Non-blocking IO, it is non-blocking IO. The APR is Apache Portable Runtime, which is the Apache portable runtime library. Using the local library can achieve high scalability and high performance; Apr is the preferred mode for running highly concurrent applications on Tomcat, but requires the installation of apr, apr-utils, tomcat -Native and other packages. (APR has not been used before, it seems that this protocol will be used under high concurrency)
2. How to specify the protocol
The protocol used by the connector can be specified through the protocol attribute in the <connector> element, or it can use the default value.
The specified protocol value and corresponding protocol are as follows:
HTTP / 1.1: the default value, the protocol used depends on the Tomcat version
org.apache.coyote.http11.Http11Protocol：BIO
org.apache.coyote.http11.Http11NioProtocol：NIO
org.apache.coyote.http11.Http11Nio2Protocol：NIO2
org.apache.coyote.http11.Http11AprProtocol：APR
If the protocol is not specified, the default value of HTTP / 1.1 is used, and the meaning is as follows: in Tomcat7, automatically choose to use BIO or APR (if you find the local library required by APR, use APR, otherwise use BIO); in Tomcat8, automatically Choose to use NIO or APR (if you find the local library required by APR, use APR, otherwise use NIO).
3. What is the difference between BIO / NIO
Whether it is BIO or NIO, the general flow of the Connector to process the request is the same:
Receive the connection in the accept queue (when the client sends a request to the server, if the client and the OS complete a three-way handshake to establish a connection, the OS puts the connection in the accept queue); obtain the requested data in the connection and generate a request; Call the servlet container to process the request; return the response. In order to facilitate the following description, first of all, the relationship between the connection and the request is clear: the connection is at the TCP level (transport layer), corresponding to the socket; the request is at the HTTP level (application layer), and must rely on the TCP connection implementation; a TCP connection Multiple HTTP requests may be transmitted.
In the Connector implemented by BIO, the main entity handling the request is the JIoEndpoint object. JIoEndpoint maintains the Acceptor and Worker: The Acceptor receives the socket, and then finds the idle thread from the Worker thread pool to process the socket. If there is no idle thread in the worker thread pool, the Acceptor will block. Worker is the thread pool that comes with Tomcat. If other thread pools are configured through <Executor>, the principle is similar to Worker.
In the NIO-implemented Connector, the main entity handling the request is the NIoEndpoint object. In addition to the Acceptor and Worker, Poller is used in NIoEndpoint, and the processing flow is shown in the following figure.
After Acceptor receives the socket, instead of directly using the threads in the Worker to process the request, the request is first sent to Poller, and Poller is the key to NIO. Acceptor sends requests to Poller through queues, using a typical producer-consumer model. In Poller, a Selector object is maintained; when Poller takes the socket from the queue, it is registered in the Selector; then iterates through the Selector, finds out the readable socket, and uses the threads in the Worker to process the corresponding request. Similar to BIO, Worker can also be replaced by a custom thread pool.
It can be seen from the above process that in the process of processing requests by NIoEndpoint, whether the Acceptor receives the socket or the thread processes the request, the blocking method is still used; but in the process of "reading the socket and handing it to the worker thread" In the implementation of non-blocking NIO, this is the most important difference between the NIO mode and the BIO mode (other differences have less impact on performance, and will not be mentioned for the time being). This difference can bring a significant improvement in Tomcat efficiency in the case of a large amount of concurrency:
At present, most HTTP requests use a long connection (HTTP / 1.1 default keep-alive is true), and long connection means that a TCP socket will not be released immediately if there is no new request after the current request ends. , But wait for timeout before releasing. If you use BIO, the process of "reading the socket and handing it to the worker thread" is blocked, which means that while the socket is waiting for the next request or waiting for the release, the worker thread that processes the socket will always be occupied. It cannot be released; therefore, the number of sockets that Tomcat can process at the same time cannot exceed the maximum number of threads, and performance is greatly limited. With NIO, the process of "reading the socket and handing it to the worker thread" is non-blocking. When the socket is waiting for the next request or waiting for release, it does not occupy the worker thread, so the number of sockets that Tomcat can handle at the same time is much larger. Due to the maximum number of threads, the concurrent performance is greatly improved.
Two, three parameters: acceptCount, maxConnections, maxThreads
Let's review the process of Tomcat processing the request: receive the connection in the accept queue (when the client sends a request to the server, if the client and the OS complete a three-way handshake to establish a connection, the OS puts the connection in the accept queue); Get the requested data, generate request; call the servlet container to process the request; return the response.
Correspondingly, several parameter functions in Connector are as follows:
1、acceptCount
The length of the accept queue; when the number of connections in the accept queue reaches acceptCount, the queue is full and all incoming requests are rejected. The default value is 100.
2、maxConnections
The maximum number of connections that Tomcat receives and processes at any time. When the number of connections received by Tomcat reaches maxConnections, the Acceptor thread will not read the connections in the accept queue; then the threads in the accept queue will block until the number of connections received by Tomcat is less than maxConnections. If set to -1, the number of connections is unlimited.
The default value is related to the protocol used by the connector: the default value of NIO is 10000, the default value of APR / native is 8192, and the default value of BIO is maxThreads (if Executor is configured, the default value is maxThreads of Executor).
Under windows, the maxConnections value of APR / native will be automatically adjusted to an integer multiple of the largest 1024 below the set value; if set to 2000, the maximum value is actually 1024.
3、maxThreads
The maximum number of request processing threads. The default value is 200 (both Tomcat 7 and 8). If the Connector is bound to an Executor, this value will be ignored, because the Connector will use the bound Executor instead of the built-in thread pool to perform tasks.
maxThreads specifies the maximum number of threads, not the actual number of running CPUs; in fact, the size of maxThreads is much larger than the number of CPU cores. This is because the processing time of the request thread may be very small, and most of the time may be blocked, such as waiting for the database to return data, waiting for the hard disk to read and write data, etc. Therefore, at a certain moment, only a few threads are actually using the physical CPU, and most of the threads are waiting; therefore, it is reasonable that the number of threads is much larger than the number of physical cores.
In other words, Tomcat can make the CPU busy by using a much larger number of threads than the number of CPU cores, greatly improving CPU utilization.
4. Parameter setting
(1) The setting of maxThreads is both related to the characteristics of the application and the number of CPU cores of the server. It can be known from the previous introduction that the number of maxThreads should be much larger than the number of CPU cores; and the larger the number of CPU cores, the larger the maxThreads; the less CPU-intensive (IO-intensive) in the application, the larger the maxThreads, in order to be able to make full use of the CPU. Of course, the value of maxThreads is not as large as possible. If maxThreads is too large, the CPU will spend a lot of time for thread switching, and the overall efficiency will decrease.
(2) The setting of maxConnections is related to the operation mode of Tomcat. If tomcat is using BIO, then the value of maxConnections should be the same as maxThreads; if tomcat is using NIO, then similar to the default value of Tomcat, the value of maxConnections should be much greater than maxThreads.
(3) Through the previous introduction, we can know that although the number of connections that tomcat can handle at the same time is maxConnections, the number of connections that can be received simultaneously in the server is maxConnections + acceptCount. The acceptCount setting is related to what the application wants to do when the connection is too high. If the setting is too large, the waiting time for incoming requests will be very long; if the setting is too small, the incoming requests will immediately return connection refused.
Three, thread pool Executor
The Executor element represents the thread pool in Tomcat and can be shared by other components; to use this thread pool, the component needs to specify the thread pool through the executor attribute.
Executor is an embedded element of Service element. In general, it is the Connector component that uses the thread pool; in order for the Connector to use the thread pool, the Executor element should be placed in front of the Connector. Executor and Connector configuration examples are as follows:
<Executor name="tomcatThreadPool" namePrefix ="catalina-exec-" maxThreads="150" minSpareThreads="4" />
<Connector executor="tomcatThreadPool" port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" acceptCount="1000" />
The main attributes of Executor include:
name: the tag of the thread pool
maxThreads: the maximum number of active threads in the thread pool, the default value is 200 (both Tomcat 7 and 8)
minSpareThreads: the minimum number of threads kept in the thread pool, the minimum value is 25
maxIdleTime: the maximum time the thread is idle, close the thread when the idle exceeds this value (unless the number of threads is less than minSpareThreads), the unit is ms, the default value is 60000 (1 minute)
daemon: whether it is a background thread, the default value is true
threadPriority: thread priority, default value 5
namePrefix: the prefix of the thread name, the thread name in the thread pool is: namePrefix + thread number
Fourth, view the current status
The concept of the number of Tomcat connections and the number of threads and how to set them are described above. The following explains how to view the number of connections and threads in the server.
Viewing the status of the server is roughly divided into two options: (1) use ready-made tools, (2) directly view using Linux commands.
Ready-made tools, such as the jconsole tool that comes with JDK, can easily view thread information (in addition, you can also view CPU, memory, class, JVM basic information, etc.), Tomcat's own manager, charging tool New Relic, etc. The following figure is the interface of jconsole to view thread information:
Let's talk about how to check the number of connections and threads in the server through the Linux command line.
1. Number of connections
Assuming that the port for Tomcat to receive http requests is 8083, you can use the following statement to view the connection:
netstat - night | grip 8083
The results are as follows:
It can be seen that there is a connection in the listen state and listening for requests; in addition, there are 4 established connections (ESTABLISHED) and 2 connections waiting to be closed (CLOSE_WAIT).
2. Thread
The ps command can view the process status, such as executing the following command:
The results are as follows:
It can be seen that only one process information is printed; 27989 is the thread id, and java refers to the executed java command. This is because starting a tomcat, all internal work is completed in this process, including the main thread, garbage collection thread, Acceptor thread, request processing thread, and so on.
Through the following command, you can see how many threads there are in the process; where, nlwp means number of light-weight process.
ps –o nlwp 27989
It can be seen that there are 73 threads in the process; however, 73 does not exclude threads in the idle state. To get the actual number of running threads, you can use the following statement:
ps -eLo pid, state | grip 27989 | grep running | toilet -l
Among them, ps -eLo pid and stat can find all the threads and print the process number and the current state of the thread; the two grep commands filter the process number and thread status; wc counts the number. Among them, ps -eLo pid, stat | grep 27989 output results are as follows:
Only part of the results are screenshots; Sl indicates that most threads are idle.
