HttpClient's keepalive for efficient use of long connections

Summary of keepalive's expiration time:
Client (httpclient creates and maintains a socket connection pool): 1. HTTP layer: It expires through the getKeepAliveDuration time negotiated with the server. By default, it returns -1, that is, it is permanently valid. 2. Connection pool layer: The expired socket is recycled through the startup job program IdleConnectionsEvictor of the connection pool, and the expiration time is determined by configuring maxIdleTime. 3. Linux operating system layer (vi /etc/sysctl.conf): configure net.ipv4.tcp_keepalive_time = 1200 net.ipv4.tcp_keepalive_probes = 5 net.ipv4.tcp_keepalive_intvl = 15 to control the recovery time.
Server side (connection failure time of the accessed end): 1. The application container layer, such as keepAliveTimeout="60000" in tomcat to control the failure time of the connection (if keepAliveTimeout is not set, take the time of connectionTimeout="10000" as the failure time of keepalive ). 2. Operating system layer: same as above.
Summary: If the expiration time on the server side is less than the expiration time on the client side, the following errors may occur in the critical state:
Error 1: java.net.SocketException: Connection reset
error 2: java.net.SocketException: Broken pipe
error 3: org.apache.http.NoHttpResponseException:
*** Failed to respond will cause access problems, so I personally recommend that the client's expiration time be less than the server's expiration time.

The first stage of literacy
It is well known that http1.1 (http1.0 is not a standard, it depends on the server) supports long connections, which can ensure that the sockets of the server and the client can be used efficiently and reduce additional overhead such as handshakes. Under normal circumstances, httpClient will bring Connection: keep-alive, which means that I support long connections. When a request is completed, it depends on the situation to decide whether to close the connection.
Under normal circumstances, the server will bring the following information in the header of the returned content:

but if the server-side connection count reaches the specified value, it will add Connection: close information to the returned content, indicating that the connection will be closed.

At this time, the connection is invalid. If the client requests the server next time, it needs to re-create a new connection. But there is a problem, whether the connection is maintained is determined by the server side. Once the connection timeout causes the connection close, the server will not specifically notify the client, so we cannot rely on the Connection: close information to decide whether to close the socket.

Second, the application container timeout setting (the server side is valid, the client connection pool is not controlled)
Let's first take a look at the configuration of tomcat for long connections. The following is a typical way to configure a long connection, among which:
keepAliveTimeout: Indicates how long tomcat keeps the connection before the next request. This means that if the client keeps getting requests and it exceeds the expiration time, the connection will always be maintained.
maxKeepAliveRequests: Indicates the maximum number of requests supported by the connection. Connections that exceed this number of requests will also be closed (in which case a Connection: close header is returned to the client).
Due to the uncertainty of keeplive, this brings certain challenges to the client to implement HTTP requests efficiently. Multiple mechanisms are used to ensure the multiplexing of connections in httpClient at the same time, but this needs to be coordinated with the server side. If the server-side and client-side keepAlive-related configurations do not match, the efficiency will be low at light, and the muscles and bones will be broken in severe cases. What are the specific points that need to be paid attention to? Come to the following one by one.
TOMCAT configuration
<Connector port="8088" protocol="org.apache.coyote.http11.Http11NioProtocol"
                connectionTimeout="10000"
                redirectPort="8443"
                URIEncoding="UTF-8"
                maxConnections=" 5000" maxThreads
                ="5000"
                maxHttpHeaderSize ="20480"
                keepAliveTimeout="60000"
                maxKeepAliveRequests="300"
               acceptCount="2500" />



httpClient provides a connection pool to reuse connections, which is the basis for efficient requests. The following is how to set a connection pool for httpClient:
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
//The maximum number of connections generated by the connection pool is 200
cm.setMaxTotal( 200 ) .
_ _ _ new HttpRoute(localhost), 50); // Create httpClient CloseableHttpClient httpClient = HttpClients.custom()         .setConnectionManager(cm)         .build(); There are several things to pay attention to: 1. If your client connects to the target server There is only one, then the maximum number of route connections can be set to be the same as the maximum number of connection pool connections, so as to efficiently utilize the connections created in the connection pool.










2. The created httpClient object is thread-safe. If there is only one connected target server, create a global object. An object is like opening a browser, multiple threads do not need to open a browser for each request, just unify one.
3. If the httpClient object is no longer used, remember to close it and release the socket that remains connected to the server, so that the server can release resources more efficiently.

4. Client connection pool management, cleaning up expired connections (the client is valid).
Although the connection pool is available, due to the particularity of the http connection (only when the communication is in progress (block) can it respond to IO events). Once the connection is put back into the connection pool, we have no way of knowing whether the connection is still keepalive, and we cannot monitor the status of the current socket at this time (that is, the server actively closes the connection, but the client does not know the status of the current connection when there is no communication) how is it). How to do it? httpClient adopts a compromise scheme to check the "status" of the connection, that is, the client itself configures itself to actively close the connection that it considers to be invalid. The specific methods are as follows:
public final class IdleConnectionEvictor {

    private final HttpClientConnectionManager connectionManager;
    private final ThreadFactory threadFactory;
    private final Thread thread;
    private final long sleepTimeMs;
    private final long maxIdleTimeMs;

    private volatile Exception exception;

    public IdleConnectionEvictor(
            final HttpClientConnectionManager connectionManager,
            final ThreadFactory threadFactory,
            final long sleepTime, final TimeUnit sleepTimeUnit,
            final long maxIdleTime, final TimeUnit maxIdleTimeUnit) {
        this.connectionManager = Args.notNull(connectionManager, "Connection manager");
        this.threadFactory = threadFactory != null ? threadFactory : new DefaultThreadFactory();
        this.sleepTimeMs = sleepTimeUnit != null ? sleepTimeUnit.toMillis(sleepTime) : sleepTime;
        this.maxIdleTimeMs = maxIdleTimeUnit != null ? maxIdleTimeUnit.toMillis(maxIdleTime) : maxIdleTime;
        this.thread = this.threadFactory.newThread(new Runnable() {
            @Override
            public void run() {
                try {
                    while (!Thread.currentThread().isInterrupted()) {
                        Thread.sleep(sleepTimeMs);
                        connectionManager.closeExpiredConnections();
                        if (maxIdleTimeMs > 0) {
                            connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS);
                        }
                    }
                } catch (Exception ex) {
                    exception = ex;
                }

            }
        });
    }
We create a new Thread, which cleans up every 5s and closes connections that are considered expired or idle for more than 30s. It is easy to understand here to close idle connections. The problem is to close the expired connection, what kind of connection is expired? I just said that the client can't detect whether the corresponding connection is alive, so through a configuration to tell httpClient about when each connection will be disconnected, httpClient will think that the connection after this time is an expired connection. By default, httpClient thinks that the socket will never be disconnected as long as it is connected to the server. Obviously this assumption is too optimistic.

Using a recycling job: You need to create and start a job for it to take effect.
IdleConnectionsEvictor idleConnectionsEvictor = new IdleConnectionsEvictor(clientConnectionManager,
    cfg.getUpdateConnectionsEvictorSleepDelay(), TimeUnit.MILLISECONDS,
    cfg.getMaxUpdateConnectionIdleTime(), TimeUnit.MILLISECONDS);
idleConnectionsEvictor.start();

Key 1: connectionManager.closeExpiredConnections(); where the general expiry will be relatively large , generally the default expiration time is more than one month
if(timeToLive > 0L) {
    this.validityDeadline = this.created + tunit.toMillis(timeToLive);
} else {
    this.validityDeadline = 9223372036854775807L;
}

this.expiry = this.validityDeadline;
重点2：connectionManager.closeIdleConnections(maxIdleTimeMs, TimeUnit.MILLISECONDS);连接池中的失效时间为设置的maxIdleTimeMs。
public void closeIdle(long idletime, TimeUnit tunit) {
    Args.notNull(tunit, "Time unit");
    long time = tunit.toMillis(idletime);
    if(time < 0L) {
        time = 0L;
    }

    final long deadline = System.currentTimeMillis() - time;
    this.enumAvailable(new PoolEntryCallback() {
        public void process(PoolEntry<T, C> entry) {
            if(entry.getUpdated() <= deadline) {
                entry.close();
            }

        }
    });
}
//updated time is the time when the object was created
this.updated = System.currentTimeMillis();

5. Client HTTP layer keepalive retention time (valid for the client, returns -1 by default, valid permanently)
keepalive strategy: In order for connMgr.closeExpiredConnections(); to work, we need to specify the connection keep alive strategy to tell httpClient which connections will expire and when , you can turn them off. Here is an example of setting a keep alive strategy:
ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        // Honor 'keep-alive' header
        HeaderElementIterator it = new BasicHeaderElementIterator(
                response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                try {
                    return Long.parseLong(value) * 1000;
                } catch(NumberFormatException ignore) {
                }
            }
        }
        HttpHost target = (HttpHost) context.getAttribute(
                HttpClientContext.HTTP_TARGET_HOST);
        if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {
            // Keep alive for 5 seconds only
            return 5 * 1000;
        } else {
            // otherwise keep alive for 30 seconds
            return 30 * 1000 ;
        }
    }
};
CloseableHttpClient client = HttpClients.custom()
        .setKeepAliveStrategy(myStrategy)
        .build();
This example specifies keeplive as a fixed value when accessing www.***.com and other unknown servers, if the server returns The keeplive expiration time tells the client when the connection will expire through the server (note that this is not a standard http protocol, and not all servers support it).

6. Summary
If httpClient is accessing a single address (such as rpc request), the connection pool of the client can be used more efficiently by cooperating with the server configuration. For example, taking tomcat as an example, setting maxKeepAliveRequests to -1 can make the connection request unlimited times, while Setting keepAliveTimeout to a larger time allows the client to use the connections in the connection pool more efficiently. Everything has a premise. If the target tomcat is used for both rpc calls and normal connection requests, the above optimization will consume a lot of server resources. At this time, it can be said that these two types of services are split into two tomcats for separate processing. In addition, if there is a proxy (apache, nginx) in front of the server, the keepAlive policy of the proxy needs to be set according to the situation to improve the utilization.