Why use an HTTP connection pooling?
With the back-end system architecture style gradually forward split architecture, micro-architecture service change, RestFul style API development and design, while SpringMVC also good support REST-style interface. Call services between the various systems they use HTTP + JSON or HTTPS + JSON way.
HTTP1.1 default connection is persistent, HTTP1.0 may be provided by the request header Connection: keep-alive becomes long so that the connection is connected. Since the HTTP protocol support long connection, HTTP connection can also use the connection pool technology to manage and maintain the connection establishment and destroyed. However, because each TCP connection request HTTP connection actually established in the transport layer, using the communication socket, and closed to establish an HTTP connection remains virtually the same TCP connection and are closed related, each HTTP request has all after three TCP connection handshake (a connection) and four wave (release the connection) process. So the use of HTTP connection pool has the following advantages:
Reduce the frequent setup time HTTP connection overhead, reducing waste TCP connection socket communication server resources when establishing and released;
Support a higher number of concurrent;
Common API connection pool HttpClient
PoolingHttpClientConnectionManager connection pool management implementation class
PoolingHttpClientConnectionManager HttpClient is a connection pool implementation class, and implements HttpClientConnectionManager ConnPoolControl interfaces.
Constructors:
PoolingHttpClientConnectionManager (): constructor with no arguments, you can see the source code from the method calls getDefaultRegistry () to register the protocol http and https protocols.
Common methods:
public void setMaxTotal (int max): This method is defined in ConnPoolControl interface, represents the maximum number of connections is provided max.
public void setDefaultMaxPerRoute (int max): This method is also defined in ConnPoolControl interface, it represents the maximum number of connections per default route set to max
public void setMaxPerRoute (HttpRoute route, int max): the maximum number of connections is provided a specified route the value of this configuration will overwrite setDefaultMaxPerRoute certain routes.
The method commonly used
static RequestConfig.Builder custom (): static methods for constructing Builder object, and then set the parameters;
int getConnectionRequestTimeout (): Get Get longest connection, in milliseconds from the connection pool;
int getConnectTimeout (): Get the maximum time to create a connection, in milliseconds;
int getSocketTimeout (): Get the maximum time for data transmission, in milliseconds;
RequestConfig has an internal static class Builder, for conveniently constructed RequestConfig object and parameter setting request, such conventional methods have the following:
public RequestConfig.Builder setConnectionRequestTimeout (connectionRequestTimeout int): Get the maximum time setting of the connection, in milliseconds from the connection pool;
public RequestConfig.Builder setConnectTimeout (int connectTimeout): Create a set maximum time of connection, in milliseconds;
public RequestConfig.Builder setSocketTimeout (int socketTimeout): setting the maximum time for data transmission, in milliseconds;
HttpRequestRetryHandler request retry Interface
boolean retryRequest (IOException exception, int executionCount , org.apache.http.protocol.HttpContext context): implements the interface must implement this method, a method determines if the IO occurred during execution of the abnormality, should retry, retry executionCount times.
Threaded - connection pool manager using HTTP requests
The main steps:
1. Create a HTTP connection pool management object cm, as shown
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
2. Set the default parameters and the maximum number and the maximum number of connections each connection routing, as shown below
// increase the maximum number of connections 200 is
cm.setMaxTotal (200 is);
// default maximum number of connections for each route increases 20 is
cm.setDefaultMaxPerRoute (20 is);
3. When HttpPost analog transmission request or a request HttpGet, attention here to create HttpClients objects need to obtain from the connection pool, i.e. to set the connection pool object, IO exception occurs when executing the processing required to implement the interface HttpRequestRetryHandler; Note that this after processing in response to the request can not be closed HttpClient object, if closing the connection pool will be destroyed. HttpClients object creates objects shown:
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.setRetryHandler(retryHandler(5)).build();
4. The thread can be turned to listen for connection pool idle connections, and clean invalid connection, a thread can set their own listening frequency.
Java implementation Source:
package com.liangpj.develop.httpclient;
import org.apache.http.HttpEntityEnclosingRequest;
import org.apache.http.HttpRequest;
import org.apache.http.NoHttpResponseException;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.conn.HttpClientConnectionManager;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLHandshakeException;
import java.io.IOException;
import java.io.InterruptedIOException;
import java.net.UnknownHostException;
/**
* 单线程-使用连接池管理HTTP请求
* @author: liangpengju
* @version: 1.0
*/
public class HttpConnectManager {
public static void main(String[] args) throws Exception {
//创建HTTP的连接池管理对象
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
// increase the maximum number of connections 200 is
connectionManager.setMaxTotal (200 is);
// default maximum number of connections is increased for each route 20 is
connectionManager.setDefaultMaxPerRoute (20 is);
// The http://www.baidu.com: connector 80 is increased to the maximum 50
// new new HttpHost HttpHost HttpHost = ( "http://www.baidu.com", 80);
//connectionManager.setMaxPerRoute(new HttpRoute (HttpHost), 50);
// initiating 3 GET request
String URL = "https://www.baidu.com/s?word=java";
Long Start = System.currentTimeMillis ();
for (int I = 0; I <100; I ++) {
the doGet (ConnectionManager, URL);
}
Long End = System.currentTimeMillis ();
System.out.println ( "Consume ->" + (End - Start));
// Cleanup invalid connections
new new IdleConnectionEvictor (ConnectionManager) .start ();
}
/ **
* request retry processing
* retry tryTimes @param
* @return
* /
public static HttpRequestRetryHandler retryHandler (Final tryTimes int) {
HttpRequestRetryHandler = new new httpRequestRetryHandler HttpRequestRetryHandler () {
@Override
public Boolean retryRequest (IOException Exception, int executionCount, the HttpContext context) {
// If the retry n times already, to give
IF (executionCount> = tryTimes) {
return to false;
}
// if the server lost connection, then retry
IF (Exception the instanceof NoHttpResponseException) {
return to true;
}
// do not retry SSL handshake exception
IF (Exception the instanceof SSLHandshakeException) {
return to false;
}
// timeout
IF (Exception the instanceof an InterruptedIOException) {
return to false;
}
// the target server is unreachable
IF (Exception the instanceof UnknownHostException) {
return to true;
}
// connection is rejected
if (exception instanceof ConnectTimeoutException) {
to false return;
}
// the SSL handshake exception
IF (Exception the instanceof an SSLException) {
return to false;
}
HttpClientContext ClientContext = HttpClientContext .adapt (context);
the HttpRequest clientContext.getRequest Request = ();
// if the request is idempotent, it again try
IF ((Request the instanceof HttpEntityEnclosingRequest)!) {
return to true;
}
return to false;
}
};
return httpRequestRetryHandler;
}
/ **
* the doGet
* @Param url request address
* @param ConnectionManager
* @throws Exception
* /
public static void the doGet (HttpClientConnectionManager ConnectionManager, String URL) throws Exception {
// Get the client object from the connection pool, many cases of
CloseableHttpClient httpClient = HttpClients.custom ()
.setConnectionManager (ConnectionManager)
. .setRetryHandler (retryHandler (. 5)) build ();
// Create http GET request
HttpGet HttpGet = new new HttpGet (URL);
// build request configuration information
RequestConfig config = RequestConfig.custom () setConnectTimeout ( . 1000) // create the longest connection
.setConnectionRequestTimeout (500) // get a connection from the connection pool maximum time
.setSocketTimeout (10 * 1000) // maximum time of data transmission 10s
before .setStaleConnectionCheckEnabled (true) // submit a request to test whether the connection is available
.build ();
// setup request configuration information
httpGet.setConfig (config);
CloseableHttpResponse Response null =;
the try {
// perform the requested
Response = httpClient.execute (HttpGet);
// determines whether the state is returned 200 is
iF {(response.getStatusLine () getStatusCode () == 200 is.)
String Content = EntityUtils.toString (Response .getEntity (), "UTF-. 8");
System.out.println ( "content-length:" + content.length ());
}
} the finally {
! IF (Response = null) {
response.close ();
}
// here can not be closed httpClient, if you turn off httpClient, connection pooling will destroy
// httpClient.close ();
}
}
/ **
* monitor the connection pool idle connections, clean up invalid connection
* /
the extends the Thread class IdleConnectionEvictor static public {
Private Final HttpClientConnectionManager ConnectionManager;
Private volatile Boolean the shutdown;
public IdleConnectionEvictor (HttpClientConnectionManager ConnectionManager) {
this.connectionManager = ConnectionManager;
}
@Override
public void RUN () {
the try {
the while {(the shutdown!)
the synchronized (the this) {
// check 3S once
the wait (3000);
// close connection failure
connectionManager.closeExpiredConnections ();
}
}
} the catch (InterruptedException EX) {
// End
ex.printStackTrace ();
}
}
public void the shutdown () {
the shutdown = to true;
the synchronized (the this) {
notifyAll ();
}
}
}
}
Multithreading -HttpClient example HTTP requests connection pool manager
The main steps:
1. Create a HTTP connection pool management object cm, as shown
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
2. Set the default parameters and the maximum number and the maximum number of connections each connection routing, as shown below
// increase the maximum number of connections 200 is
cm.setMaxTotal (200 is);
// default maximum number of connections for each route increases 20 is
cm.setDefaultMaxPerRoute (20 is);
3. Create HttpClients connection pool object and objects, HttpClients create an object shown in the object:
CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(connectionManager).build();
4. Thread class inheritance to achieve a Get request execution thread class GetThread, overloaded run () method is executed HttpGet request;
5. To request URI address is defined as an array, a GetThread create a thread for each URI, and start all threads;
the Java source code to achieve:
package com.liangpj.develop.httpclient;
import org.apache.http.HttpEntity;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.protocol.HttpContext;
import java.io.IOException;
/**
* 多线程-HttpClient连接池管理HTTP请求实例
*/
public class MultiThreadHttpConnManager {
static void main public (String [] args) {
// connection pool object
PoolingHttpClientConnectionManager new new PoolingHttpClientConnectionManager ConnectionManager = ();
// increase the maximum number of connections 200 is
connectionManager.setMaxTotal (200 is);
// default route for each maximum connection increased number of 20 is
connectionManager.setDefaultMaxPerRoute (20 is);
// HttpClient objects
CloseableHttpClient httpClient = HttpClients.custom () setConnectionManager (ConnectionManager) .build ();.
// URIs to doGet
String [] = {urisToGet
"HTTPS: // WWW .baidu.com / S? = the Java Word ",
" https://www.baidu.com/s?word=java ",
" https://www.baidu.com/s?word=java ",
"https://www.baidu.com/s?word=java"
};
//为每一个URI创建一个线程
GetThread[] threads = new GetThread[urisToGet.length];
for (int i=0;i<threads.length;i++){
HttpGet httpGet = new HttpGet(urisToGet[i]);
threads[i] = new GetThread(httpClient,httpGet);
}
//启动线程
for (int j=0;j<threads.length;j++){
threads[j].start();
}
//join 线程
for(int k=0;k<threads.length;k++){
try {
threads[k].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
/**
* 执行Get请求线程
*/
public static class GetThread extends Thread{
private final CloseableHttpClient httpClient;
private final HttpContext context;
private final HttpGet httpget;
public GetThread(CloseableHttpClient httpClient, HttpGet httpget) {
this.httpClient = httpClient;
this.context = HttpClientContext.create();
this.httpget = httpget;
}
@Override
public void run() {
try {
CloseableHttpResponse response = httpClient.execute(httpget,context);
{the try
the HttpEntity = response.getEntity Entity ();
} {the finally
response.close ();
}
} the catch (ClientProtocolException EX) {
// handle client protocol anomaly
} the catch (IOException EX) {
// IO exception handling client
}
}
}
}