--- --- restore content begins
1, HttpClient Profile
Compared to traditional JDK comes with URLConnection HttpClient, increased ease of use and flexibility, it is not only the client sends Http request easier, but also easier for developers to test interface (Http-based protocol), improves the efficiency of the development also convenient to improve the robustness of the code. Therefore, it is important to master HttpClient compulsory content, the master HttpClient, I believe will be more in-depth understanding Http protocol.
The difference Commons HttpClient project org.apache.commons.httpclient.HttpClient and org.apache.http.client.HttpClient now is the end of life, no longer being developed, has been replaced by Apache HttpComponents project HttpClient and HttpCore modules, providing more good performance and greater flexibility.
2、HTTP的Keep-Alive
In front of the blog http protocol , the http request header can be seen that the connection can be set can be set to Keep-Alive, using the Keep-Alive to default values, if necessary to close the need to manually close the HTTP / 1.1.
In HTTP 1.0 Previously, each http request is required to open a TCP socket connection and once after the TCP connection is disconnected, which can lead to frequent creation and destruction of TCP. HTTP 1.1 can be improved by using such a keep-alive state, which can continue to send data in multiple without a TCP connection is disconnected, in order to improve performance and increase throughput http server (fewer connecting means tcp with fewer system kernel calls, socket the accept () and close () call).
2.1 HTTP的Keep-Alive
When maintain a long connection, how to judge a request has been completed?
The Length-the Content
the Content-length indicates the Length of the substantial contents. With this field the browser to determine whether the current requested data have all been received.
So, when a browser request is a static resource, that the server can know exactly when the return length of the content, Content-Length can be set to control the end of the request. But when the server does not know the result of the length of the request, such as a page or dynamic data, Content-Length can not solve the above problem, this time need to use Transfer-Encoding field.
Transfer-Encoding
Transfer-Encoding refers to the transmission coding, the above problem, when the server can not know the length of the content of an entity, it can specify Transfer-Encoding: to inform the current encoding browser chunked it is the data into a one transfer. Of course, can also specify Transfer-Encoding: gzip, chunked entity indicates that the content is not only gzip compression, or block transfer. Finally, when the browser receives a length chunked 0, knowing that the current request have been received content.
Alive timeout-the Keep:
Httpd daemon, generally provide a keep-alive timeout time setting parameters. For example, the nginx keepalive_timeout, and Apache KeepAliveTimeout. This keepalive_timout time value means: tcp connection http generated after a transfer after the last one response, but also hold live keepalive_timeout seconds later, I began to close the connection.
When the httpd daemon after sending a response, should immediately take the initiative to close the corresponding tcp connection, after setting keepalive_timeout, httpd daemon will say: "wait, and see there is no browser requests come," and this and other , is keepalive_timeout time. If the daemon in the waiting time, never received the browser sent me http request, then close the http connection.
2.2 Tcp的Keep-Alive
After tcp link is established, if the application or upper-layer protocol has not sending data, or across a very long time before sending data, how to determine when the other side also online link for a long time there is no data packet transmission, in the end is dropped, or does not data transmission link need not need to keep this situation is the need to take into account in the design of the TCP protocol.
TCP protocol to solve this problem through an ingenious way, when over a period of time, TCP automatically sends a data packet to empty the other side, if the other party to respond to this message, indicating that the other side also online, the link may remain, If they do not return messages, and retry several times after the link is considered lost, there is no need to maintain the link.
2.3 http keep-alive与tcp keep-alive
The application layer HTTP network protocol stack, and the TCP transport layer is located in a network protocol stack, both the KEEP-ALIVE Although the same name, but different functions. http keep-alive is to allow tcp live a little longer, in order to transmit multiple http on the same connection, improve the efficiency of the socket. And preservation is tcp keep-alive TCP connection status detecting mechanism of the TCP. T detect whether the peer is still alive.
2.4 turn on the advantages and disadvantages of Keep-Alive
Advantage: Keep-Alive mode is more efficient, because avoiding the overhead of connection establishment and release.
Drawback: Tcp connection for a long time easily lead to invalid system resources consumption, waste of system resources.
So for applications requiring frequent transmission of the HTTP request, keep-alive need to open the client using HTTP long connection.
3 HttpClient settings
= httpClient HttpClients.custom () // connection pool configuration .setConnectionManager (poolingHttpClientConnectionManager) // requestConfig arranged .setDefaultRequestConfig (requestConfig) .disableCookieManagement () .disableConnectionState () .disableAuthCaching () // Default configuration socketConfig .setDefaultSocketConfig (socketConfig) // default head is configured .setDefaultHeaders (defaultHeaders) // retry handle .setRetryHandler(httpRequestRetryHandler) .build ();
3.1 PoolingHttpClientConnectionManager connection pool provided
Two hosts during connection establishment is a very complex process, involving a plurality of packets exchanged, and also time consuming. Http connection requires three-way handshake much overhead, this overhead is relatively small for the bigger news is http. But if we use the established direct http connection, this cost is relatively small, greater throughput. When a large amount of high concurrent requests from the network, connection pool can improve throughput.
= Cm & lt PoolingHttpClientConnectionManager new new PoolingHttpClientConnectionManager (Registry); // Set the entire connection pool maximum number of connections cm.setMaxTotal (maxTotal); // Set default route for each of the maximum number of connections cm.setDefaultMaxPerRoute (maxPerRoute); HttpHost HttpHost = new new HttpHost ( hostname, Port); // set the maximum number of connections a route, giving priority to defaultMaxPerRoute. cm.setMaxPerRoute ( new new HttpRoute (HttpHost), maxRoute); // This method continuously connected to a closed over time, and removed from the pool. cm.closeExpiredConnections (); // This method closes the connection idle for more than timeout, idle time from when returned to the connection pool, regardless of whether expired, more than the idle time is closed. cm.closeIdleConnections (timeout, tunit);
connectionConfig Configuration
// message constraint MessageConstraints messageConstraints = MessageConstraints.custom () .setMaxHeaderCount ( 200 is ) .setMaxLineLength ( 2000 ) .build (); // the Http Connection configuration ConnectionConfig connectionConfig = ConnectionConfig.custom () .setMalformedInputAction (CodingErrorAction.IGNORE) .setUnmappableInputAction (CodingErrorAction.IGNORE) .setCharset (Consts.UTF_8) .setMessageConstraints (messageConstraints) .build (); // not generally modify HTTP connection configuration, which is not provided // cm & lt .setDefaultConnectionConfig (connectionConfig); //cm.setConnectionConfig(new HttpHost("somehost", 80), ConnectionConfig.DEFAULT);
Specific reference may parse source: https: //www.cnblogs.com/shoren/p/httpclient-leaseConnection.html
3.2 RequestConfig settings
Mainly used for external access and configure some network environments
RequestConfig = RequestConfig RequestConfig.custom () // set obtained from connectManager Connection Timeout .setConnectionRequestTimeout (1000 ) // set connection time .setConnectTimeout (10000 ) // Request timeout to .setSocketTimeout (10000 ) // determine whether should automatically handles authentication .setAuthenticationEnabled ( to true ) // determine redirect loop (redirects to the same location) whether to redirect .setCircularRedirectsAllowed ( false ) // maximum number of redirects. Restrictions on the number of redirects is to prevent infinite loop .setMaxRedirects (. 5 ) // determines whether an automatic processing should be redirected .setRedirectsEnabled ( to true ) // determines whether to reject relatively redirection. HTTP specification requires absolute position value is a URI of the .setRelativeRedirectsAllowed ( to true ) // determine whether it should automatically decompress compressed entity .setContentCompressionEnabled ( to true ) // determine the name of the cookie specification for HTTP state management of .setCookieSpec ( "" ) // returns the local address requested to be executed. On a computer having a plurality of network interfaces, this parameter can be used to select a network interface is generated. .setLocalAddress () // proxy configuration .setProxy () // When using a proxy host to authenticate, prioritize supported authentication scheme. .setProxyPreferredAuthSchemes () // When using the target host authentication, determine order of preference by the authentication mode supported .setTargetPreferredAuthSchemes () .build ();
3.3 SocketConfig Configuration
SocketConfig.custom () // open the monitor TCP connection is valid .setSoKeepAlive ( false ) // After Can I turn off Socket in a process, even if it did not release the port, other processes can also be reused immediately port .setSoReuseAddress ( to true ) // timeout time in ms reception data .setSoTimeout (10000 ) // if the data is transmitted immediately, set to true off Socket buffer, defaults to false .setTcpNoDelay ( to false ) .build ();
3.4 defaultHeader Configuration
Collection<Header> defaultHeaders = new ArrayList<>(); defaultHeaders.add(new BasicHeader(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate")); defaultHeaders.add(new BasicHeader(HttpHeaders.ACCEPT_LANGUAGE, "zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3")); defaultHeaders.add(new BasicHeader(HttpHeaders.ACCEPT, "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")); defaultHeaders.add(new BasicHeader(HttpHeaders.CONNECTION, "keep-alive"));
3.5 HttpRequestRetryHandler配置
// Disable Retry (parameters: retryCount, requestSentRetryEnabled) HttpRequestRetryHandler requestRetryHandler = new new DefaultHttpRequestRetryHandler (0, to false ); // custom retry strategy httpRequestRetryHandler = new new HttpRequestRetryHandler () { public Boolean retryRequest (IOException Exception, int executionCount, the HttpContext context) { IF (executionCount> = 3) { // If the retry has three times, to give return to false ; } IF (Exception the instanceof NoHttpResponseException) { // If the server lost connection, then try again return to true ; } IF (Exception the instanceof SSLHandshakeException) { // Do not retry SSL handshake exception return to false ; } IF (Exception the instanceof an InterruptedIOException) { // timeout return to false ; } IF (Exception the instanceof UnknownHostException) { // the target server is unreachable return to false ; } IF (Exception the instanceof ConnectTimeoutException) { // connection refused return false; } if (exception instanceof SSLException) {// SSL握手异常 return false; } return false; }
}
In fact, we are using the default socketConfig and connectionConfig in actual use. Related connections (e.g. maxTotal, maxPerRoute) in practical applications, as well as relating to the request timeout (e.g. connectionTimeout, socketTimeout, connectionRequestTimeout) is more important.
Specific connection pooling principle Reference document:
HttpClient 4.3 connection pool configuration parameters and interpret source
httpClient 4.3.x configuration official sample
Use httpclient must know the parameters, and code writing, risks
HttpClient connection pool maintained overtime and failure mechanisms
HttpClient connections and a connection pool principle timing diagram
--- end --- restore content