background
Use HTTP to interact with services outside the company, and use the default retry strategy DefaultHttpRequestRetryHandler of httpClient 4.5.2. After finding that the interface connection of the other party times out (3 seconds), it will not retry but will be thrown directly ConnectTimeoutException, which does not meet expectations.
conclusion first
The default retry strategy of httpClient, DefaultHttpRequestRetryHandler, does not retry for connection timeout and data acquisition timeout. You need to customize the retry strategy.
problem analysis
Detailed scene
Use HttpClient 4.5.2 to initiate an http request, the code is as follows, the usage is normal
//http设置，retryHandler采用组件提供默认的重试策略DefaultHttpRequestRetryHandler，重试次数为3
return HttpClients.custom()
.setConnectionManager(****)
.setDefaultRequestConfig(****)
.setRetryHandler(new DefaultHttpRequestRetryHandler())
.build();
//请求发起主要代码
HttpPost httpPost = new HttpPost(url);
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs, UTF_8));
CloseableHttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
……其他关闭资源代码
After the request has passed the timeout time of 3s, a ConnectTimeoutException exception is obtained, and there is no retry log in the log. The stack is as follows:
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to www.XXX.com:443 [www.XXX.com/XX.1.XX.12] failed: connect timed out
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:150)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at com.sankuai.meituan.resv.platform.utils.HttpUtil.getResult(HttpUtil.java:124)
... 36 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:337)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
... 46 more
Cause Analysis
When this problem occurs, we have to look at the code of the default retry strategy. DefaultHttpRequestRetryHandler implements HttpRequestRetryHandler, which has such a method public boolean retryRequest(final IOException exception,final int executionCount,final HttpContext context)to determine whether to retry.
Then we try to request facebook (which cannot be accessed by the experimental machine) to reproduce the connection timeout, and perform breakpoint debugging to see the return value of this method. The thrown exception is as expected, that is, ConnectTimeoutException, and the breakpoint is as follows:
The method does return false, i.e. no retry.
code analysis
When the DefaultHttpRequestRetryHandler is instantiated, nonRetriableClasses will be initialized. The result is shown in the red circle in the figure above, and then the inheritance relationship is found public class ConnectTimeoutException extends InterruptedIOException {, so enter the if branch and return false.
Infer from one another and read data timeout
If the connection times out, it will not be retried. Will it be retried if the read data times out? It will not be retried according to the inheritance relationship public class SocketTimeoutException extends java.io.InterruptedIOException. Let's reproduce it by testing.
Reproduction method: http requests a local service, set SocketTimeout to 3s, set a breakpoint on the server, and let the client call timeout. The result is indeed no retry, the exception stack is as follows,
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
solution
Customize the retry strategy, let it return true to retry when some exceptions occur.
private class MyHttpRequestRetryHandler implements HttpRequestRetryHandler{
@Override
public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
if (executionCount > retryTimes){
return false;
}
if (exception instanceof InterruptedIOException
|| exception instanceof NoHttpResponseException) {
// Timeout or 服务端断开连接
return true;
}
// Unknown host
if (exception instanceof UnknownHostException) {
return false;
}
// SSL handshake exception
if (exception instanceof SSLException) {
return false;
}
final HttpClientContext clientContext = HttpClientContext.adapt(context);
final HttpRequest request = clientContext.getRequest();
boolean idempotent = !(request instanceof HttpEntityEnclosingRequest);
if (idempotent) {
// Retry if the request is considered idempotent
return true;
}
return false;
}
}
Several timeout setting parameters
/** 从连接池获取连接超时时间 **/
private int connectionRequestTimeout = 500;
/** 连接一个url的连接等待时间,建立socket连接时间 **/
private int connectTimeout = 3000;
/** 传输数据等待返回的超时时间 **/
private int socketTimeout = 3000;
reflection summary
When coding, it is believed that the default retry strategy will retry when the connection times out, but the test of this abnormal case is not actually carried out, nor the code implementation of the default strategy, so there is a timeout and no retry on the line.
Summary : The processing method developed for the abnormal process, although it will be more troublesome when testing, but it should be covered to ensure that the recovery process can work normally when an abnormality occurs on the line.