httpClient notes

NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity

Error message: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
Solution:

public static InputStream getAsStream(String url) {
    
    
    try (CloseableHttpClient client = HttpClients.createDefault();
         CloseableHttpResponse response = client.execute(new HttpGet(url))) {
    
    
        HttpEntity httpEntity = response.getEntity();
        // 包装一层解决
        BufferedHttpEntity bhe = new BufferedHttpEntity(httpEntity);
        return bhe.getContent();
    } catch (IOException e) {
    
    
        LOGGER.error("doGet failed: " + e.getMessage());
    }
    return null;
}

Reference:
stackoverflow

Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.

Using HttpClient, a large number Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommendedof WARN logs reported , locate the source code of HttpClient as follows:

public abstract class HttpMethodBase implements HttpMethod {
    
    
    public byte[] getResponseBody() throws IOException {
    
    
        if (responseBody == null) {
    
    
            InputStream instream = getResponseBodyAsStream();
            if (instream != null) {
    
    
                long contentLength = getResponseContentLength();
                if (contentLength > 2147483647L) {
    
    
                    throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
                }
                int limit = getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
                if (contentLength == -1L || contentLength > (long) limit) {
    
    
                    LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
                }
                LOG.debug("Buffering response body");
                ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength <= 0L ? 4096: (int) contentLength);
                byte buffer[] = new byte[4096];
                int len;
                while ((len = instream.read(buffer)) > 0) {
    
    
                	outstream.write(buffer, 0, len);
                }
                outstream.close();
                setResponseStream(null);
                responseBody = outstream.toByteArray();
            }
        }
        return responseBody;
    }
}

The condition for reporting WARN (contentLength == -1) || (contentLength > limit), that is, the returned HTTP header does not specify contentLength, or contentLength is greater than the upper limit (default 1M). If it can be determined that the size of the returned result has no significant impact on the program, this WARN can be ignored, and the log level of HttpClient can be adjusted to ERROR in the log configuration.

But this is all about ignoring potential problems and not solving the problem.

HttpClient recommends using InputStream getResponseBodyAsStream() instead of byte[] getResponseBody(). For situations where the returned result is large or unpredictable, use InputStreamgetResponseBodyAsStream() to avoid the memory exhaustion problem caused by byte[] getResponseBody() .

The solution is to convert stream to string:

private String convert(InputStream inputStream) throws IOException {
    
    
    BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
    StringBuilder sb = new StringBuilder();
    String str;
    while ((str = br.readLine()) != null) {
    
    
        sb.append(str);
    }
    return sb.toString();
}

reference

https://blog.csdn.net/xiaoshuji/article/details/71077619