OkHttp Cache

• This article is not authorized to be reproduced by the Programmer Lighthouse website, and the source is not indicated. It is a junk website and stole my article.
  https://www.wangt.cc/2021/11/okhttp cache article

OkHttp

• Official website documentation: https://square.github.io/okhttp/
• Design Patterns: Builder Pattern, Chain of Responsibility Pattern
• object pool, connection pool

Connection Interceptor

• https://blog.csdn.net/followYouself/article/details/121086869

Demo code

• addInterceptor:Application layer interceptor, intercepts before network requests. addNetworkInterceptor:Network layer interceptor, intercepts after initiating a network request.
• callTimeout:The overall timeout for this request. Including connect, write, read and other phase times.
• connectTimeout: Connection phase timeout, default is 10 seconds. Configure tcp layer socket parameters, java.net.Socket#connect(java.net.SocketAddress, int). Refer to Okhttp source code RealConnection#connectSocket.
• readTimeout:socket readfunction timeout, default is 10 seconds. Configure tcp layer socket parameters, java.net.Socket#setSoTimeout. Refer to Okhttp source code RealConnection#connectSocket.
• writeTimeout: Connection phase timeout, default is 10 seconds.

// 定义一个拦截器
Interceptor interceptor = chain -> {
    
    
    Request request = chain.request();

    long t1 = System.nanoTime();
    Log.i(TAG, String.format("Send request %s on %s%n%s", request.url(), chain.connection(), request.headers()));

    Response response = chain.proceed(request);

    long t2 = System.nanoTime();
    Log.i(TAG, String.format("Received response for %s in %.1fms%n%s",
            response.request().url(), (t2 - t1) / 1e6d, response.headers()));

    return response;
};
OkHttpClient client = new OkHttpClient.Builder()
        .addInterceptor(interceptor) // 应用层拦截器，在网络连接前生效
        .addNetworkInterceptor(interceptor) // 网络请求前的拦截器，在网络连接后生效
        .cache(new Cache(new File("/data/data/com.test.http/cache", "http_cache"), 50 * 1024 * 1024)) // 50 MB
        .callTimeout(5, TimeUnit.SECONDS)
        .connectTimeout(5, TimeUnit.SECONDS)
        .readTimeout(5, TimeUnit.SECONDS)
        .writeTimeout(5, TimeUnit.SECONDS)
        .eventListener(new EventListener(){
    
    })
        .build();
CacheControl cacheControl = new CacheControl.Builder()
        .noCache() // 即使有缓存，每次也都发起网络请求，收到304相应后，使用缓存。否则更新缓存。
        .maxAge(60, TimeUnit.SECONDS)
        // .onlyIfCached() //
        .build();
Request request = new Request.Builder()
        .url("https://github.com/square/okhttp")
        .cacheControl()
        .build();

client.newCall(request).enqueue(new Callback() {
    
    
    @Override
    public void onFailure(Call call, IOException e) {
    
    
        e.printStackTrace();
        Log.i(TAG, "onFailure " + call.request());
    }

    @Override
    public void onResponse(Call call, Response response) throws IOException {
    
    
        Log.i(TAG, "onResponse " + call.request() + ", Response Content" + response);
    }
});

flow chart

Interceptor Interceptor

• Official website documentation: https://square.github.io/okhttp/interceptors/
• Cache Interceptor: https://juejin.cn/post/6845166891476992008
• Whether it is an asynchronous request or a synchronous request, RealCall#getResponseWithInterceptorChainthis function will be used to traverse the interceptor and initiate a network request. Interceptors are traversed in the order they are added.
• The traversal execution order of interceptors: client.interceptors(), RetryAndFollowUpInterceptor, BridgeInterceptor, CacheInterceptor, ConnectInterceptor, client.networkInterceptors(), CallServerInterceptor.
• okhttp3.Interceptor.ChainThe only implementation class of the interface is RealInterceptorChain. RealInterceptorChain#proceed(okhttp3.Request)Traverse the interceptor's operations through the implementation of this class .

// okhttp3.RealCall#getResponseWithInterceptorChain
Response getResponseWithInterceptorChain() throws IOException {
    
    
  // Build a full stack of interceptors.
  List<Interceptor> interceptors = new ArrayList<>();
  interceptors.addAll(client.interceptors()); // 自定义应用拦截器
  interceptors.add(new RetryAndFollowUpInterceptor(client)); // 重试、重定向拦截器
  interceptors.add(new BridgeInterceptor(client.cookieJar())); // 桥接拦截器，主要是修改header参数、gzip压缩
  interceptors.add(new CacheInterceptor(client.internalCache())); // 缓存
  interceptors.add(new ConnectInterceptor(client)); // 待研究
  if (!forWebSocket) {
    
    
    interceptors.addAll(client.networkInterceptors()); // 待研究
  }
  interceptors.add(new CallServerInterceptor(forWebSocket)); // 待研究

  // okhttp3.Interceptor#intercept方法chain参数的实际类型
  Interceptor.Chain chain = new RealInterceptorChain(interceptors, transmitter, null, 0,
      originalRequest, this, client.connectTimeoutMillis(),
      client.readTimeoutMillis(), client.writeTimeoutMillis());

  boolean calledNoMoreExchanges = false;
  try {
    
    
    Response response = chain.proceed(originalRequest);
    if (transmitter.isCanceled()) {
    
    
      closeQuietly(response);
      throw new IOException("Canceled");
    }
    return response;
  } catch (IOException e) {
    
    
    calledNoMoreExchanges = true;
    throw transmitter.noMoreExchanges(e);
  } finally {
    
    
    if (!calledNoMoreExchanges) {
    
    
      transmitter.noMoreExchanges(null);
    }
  }
}

• Through indexthe accumulation, the pair interceptorstraversal is achieved. design pattern 责任链模式.

// okhttp3.internal.http.RealInterceptorChain#proceed(okhttp3.Request, okhttp3.internal.connection.Transmitter, okhttp3.internal.connection.Exchange)

  public Response proceed(Request request, Transmitter transmitter, @Nullable Exchange exchange)
      throws IOException {
    
    

    // 省略异常检查代码.....

    // Call the next interceptor in the chain.
    RealInterceptorChain next = new RealInterceptorChain(interceptors, transmitter, exchange,
        index + 1, request, call, connectTimeout, readTimeout, writeTimeout);
    Interceptor interceptor = interceptors.get(index);
    Response response = interceptor.intercept(next);

    // 省略对Response的异常检查代码.....

    return response;
  }

RetryAndFollowUpInterceptor

• The default is to retry. The default configuration parameters areOkHttpClient.Builder#retryOnConnectionFailure(boolean)
• Retry, redirect interceptor
• Redirect function RetryAndFollowUpInterceptor#followUpRequest. long redirect20次

BridgeInterceptor

• It is relatively simple, mainly because I want to add some field configurations to the Http Header.
• gzipThe most important thing is that compression configuration is automatically added . Simultaneously decompress the received data
• Configured Connection:Keep-Aliveto represent the need for a long connection\

// okhttp3.internal.http.BridgeInterceptor#intercept
if (userRequest.header("Connection") == null) {
    
    
  requestBuilder.header("Connection", "Keep-Alive");
}

// If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
// the transfer stream.
boolean transparentGzip = false;
if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) {
    
    
  transparentGzip = true;
  requestBuilder.header("Accept-Encoding", "gzip");
}

CacheInterceptor

• Reference CacheInterceptor缓存详解chapter.

ConnectInterceptor

• Reference 连接拦截器篇: https://blog.csdn.net/followYouself/article/details/121086869

networkInterceptors

• Custom network parsing interceptor.
• After establishing a connection with the server, intercept the network request before actually initiating it.

CallServerInterceptor

• Actual use Okiois to interact with the server, make requestrequests, and receive response.
• Http2Related request classes Http2Stream.java, Http2Reader, Http2Writer.

CacheInterceptorDetailed explanation of cache

• By default, no caching is enabled. To enable caching, you need OkHttpClient.Builder#cache(@Nullable Cache cache)to configure it through a method and specify the path to the cached File file.
• okhttp will only cache get requests.
• URL serves as cached key information.
• CacheControlSome key configurations no-cache, no-store, max-age, only-if-cached, max-stale. Refer to Http headerthe sections
• If the server does not support caching configuration, caching can also be implemented on the source side. You can pass addNetworkInterceptor, Responseadd in cache-control, configure max-age(high priority), Last-Modify(low priority) and other parameters. Refer to the source code CacheStrategy.Factory#computeFreshnessLifetime.
• Official website: https://square.github.io/okhttp/caching/
• Reference: https://juejin.cn/post/6850418120729985038 . There is a problem with this wiki, confusing the noCacheandnoStore

Cache.java

• get, put, remove, updaterespectively correspond to the read, write, delete, and update operations of the cache.
• DiskLruCacheIt encapsulates the cache read and write capabilities and takes advantage of Okiothe read and write capabilities. Reference: https://blog.csdn.net/zwlove5280/article/details/79916662
• The cached key value is the request URL. Reference okhttp3.Cache#keymethod.
• Response Headerinformation is stored in .0the file, body information is stored in .1the file, and all operation logs are recorded in journalthe file.
• The specifics and DiskLruCachedetails of how to read and write have not been studied.

CacheStrategy.java cache strategy

• Determine the subsequent network request steps based on the request request and the Response in the cache. The key method is CacheStrategy.Factory#get(),CacheStrategy.Factory#getCandidate()
• This class has two member variables networkRequestthat are not empty, indicating that a network request needs to be initiated. null means no network request is required. cacheResponseIf not null, the cache needs to be validated or used directly as a result. Is null, indicating not to use cache.
• What needs special attention is that if networkRequestit is not empty and the request is also configured , an error only-if-cachedwill be reported .504Unsatisfiable Request (only-if-cached)
• CacheStrategy#isCacheable, first determine Responsewhether the status code supports caching, and then check Responsethe Requestheader of the sum. If configured no-store, caching is not supported.
• A function of the caching strategy CacheStrategy.Factory#getCandidate. Comments 1, 2, and 3 in the code correspond to the following respectively

1. If it is not found in the cache cacheResponse, a network request is required.
2. If it is an https request, but cacheResponseit is not included in the request handshake, a network request is required.
3. isCacheableDetermine cacheResponsewhether the status code in the request supports caching, and also determine whether the request request is configured noStore. If configured noStore, caching is disabled.
4. If the request is configured noCacheor configured If-Modified-Sinceor configured If-None-Match, then the network request is initiated directly. According to CacheInterceptor.javathe code, after the network request is completed, if cacheResponseit is not empty and 304the status code is received, then use it cacheResponseas the request result.
5. Based on request, maxage, min-fresh, maxStaleand based on cacheResponse, sentRequestAtMillis, receivedResponseAtMillis, servedDateand must-revalidateother configurations, calculate cacheResponsewhether it has expired and whether it meets the requirements of this request. If satisfied, then the cache is used directly as the result of the network request without initiating an actual network request. The specific calculation details have not been studied...
6. Here, time calculation is used to determine whether to use cache. ageMillis: cacheResponseThe time taken from generation to now; minFreshMillis:Minimum freshness, the shortest time until final expiration, requestconfigured in; freshMillis: The smaller value of configured cacheResponseand ; after exceeded , the time that is still acceptable, configured in.requestmax-agemaxStaleMillismax-agerequest
7. If none of the above conditions are met and the cache has expired, it will be judged in turn whether fields such as , , , etc. are cacheResponsecarried , converted into and fields, and added to the request header.ETagLast-ModifiedDateIf-None-MatchIf-Modified-Since
8. If networkRequestit is not empty and request is configured only-if-cached, an error CacheInterceptor.javawill be reported according to the code .504Unsatisfiable Request (only-if-cached)
   

// okhttp3.internal.cache.CacheStrategy.Factory
/**
 * Returns a strategy to satisfy {@code request} using the a cached response {@code response}.
 */
public CacheStrategy get() {
    
    
  CacheStrategy candidate = getCandidate(); // 根据request请求和缓存中的Response决定后续的网络请求步骤
  // 禁止同时有networkRequest 和 onlyIfCached配置。
  if (candidate.networkRequest != null && request.cacheControl().onlyIfCached()) {
    
     // 注释8
    // We're forbidden from using the network and the cache is insufficient.
    return new CacheStrategy(null, null); // 此处为空，会报`504`错误。
  }
  return candidate;
}

/** Returns a strategy to use assuming the request can use the network. */
private CacheStrategy getCandidate() {
    
    
  // No cached response.
  if (cacheResponse == null) {
    
     // 注释1
    return new CacheStrategy(request, null);
  }

  // Drop the cached response if it's missing a required handshake.
  if (request.isHttps() && cacheResponse.handshake() == null) {
    
      // 注释2
    return new CacheStrategy(request, null);
  }

  // If this response shouldn't have been stored, it should never be used
  // as a response source. This check should be redundant as long as the
  // persistence store is well-behaved and the rules are constant.
  if (!isCacheable(cacheResponse, request)) {
    
      // 注释3
    return new CacheStrategy(request, null);
  }

  CacheControl requestCaching = request.cacheControl(); // 获取请求中的cacheControl配置
  if (requestCaching.noCache() || hasConditions(request)) {
    
     // 注释4
    return new CacheStrategy(request, null);
  }

  CacheControl responseCaching = cacheResponse.cacheControl();  // 注释5
  long ageMillis = cacheResponseAge(); // 计算缓存从生成开始，已经度过了多长时间
  long freshMillis = computeFreshnessLifetime(); // 返回Response配置的max-age,最大可存活的生命时间

  if (requestCaching.maxAgeSeconds() != -1) {
    
    
    // cacheResponse中配置的max-age 和 request中配置的max-age中，取最小的一个值。
    // 此处就最终计算出来允许Response有效时间
    freshMillis = Math.min(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds()));
  }

  long minFreshMillis = 0;
  if (requestCaching.minFreshSeconds() != -1) {
    
    
    minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds());
  }

  long maxStaleMillis = 0;
  // 计算可以接受的超期时间
  if (!responseCaching.mustRevalidate() && requestCaching.maxStaleSeconds() != -1) {
    
    
    maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds());
  }

  // 如果缓存没有过期，满足当前request的要求，那么直接使用缓存作为结果。
  if (!responseCaching.noCache() && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) {
    
     // 注释6
    Response.Builder builder = cacheResponse.newBuilder();
    if (ageMillis + minFreshMillis >= freshMillis) {
    
    
      builder.addHeader("Warning", "110 HttpURLConnection \"Response is stale\"");
    }
    long oneDayMillis = 24 * 60 * 60 * 1000L;
    if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) {
    
    
      builder.addHeader("Warning", "113 HttpURLConnection \"Heuristic expiration\"");
    }
    return new CacheStrategy(null, builder.build());
  }

  // Find a condition to add to the request. If the condition is satisfied, the response body
  // will not be transmitted.
  String conditionName;
  String conditionValue;
  if (etag != null) {
    
       // 注释7
    conditionName = "If-None-Match";
    conditionValue = etag;
  } else if (lastModified != null) {
    
    
    conditionName = "If-Modified-Since";
    conditionValue = lastModifiedString;
  } else if (servedDate != null) {
    
    
    conditionName = "If-Modified-Since";
    conditionValue = servedDateString;
  } else {
    
    
    return new CacheStrategy(request, null); // No condition! Make a regular request.
  }

  Headers.Builder conditionalRequestHeaders = request.headers().newBuilder();
  Internal.instance.addLenient(conditionalRequestHeaders, conditionName, conditionValue); // 添加到request

  Request conditionalRequest = request.newBuilder()
      .headers(conditionalRequestHeaders.build())
      .build();
  return new CacheStrategy(conditionalRequest, cacheResponse);
}

okhttp3.Dispatcher asynchronous request dispatch

• maxRequestsMaximum number of concurrent requests, default 64. maxRequestsPerHostThe maximum number of concurrent requests supported by each host, default 5. Configurable.
• Contains a thread pool executorServicefor asynchronous network request scheduling. The default configuration is that the core thread is 0, the maximum number of threads is not limited, and the survival time is 60 seconds.

// okhttp3.Dispatcher#executorService
executorService = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60, TimeUnit.SECONDS,
    new SynchronousQueue<>(), Util.threadFactory("OkHttp Dispatcher", false));

Http Header configuration knowledge

How to view http request headers

• In the browser, F12
• F5 refresh web page
• NetWork tab, Doc tab, select a request in Name
• Select the Headers tab. (Juxtaposed with the Header tab are Preview, Response, and Cookies)

Common Configuration Items

• Accept-Encoding:gzipRepresents compressed encoded data
• Connection:Keep-AliveRepresents the need for a long connection

Cache-Control

• no-cache: The resource can be cached by the client, but not by the proxy server. But each request needs to go to the server to verify whether the resource is valid. Related configuration referenceLast-Modify / If-Modify-Since
• no-store:No caching of any kind. Both the client and server can be configured to take effect.
• max-age:The time the resource can be cached, in seconds. max-age will be overwritten expires. After expiration, access the server to verify validity.
• s-maxage: Set the maximum effective time of the proxy server cache, in seconds. s-maxagewill be overwritten max-age.
• public: Indicates that the current request is a kind of general business data, which can be cached by the client, proxy server, and intermediate node server.
• private:Defaults. Indicates that the current operation is a special behavior that is strongly related to the specific user and should not be cached in proxy servers, intermediate node servers, etc. Because caching doesn't make much sense.
• only-if-cached: No network requests are made, and only the cache is used. If the cache misses, a 504 error is returned. And the priority here is higher no-cache, and the network request will definitely not be initiated.
• must-revalidate: Once the resource expires, a request must be made to the server to confirm the validity of the resource. If the server cannot be accessed, 504 Gateway Timeout is reported. The priority is higher and becomes invalid max-staleafter setting .max-stale
• max-stale:The client requires the cache proxy to return the resources within a certain period of time (no time limit by default) to the client regardless of whether the cache has expired. This parameter indicates the expiration time of the response that the business can accept.
• no-transform: The caching proxy cannot change the media type, which prevents the operation of compressing images and compressing resources.
• min-fresh: The minimum time remaining before cache Responseexpiration ( sum of , ) to ensure that the cache will not expire within a short period of time ( ) max-age. For example , , . Then the + sum is 600. Due to the configuration , it is required that when using the cache, the final expiration time point of 600 must be guaranteed to retain the freshness of 200. , then it means that the cache can only be used after a maximum of 400 seconds from the beginning of the cache to the present .max-stalemin-freshmax-age=100max-stale=500min-fresh=200Responsemax-agemax-stalemin-fresh600 - 200 = 400ResponsenowTime
• Reference materials https://segmentfault.com/a/1190000022336086

negotiate cache Last-Modify/If-Modify-Since,If-None-Match/ETag

• etagThe verification priority is higher than Last-Modify.
• Last-Modify: Carried in the server's Response, indicating the time when the business data was last modified.
• If-Modify-Since: When the cache expires, it is carried when the client makes a request. The server checks the request time and verifies whether the server resources have been modified. If modified, the latest data is returned and returned HTTP 200 OK. If the resource has not been modified, then only the status code is returnedHTTP 304
• etag:The resource tag carried by the server Response. Any modification of the resource will cause the tag to change. If the tag remains unchanged, it means that the resource data has not changed.
• If-None-Match: When the cache expires, the client request carries the tag value of the Response in the cache. It also determines whether the resource data has been modified based on the tag and responds to the client request.

Range and Content-Range

• The range parameter allows you to specify the range of files to be obtained from the server.
• Design purpose: used for resumed downloads. For example, when downloading a large file, the network is suddenly interrupted. When the network is restored, the content can continue to be downloaded.

User-Agent

• Program software that represents user behavior. For example, a web browser is a user agent that "helps users obtain, render and interact with web content"; an email reader can also be called a mail agent.
• Example

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36

SSL encryption method configuration

• Configuration function OkHttpClient#sslSocketFactory, default configuration function OkHttpClient#newSslSocketFactory.
• It is divided into Android platform and java platform. The configuration function of Android platform is AndroidPlatform#getSSLContext, the source code is as follows

@Override
public SSLContext getSSLContext() {
    
    
  boolean tryTls12;
  try {
    
    
    tryTls12 = (Build.VERSION.SDK_INT >= 16 && Build.VERSION.SDK_INT < 22);
  } catch (NoClassDefFoundError e) {
    
    
    // Not a real Android runtime; probably RoboVM or MoE
    // Try to load TLS 1.2 explicitly.
    tryTls12 = true;
  }

  if (tryTls12) {
    
    
    try {
    
    
      return SSLContext.getInstance("TLSv1.2");
    } catch (NoSuchAlgorithmException e) {
    
    
      // fallback to TLS
    }
  }

  try {
    
    
    return SSLContext.getInstance("TLS");
  } catch (NoSuchAlgorithmException e) {
    
    
    throw new IllegalStateException("No TLS provider", e);
  }
}

• CLEARTEXT is the http:// URL used for insecure configurations.

RealCall class

http2.0

• The difference between http2.0 and http1.0: http2.0 supports multiplexing and supports concurrent transmission of multiple requests. http1.0 is executed sequentially. After sending a request, wait for the response before initiating the next request.
• The important concepts in http2.0 are 帧(frame)and 流(stream). Each stream has a unique number.
• The compression algorithm of HTTP2.0 message header adopts HPACK.
• http2.0:https://segmentfault.com/a/1190000016975064
• http2.0:https://juejin.cn/post/6844903935648497678

java method

• java.lang.Thread#holdsLock determines whether the current thread holds a lock

Not studied

• The differences between each version of httpthe protocol and the details of the protocol.
• OkioThe open source library is a basic reading and writing library and does not read the source code. OkioI don’t understand the implementation principle and segmentmechanism.
• DiskLruCache.javaFile caching solution, no source code read
• Http2Connection.javahttp2 protocol connection, data sending and receiving, streamusage related information