OkHttp source depth analysis

This article from OPPO Internet-based technical team, reprint please note the name of. At the same time the public are welcome to our number: OPPO_tech, OPPO share cutting-edge Internet technologies and activities with you.

OkHttp should use the Android platform is the most widely used open source network library up, Android will also be implemented to replace the default internal HttpUrlConnection after 6.0 became OkHttp.

Most developers of students may have been exposed OkHttp source, but few have a more comprehensive reading and understanding of the current parts of the source analytic article on the network are also go beyond that, and a large section of source code posted this I kind of analytical methods is not recognized, so I have the idea you want to re-write an analytical OkHttp source.

The purpose of this article is to compare a comprehensive introduction OkHttp source, and the other is to try to avoid posting a large segment of the source code, source code related to the part, we will try to show through the call graph.

Benpian chosen OkHttp source code is the latest version 4.4.0, the intended audience is a certain use-based Android development classmates.

OkHttp source code can be downloaded from github to ( https://github.com/square/OkHttp ).

The point, the article will look at several ways to start dismantling OkHttp source:

the whole frame
Interceptor
Task Queue
Multiplexing and connection pools

1. From an example

First look at a simple Http request is sent what.

OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url("https://www.google.com").build();
Response response = client.newCall(request).execute();
return response.body().string();

This piece of code is the most common OkHttp daily usage, follow up the source code, a more detailed flow diagram can get through this picture look at the internal logic of how the flow.

Involves a number of core classes, we look at the next one.

OkHttpClient
Request 和 Response
RealCall

OkHttpClient : This is the core of the entire OkHttp management class, all the internal logic and objects owned by OkHttpClient to unified management, which are generated by Builder constructor, a lot of configuration parameters and class members, here to do a specific analysis.

Request and the Response : Request transmission request is our wrapper class, the common internal parameters url, header, method, body, etc., Response is the result of the request, comprising code, message, header, body; definitions of these two classes is entirely consistent with Http protocol defined in response to content request and content.

RealCall : responsible for scheduling requests (synchronous, then take the current thread sends a request, asynchronous, then use the internal OkHttp thread pool); at the same time responsible for building the internal logic of the chain of responsibility, and responsibility for implementation chain of logic, until obtaining results. Although OkHttpClient is the core of the entire OkHttp management class, but it really makes a request and organizational logic is RealCall class, it also shoulders the important task of scheduling the two organizations and the chain of responsibility, then we have to focus on the next logical analysis RealCall class.

RealCal class source address: HTTPS: //github.com/square/OkH ...

RealCall class is not complicated, there are two of the most important methods, Execute () and the enqueue (), a synchronization request is processed, a request is asynchronous processing. After follow-up enqueue find the source code, it just did a package through asynchronous call asynchronous thread and callback, or will eventually call logic to execute () This method, then call the getResponseWithInterceptorChain () request to obtain the results.

Appears to be getResponseWithInterceptorChain () method carries the core logic of the entire request, we just need to clear this analysis method, really a OkHttp the request process will generally figured it out. Since such an important method, still can not escape the posted under the complete source code.

val interceptors = mutableListOf<Interceptor>()
    interceptors += client.interceptors
    interceptors += RetryAndFollowUpInterceptor(client)
    interceptors += BridgeInterceptor(client.cookieJar)
    interceptors += CacheInterceptor(client.cache)
    interceptors += ConnectInterceptor
    if (!forWebSocket) {
      interceptors += client.networkInterceptors
    }
    interceptors += CallServerInterceptor(forWebSocket)

    val chain = RealInterceptorChain(interceptors, transmitter, null, 0, originalRequest, this,
        client.connectTimeoutMillis, client.readTimeoutMillis, client.writeTimeoutMillis)

    var calledNoMoreExchanges = false
    try {
      val response = chain.proceed(originalRequest)
     ............

You can see from the source code, even if the logic getResponseWithInterceptorChain () method is actually very simple, it generates a Interceptors Interceptor List list, according to the order will be:

client.Interceptors
RetryAndFollowUpInterceptor,
BridgeInterceptor
CacheInterceptor
ConnectInterceptor
client.networkInterceptors
CallServerInterceptor

These members are added to the List, and then created a class called RealInterceptorChain last Response is acquired by chain.proceed.

Further analysis RealInterceptorChain and of Interceptors, we get a conclusion, OkHttp complex logic entire request cut into a one independent module named interceptor (Interceptor), connected in series through the design pattern chain of responsibility together, the final completion request get response results.

Specifically how these interceptors are connected in series, each interceptor has what features the following content will make a more detailed analysis.

2. OkHttp core: the interceptors

We already know the core logic OkHttp is a bunch of interceptors, then they are linked together and how to construct it? Here we must analyze this class of RealInterceptorChain.

After passing through the previous analysis found, RealCall a Interceptors will add to a List, configured to generate a RealInterceptorChain object and call chain.proceed obtain a response result. Then you have to analyze chain.proceed this approach in the end did what. To keep the length is too long, there is not posted source content, is given only after the conclusion of the analysis, we control the source code can be read quickly.

RealInterceptorChain source: HTTPS: //github.com/square/OkH ...

The source of RealInterceptorChain analysis, gives the following schematic (not part of the interceptor):

Source and binding the schematic, the following conclusions can be obtained:

Interceptors are executed in the order of addition
Blocker performed starting from RealInterceptorChain.proceed (), the logic proceeds to execute a first interceptor
Each interceptor prior to execution, the interceptor will make up the remainder not yet implemented the new RealInterceptorChain
Interceptor logic is a new chain of responsibility calls next.proceed () cut into start, next.proceed, end three parts in order to perform
next.proceed () is actually represented by the remaining logic to perform all the interceptor
All interceptors eventually form a nested structure embedded layer

Understand the construction process above interceptors, let us analyze one of the function and role of each interceptor.

From this perspective partial view, add a total of five interceptor (not included as a custom interceptor client.interceptors and client.networkInterceptors, two explain later).

First come the next general understanding of the role of each interceptor

retryAndFollowUpInterceptor-- failures and redirect interceptor
BridgeInterceptor-- request and response package interceptors
CacheInterceptor-- cache associated filter, return directly responsible for reading the cache, the cache is updated
ConnectInterceptor-- connection services, is responsible for establishing the actual request and the server network connection is here
CallServerInterceptor-- execution flow operation (write request body, response data is obtained) is responsible for sending request data to the server, parsing the data read response packet encapsulation http request packets and requests from the server

To be analyzed on one interceptor below.

2.1 RetryAndFollowUpInterceptor

Source Address: HTTPS: //github.com/square/OkH ...

The source logic to directly draw the corresponding flow chart (this internal logic () method of RetryAndFollowUpInterceptor intercept):

As can be seen from the figure above, RetryAndFollowUpInterceptor opened a while (true) loop, and completed two important in determining the inner loop, as shown in the blue box:

When an exception is thrown internal request, determines whether a retry is needed
When the result is a 3xx redirection response, constructing a new request and the transmission request

Retry logic is relatively complex, the following decision logic (specifically recover code RetryAndFollowUpInterceptor class method):

Rule 1: client retryOnConnectionFailure parameter is set to false, no retries
Rule 2: requesting body has been issued, no retries
Rule 3: special exception type no retries (e.g. ProtocolException, SSLHandshakeException etc.)
Rule 4: no more route (comprising a proxy and inetaddress), no retries

Under the conditions previously these four rules do not comply, it will retry the current request. Redirection logic is relatively simple, there is not deeply.

The difference between 2.2 Interceptors and NetworkInterceptors

As mentioned earlier, there are two arguments in the constructor OkHttpClient.Builder, the user can add custom interceptors by addInterceptor and addNetworkdInterceptor, finished RetryAndFollowUpInterceptor analysis we can know the difference between these two is the automatic interceptor.

The order of addition of the interceptor can know from the front and networkInterceptors Interceptors RetryAndFollowUpInterceptor immediately in front of a one behind.

Binding chain of responsibility in front of the call graph can be analyzed, if a retry request interceptor in the inside or redirect RetryAndFollowUpInterceptor N times, then all nested inside the interceptor is also called N times, the same custom networkInterceptors interceptor will be called N times. And is a relatively Interceptors call request only once, so within OkHttp also be referred to as Application Interceptor.

2.3 BridgeInterceptor 和 CacheInterceptor

BridageInterceptor interceptor functions are as follows:

The user is responsible for the configuration of the conversion request requesting to transmit to the server, the server returns the response into user friendly response, the application code from the code to the network bridge
Setting content length, content coding
Gzip compression set, and decompressed after receiving the content. Eliminating the application layer data decompression trouble
Add cookie
Other headers provided, such as User-Agent, Host, Keep-alive like. Keep-Alive wherein the step of connecting is necessary to achieve multiplexing

CacheInterceptor logic flow interceptor follows:

By Request try to get the Cache cache, of course, the premise is OkHttpClient configure the cache, the default is not supported.
According to response, time, request to create a caching strategy for how to judge the use of cache.
If the cache policy setting prohibited network, and the cache is empty and then directly back to build a Response note return code = 504
Cache policy settings do not use the network, but it is a cache simply returns the cached
Then take the subsequent filter process, chain.proceed (networkRequest)
When present in the cache, if the network 304 is returned Resposne, Resposne cache is used.
Construction of network requests Resposne
When configured in OkHttpClient in the cache, then this Resposne cached.
The first step is cached cache header, then the cache body.
Return Resposne

The next two should be all internal interceptors in the two most important, and a Socket connection handles Dns and the other is responsible for body sends Http request.

2.4 ConnectInterceptor

As already mentioned, connectInterceptor should be one of the most important interceptor, and it is also responsible for the Dns parsing and Socket connections (including tls connection).

Source Address: HTTPS: //github.com/square/OkH ...

The class itself is very simple, from the source point of view, only one key code.

val exchange = transmitter.newExchange(chain, doExtensiveHealthChecks)

Get a new ExChange from Transmitter objects, simple code phrase after careful with the inside, you will find in fact, buried a lot of logic, a process involving the entire network connection is established, including the process dns process and socket connections, here we have to understand the whole process under network connected by two graphs.

First look at the timing diagram of method calls, to tease out the key steps:

ConnectInterceptor调用transmitter.newExchange
Transmitter call ExchangeFinder the first find () to get ExchangeCodec
ExchangeFinder calls itself findHealthConnectio get RealConnection
ExchangeFinder () method to obtain ExchangeCodec by RealConnection the codec just acquired
Transmitter get to the ExchangeCodec, then a new ExChange, will be included just ExchangeCodec

Just 5 steps, eventually obtained by Connectinterceptor Exchange Transmitter to a class, this class has two implementations, one Http1ExchangeCodec, one Http2Exchangecodec, respectively corresponding to the protocol and is Http1 Http2 protocol.

So get to the Exchange class what use is it? Look at these classes diagram, as follows:

Can be seen from the above, get in front of the class which contains ExchangeCodec Exchange object, and the object which also contains a RealConnection objects, properties are members RealConnection socket, handlShake, protocol and so on, we can see that it should be a wrapper class Socket connections , the object is a package of RealConnection ExchangeCode operation (writeRequestHeader, readResposneHeader) a.

By these two figures can clearly know that, ultimately, it is a Socket object connection has been established, that is, within ConnectInterceptor has been completed socket connection, specifically what step to complete it?

See timing diagram above, you can know, get findHealthConnection RealConnection of ExchangeFinder call () method, therefore, socket connections are established and get done here.

Also, before socket connection, there is also a dns process, but also the internal logic implicit in findHealthConnection in detail the process of further analysis in the process of DNS later, here ConnectionInterceptor the task has been completed.

Also to note is that after performing ConnectInterceptor, in fact, add a custom network interceptors networkInterceptors, in accordance with the provisions of the order, all networkInterceptor execution, socket connection is already established, you can get socket by realChain do something, and this is why network Interceptor called.

2.5 CallServerInterceptor

CalllServerInterceptor is the last interceptor, the interceptor has been completed in front of the socket connection and tls connection, then this step is head and body http transmission of the data.

CallServerInterceptor Source: HTTPS: //github.com/square/OkH ...

CallServerInterceptor comprising the steps of:

Send request header to the server
If there is request body, is sent to the server
Reads the response header, to construct a Response object
If there is response body, to add body to construct a new Response object on the basis of 3

Here we can see, the core of the work completed by HttpCodec objects, and HttpCodec actually utilized is Okio, while Okio actually still use Socket, only one set level, a bit more than the number of layers.

3. The overall architecture

Thus far, all of the interceptors are finished, we already know a complete request process is how it happened. Well, this time look OkHttp architecture diagram relatively clear

OkHttp entire architecture is five internal longitudinal view interceptor, is cut crosswise, divided into several parts, and the longitudinal interceptors is to complete the call request procedure by lateral stratification, these two aspects grasp and understand OkHttp is more comprehensive.

The detailed analysis in the following sections for the lateral portion.

3.1 connection multiplexing, DNS, and Socket connection

Known through the above analysis, and Dns Socket connection process is in ConnecInterceptor by Transmitter and ExchangeFinder to complete, and can be seen in the timing chart of the foregoing, the final connection is established by a method Socket findConnection ExchangeFinder to complete, It can be said that all secrets are findConnection method.

Thus the following detailed parsing the findConnection (), where the source code and comments posted.

synchronized(connectionPool) {
      //前面有一大段判定当前的conencection是否需要释放，先删除
      .....

      if (result == null) {
        // 1, 第一次尝试从缓冲池里面获取RealConnection(Socket的包装类)
        if (connectionPool.transmitterAcquirePooledConnection(address, transmitter, null, false)) {
          foundPooledConnection = true
          result = transmitter.connection
        } else if (nextRouteToTry != null) {
          //2, 如果缓冲池中没有，则看看有没有下一个Route可以尝试，这里只有重试的情况会走进来
          selectedRoute = nextRouteToTry
          nextRouteToTry = null
        } else if (retryCurrentRoute()) {
          //3,如果已经设置了使用当前Route重试，那么会继续使用当前的Route
          selectedRoute = transmitter.connection!!.route()
        }
      }
    }
    if (result != null) {
      // 4，如果前面发现ConnectionPool或者transmiter中有可以复用的Connection，这里就直接返回了
      return result!!
    }

    // 5, 如果前面没有获取到Connection，这里就需要通过routeSelector来获取到新的Route来进行Connection的建立
    var newRouteSelection = false
    if (selectedRoute == null && (routeSelection == null || !routeSelection!!.hasNext())) {
      newRouteSelection = true
      //6，获取route的过程其实就是DNS获取到域名IP的过程，这是一个阻塞的过程，会等待DNS结果返回
      routeSelection = routeSelector.next()
    }

    var routes: List<Route>? = null
    synchronized(connectionPool) {
      if (newRouteSelection) {
        // Now that we have a set of IP addresses, make another attempt at getting a connection from
        // the pool. This could match due to connection coalescing.
        routes = routeSelection!!.routes
        //7，前面如果通过routeSelector拿到新的Route，其实就是相当于拿到一批新的IP，这里会再次尝试从ConnectionPool
        // 中检查是否有可以复用的Connection
        if (connectionPool.transmitterAcquirePooledConnection( address, transmitter, routes, false)) {
          foundPooledConnection = true
          result = transmitter.connection
        }
      }
 if (!foundPooledConnection) {
        if (selectedRoute == null) {
           //8，前面我们拿到的是一批IP，这里通过routeSelection获取到其中一个IP，Route是proxy和InetAddress的包装类
          selectedRoute = routeSelection!!.next()
        }

        // Create a connection and assign it to this allocation immediately. This makes it possible
        // for an asynchronous cancel() to interrupt the handshake we're about to do.
        //9，用新的route创建RealConnection，注意这里还没有尝试连接
        result = RealConnection(connectionPool, selectedRoute!!)
        connectingConnection = result
      }
  }

    // If we found a pooled connection on the 2nd time around, we're done.
    // 10，注释说得很清楚，如果第二次从connectionPool获取到Connection可以直接返回了
    if (foundPooledConnection) {
      eventListener.connectionAcquired(call, result!!)
      return result!!
    }

    // Do TCP + TLS handshakes. This is a blocking operation.
    //11，原文注释的很清楚，这里是进行TCP + TLS连接的地方
    result!!.connect(
        connectTimeout,   
 readTimeout,
        writeTimeout,
        pingIntervalMillis,
        connectionRetryEnabled,
        call,
        eventListener
    )
 //后面一段是将连接成功的RealConnection放到ConnectionPool里面，这里就不贴出来了
    }
    return result!!

You can see from the above process, findConnection this method to do the following things:

Check whether the current exchangeFinder saved Connection to satisfy the request
Check the current connection pool ConnectionPool whether the request meets Connection
Check the current RouteSelector list, if there is available Route (Route is a proxy, IP address packaging), if not initiate DNS requests
After obtaining a new Route through DNS, look for the second time from ConnectionPool Connection has no multiplexing, or to create a new RealConnection
A TCP and TLS with RealConnection connection, the connection is successfully saved to ConnectionPool

Connection multiplexed connection

Will write to ConnectionPool it will be seen after the second step and the fourth step of multiplexing ConnectionPool made two checks, the fifth step to create a new RealConnection.

Thus, this is connected to the multiplexing OkHttp actually achieved by ConnectionPool foregoing is also reflected in the class diagram, there ArrayDeque ConnectionPool internal connections of the object is used to save a cached connection pool.

DNS process

Seen from the front steps of the analytical process Dns hidden in a third step RouteSelector inspection, the entire process of writing more casual in findConnection methods, may not be particularly easy to understand, but if you understood the RouteSelector, RouteSelection, Route three relationship classes, in fact, it is easier to understand, the following figure shows the relationship between the three classes.

It is concluded from the figures:

RouteSelector next call proxy traversal in different circumstances to obtain the next Selection package type, Route Selection holding a list, that is, each corresponds with a proxy Route List
Selection for actually List <Route> package an iterator () method to get through the next Route Next, Route holding proxy, address and InetAddress, it will be understood as a package and for the IP Proxy Route is paired
Internal RouteSelector the next () method calls nextProxy (), nextProxy () will call resetNextInetSocketAddres () method
resetNextInetSocketAddres get InetSocketAddress by address.dns.lookup, that is, IP address

Through the above series of processes know, IP addresses, ultimately acquired by dns address, and this dns is how to build it?

Traceback code is targeted to address the transmitter when building dns address will pass client.dns built in, and is passed in client.dns Dns.System OkHttpclient during construction, the inside by lookup InetAddress .getAllByName method to obtain the corresponding domain name IP, which is the default Dns achieved.

At this point, the entire DNS process truth is out. OkHttp when this one designed to emphasize the connection coupling and openness, the whole process DNS hidden deeper, if not carefully debug with code words, may not be very easy to find.

3.2 Socket connection established

After obtaining Connectoin by Dns, it is the process of establishing a connection in a reflected only findConnection code as follows:

result!!.connect(
        connectTimeout,
        readTimeout,
        writeTimeout,
        pingIntervalMillis,
        connectionRetryEnabled,
        call,
        eventListener
)

Here is the result of type Object RealConnection, RealConnection.connect method is called, finally left findConnection, the next look at the source code connect method.

//省略前面一大段 
....

    while (true) {
      try {
        if (route.requiresTunnel()) {
 //这里进入的条件是，通过http代理了https请求，有一个特殊的协议交换过程
          connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener)
        } else {
 //建立socket连接
          connectSocket(connectTimeout, readTimeout, call, eventListener)
        }
 //如果前面判定是https请求，这里就是https的tls建立过程
        establishProtocol(connectionSpecSelector, pingIntervalMillis, call, eventListener)
        break
      } catch (e: IOException) {
 //清理资源
        socket?.closeQuietly()
 //对异常做二次封装，然后抛出
        if (routeException == null) {
          routeException = RouteException(e)
        } else {
          routeException.addConnectException(e)
        }
        if (!connectionRetryEnabled || !connectionSpecSelector.connectionFailed(e)) {
          throw routeException
        }
      }
    }

connect method is not complicated, will first determine whether there has been a proxy to do some special treatment, and then call a systematic approach to establish socket connections.

If the request is https, tls there is a connection to be established, the middle if there is an exception is thrown, it will do a second package throw it out.

4. Summary

So far, basically OkHttp source design of a complete picture have, because there are some elements often encountered in everyday use, such as OkHttpClient of Builde parameters, such as Request and Response usage, there is not more talk. In addition to support for the https http2 and, because it involves a more complex mechanism and principle, have the opportunity to open another article speaking.

http://weixin.qq.com/r/3By8pK7EefMGrerH90nO (Automatic Identification dimensional code)