拦截器上篇我们看了三个拦截器,分别是RetryAndFollowUpInterceptor,BridgeInterceptor和CacheInterceptor
这篇我们看后面的两个拦截器,也是最重要的两个,在看源码之前,先做一下铺垫:
为什么要做app网络优化
keepalive
在http请求中,对于请求速度提升和降低延迟,keepalive在网络连接中发挥着重大作用。
我们知道,做http请求都要经过tcp三次握手和4次挥手,所以我们一般的http连接都是先tcp握手,然后传输数据,最后释放资源。
如果我们在每次传输资源的时候都先握手,和挥手,重复的连接和释放,而且每一个连接大概是TTL的一次时间,特别是在TLS环境下消耗的时间更多(https),所以在http中有一个keepalive connections的机制,它可以在传输数据后仍然保持连接,当客户端需要再次获取数据的时候,知道使用刚刚空闲下来的连接而不需要再次握手。
在PC的浏览器里,一般会同时开启6-8个keepalive connections的socket链接,并保持一定的链路生命,当不需要时再关闭。而在服务器中,一般由软件根据负载的情况,觉得是否关闭。
当然keepalive也有缺点,它在减轻客户端的延迟的同时,也方案了其他客户端的链路速度,如果存在大量的空闲链接,其他客户端正常的链接的速度就会受到影响。
ConnectionInterceptor
连接拦截器,这个是真正开始向服务器发起连接。
/** Opens a connection to the target server and proceeds to the next interceptor. */
public final class ConnectInterceptor implements Interceptor {
public final OkHttpClient client;
public ConnectInterceptor(OkHttpClient client) {
this.client = client;
}
@Override public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
Request request = realChain.request();
StreamAllocation streamAllocation = realChain.streamAllocation();
// We need the network to satisfy this request. Possibly for validating a conditional GET.
boolean doExtensiveHealthChecks = !request.method().equals("GET");
HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
RealConnection connection = streamAllocation.connection();
return realChain.proceed(request, streamAllocation, httpCodec, connection);
}
}
在它的Interceptor()方法中,首先通过streamAllcation的newStream
方法获取一个流,HttpCodec,然后就是获取对应的RealConnection
。
这个StreamAllocation是在RetryAndUpFollowInterceptor中得到的。
在看这个类的时候,先知道一些背景:
HTTP版本:
HTTP的版本从最初的1.0版本,到后续的1.1版本,再到后续Google推出的SPDY,后面再推出了2.0版本,HTTP协议越来越完善。2.0主要解决了1.1和1.0最重要的两个问题:连接无法复用和head of line blocking(HOL)问题。2.0使用多路复用的技术,多个stream可以共用一个socket连接,每个tcp连接都是通过一个socket来完成的,socket对应一个host和port,如果有多个stream(也就是多个request)都是连接在一个host和port上,那么它们就可以共同使用同一个socket,这样做的好处就是可以减少TCP的一个三次握手的问题。在OKHttp中,记录一次连接的是RealConnection,这个负责连接,在这个类中用socket连接,用HandShadke来处理握手。
我们先知道3个概念:请求,连接,流。要知道HTTP通信执行网络“请求”需要在“连接”上连接一个新的“流”,我们将StreamAllocation称为流的桥梁。
This class coordinates the relationship between three entities:
该类会协调下面这三个实体之间的关系
Connections: physical socket connections to remote servers. These are
potentially slow to establish so it is necessary to be able to cancel
a connection currently being connected.
socket连接到远程服务器。这个连接可能很慢,因此有必要能够取消一个正在连接的的connection
Streams: logical HTTP request/response pairs that are layered on
connections. Each connection has its own allocation limit, which defines how many
concurrent streams that connection can carry. HTTP/1.x connections can carry 1 stream
at a time, HTTP/2 typically carry multiple.
HTTP 请求/响应对的连接,每个连接都都自己的自己的分配限制,定义了现在连接
可以携带的并发流。Http/1.x每次只能携带一个,HTTP/2通常可以携带多个
Calls: a logical sequence of streams, typically an initial request and
its follow up requests. We prefer to keep all streams of a single call on the same
connection for better behavior and locality.
流的逻辑顺序,通常是初始请求和后续request,一般将一个call里面的所有stream
放在同一个连接中。
从注释我们可以看到,Connection是建立在Socket之上的物流通信信道,而Stream代表的是逻辑的流,Call是对一次请求的封装。一个Call可能涉及到多个流(比如重定向或者auth认证等情况)。
我们先看它的属性:
public final Address address;//地址
private RouteSelector.Selection routeSelection;//被选择路由的设置
private Route route;//路由
private final ConnectionPool connectionPool;//连接池
public final Call call;//请求
public final EventListener eventListener;
private final Object callStackTrace;//地址
// State guarded by connectionPool.
private final RouteSelector routeSelector;//路由选择器
private int refusedStreamCount;//拒绝次数
private RealConnection connection;//连接
private boolean reportedAcquired;
private boolean released;//是否释放了
private boolean canceled;//是否取消了
private HttpCodec codec;
然后看一下它的构造:
public StreamAllocation(ConnectionPool connectionPool, Address address, Call call,
EventListener eventListener, Object callStackTrace) {
this.connectionPool = connectionPool;
this.address = address;
this.call = call;
this.eventListener = eventListener;
this.routeSelector = new RouteSelector(address, routeDatabase(), call, eventListener);
this.callStackTrace = callStackTrace;
}
其实也就是进行一些赋值操作
然后我们来看它的newStream
方法:
public HttpCodec newStream(
OkHttpClient client, Interceptor.Chain chain, boolean doExtensiveHealthChecks) {
int connectTimeout = chain.connectTimeoutMillis();
int readTimeout = chain.readTimeoutMillis();
int writeTimeout = chain.writeTimeoutMillis();
int pingIntervalMillis = client.pingIntervalMillis();
boolean connectionRetryEnabled = client.retryOnConnectionFailure();
try {
//获取一个连接
RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
writeTimeout, pingIntervalMillis, connectionRetryEnabled, doExtensiveHealthChecks);
//实例化HttpCodec,如果是HTTP/2则是Http2Codec,否则是Http1Codec
HttpCodec resultCodec = resultConnection.newCodec(client, chain, this);
synchronized (connectionPool) {
codec = resultCodec;
return resultCodec;
}
} catch (IOException e) {
throw new RouteException(e);
}
}
可以看到,在这里,显示获取一个连接,然后获取了一个流
我们先看获取连接这个:
private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled,
boolean doExtensiveHealthChecks) throws IOException {
while (true) {
RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
pingIntervalMillis, connectionRetryEnabled);
// If this is a brand new connection, we can skip the extensive health checks.
synchronized (connectionPool) {
if (candidate.successCount == 0) {
return candidate;
}
}
// Do a (potentially slow) check to confirm that the pooled connection is still good. If it
// isn't, take it out of the pool and start again.
if (!candidate.isHealthy(doExtensiveHealthChecks)) {
noNewStreams();
continue;
}
return candidate;
}
}
我们可以看到这个方法是通过findConnection
方法来获取一个RealConnection,如果这个RealConnection没有连接过,那么它就是一个新的连接,不用判断它是否health,否则调用它自身的isHealthy
方法,判断连接是否健康,如果是健康的连接,则重用,否则就继续查找
我们继续往深了看:
如果获取连接的:
/**
* Returns a connection to host a new stream. This prefers the existing connection if it exists,
* then the pool, finally building a new connection.
*/
private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
boolean foundPooledConnection = false;
RealConnection result = null;
Route selectedRoute = null;
Connection releasedConnection;
Socket toClose;
synchronized (connectionPool) {
if (released) throw new IllegalStateException("released");
if (codec != null) throw new IllegalStateException("codec != null");
if (canceled) throw new IOException("Canceled");
// Attempt to use an already-allocated connection. We need to be careful here because our
// already-allocated connection may have been restricted from creating new streams.
//获取存在的连接
releasedConnection = this.connection;
toClose = releaseIfNoNewStreams();
if (this.connection != null) {
// We had an already-allocated connection and it's good.
//如果连接不为空,且好着,就赋值给result
result = this.connection;
releasedConnection = null;
}
if (!reportedAcquired) {
// If the connection was never reported acquired, don't report it as released!
releasedConnection = null;
}
if (result == null) {
// Attempt to get a connection from the pool.
//如果不存在已经存在的连接,就从connectionPool中获取依欧格
Internal.instance.get(connectionPool, address, this, null);
if (connection != null) {
foundPooledConnection = true;
result = connection;
} else {
selectedRoute = route;
}
}
}
closeQuietly(toClose);
if (releasedConnection != null) {
eventListener.connectionReleased(call, releasedConnection);
}
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
}
if (result != null) {
// If we found an already-allocated or pooled connection, we're done.
//如果已经找到了一个已经分配好了的连接,就直接返回
return result;
}
// If we need a route selection, make one. This is a blocking operation.
//线路选择,多ip的支持
boolean newRouteSelection = false;
if (selectedRoute == null && (routeSelection == null || !routeSelection.hasNext())) {
newRouteSelection = true;
//里面是一个递归的操作
routeSelection = routeSelector.next();
}
synchronized (connectionPool) {
if (canceled) throw new IOException("Canceled");
if (newRouteSelection) {
// Now that we have a set of IP addresses, make another attempt at getting a connection from
// the pool. This could match due to connection coalescing.
//如果有新的线路的选择,重新尝试获连接
List<Route> routes = routeSelection.getAll();
for (int i = 0, size = routes.size(); i < size; i++) {
Route route = routes.get(i);
Internal.instance.get(connectionPool, address, this, route);
if (connection != null) {
foundPooledConnection = true;
result = connection;
this.route = route;
break;
}
}
}
//以上都不行,也就是无法从ConnectionPool中获取,就重新新建一个
if (!foundPooledConnection) {
if (selectedRoute == null) {
//得到选择的线路
selectedRoute = routeSelection.next();
}
// Create a connection and assign it to this allocation immediately. This makes it possible
// for an asynchronous cancel() to interrupt the handshake we're about to do.
route = selectedRoute;
refusedStreamCount = 0;
result = new RealConnection(connectionPool, selectedRoute);
//关联到Connection.allocations上
acquire(result, false);
}
}
// If we found a pooled connection on the 2nd time around, we're done.
//如果是从ConnectionPool中找到的,就直接返回
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
return result;
}
// Do TCP + TLS handshakes. This is a blocking operation.
//连接并握手
result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis,
connectionRetryEnabled, call, eventListener);
routeDatabase().connected(result.route());
Socket socket = null;
synchronized (connectionPool) {
reportedAcquired = true;
// Pool the connection.
//把连接放到ConnectionPool中
Internal.instance.put(connectionPool, result);
// If another multiplexed connection to the same address was created concurrently, then
// release this connection and acquire that one.
//如果这个连接是多路复用
if (result.isMultiplexed()) {
//调用connectionPool的deduplicate方法去重。
socket = Internal.instance.deduplicate(connectionPool, address, this);
result = connection;
}
}
//如果是重复的socket则关闭socket,不是则socket为nul,什么也不做
closeQuietly(socket);
eventListener.connectionAcquired(call, result);
return result;
}
public void acquire(RealConnection connection, boolean reportedAcquired) {
assert (Thread.holdsLock(connectionPool));
if (this.connection != null) throw new IllegalStateException();
this.connection = connection;
this.reportedAcquired = reportedAcquired;
connection.allocations.add(new StreamAllocationReference(this, callStackTrace));
}
在这段代码里面,不停的在ConnectionPool里寻找一个符合条件的Connection,可以看到一个连接是RealConnection
那么RealConnection是什么,它是怎么连接的?
RealConnection
RealConnection是Connection的实现类,代表这链接Socket的链路,如果拥有了一个RealConnection就代表已经跟服务器有了一条通路连接,有了通路连接就可以进行三次握手了
先看它的属性和构造:
public final class RealConnection extends Http2Connection.Listener implements Connection {
private static final String NPE_THROW_WITH_NULL = "throw with null exception";
private static final int MAX_TUNNEL_ATTEMPTS = 21;
private final ConnectionPool connectionPool;
private final Route route;
// The fields below are initialized by connect() and never reassigned.
//下面这些字段通过connect方法初始化,而且绝对不会再一次赋值
/** The low-level TCP socket. */
private Socket rawSocket; //底层socket
/**
* The application layer socket. Either an {@link SSLSocket} layered over {@link #rawSocket}, or
* {@link #rawSocket} itself if this connection does not use SSL.
*/
private Socket socket;//应用层socket
private Handshake handshake;//握手
private Protocol protocol;//协议
private Http2Connection http2Connection;//Http2的连接
private BufferedSource source;
private BufferedSink sink;
// The fields below track connection state and are guarded by connectionPool.
//下面这个字段 表示链接状态,并且由ConnectionPool管理
/** If true, no new streams can be created on this connection. Once true this is always true. */
//如果noNewStreams被设为true,则noNewStreams一直为true,不会被改变,并且表示这个链接不会再创新的stream流
public boolean noNewStreams;
//成功的次数
public int successCount;
/**
* The maximum number of concurrent streams that can be carried by this connection. If {@code
* allocations.size() < allocationLimit} then new streams can be created on this connection.
*/
//此链接可以承载最大并发流的限制,如果不超过限制,可以随意增加
public int allocationLimit = 1;
/** Current streams carried by this connection. */
//由此连接承载的流
public final List<Reference<StreamAllocation>> allocations = new ArrayList<>();
/** Nanotime timestamp when {@code allocations.size()} reached zero. */
public long idleAtNanos = Long.MAX_VALUE;
public RealConnection(ConnectionPool connectionPool, Route route) {
this.connectionPool = connectionPool;
this.route = route;
}
然后我们来看它的连接:
public void connect(int connectTimeout, int readTimeout, int writeTimeout,
int pingIntervalMillis, boolean connectionRetryEnabled, Call call,
EventListener eventListener) {
if (protocol != null) throw new IllegalStateException("already connected");
//线路的选择
RouteException routeException = null;
List<ConnectionSpec> connectionSpecs = route.address().connectionSpecs();
ConnectionSpecSelector connectionSpecSelector = new ConnectionSpecSelector(connectionSpecs);
if (route.address().sslSocketFactory() == null) {
if (!connectionSpecs.contains(ConnectionSpec.CLEARTEXT)) {
throw new RouteException(new UnknownServiceException(
"CLEARTEXT communication not enabled for client"));
}
String host = route.address().url().host();
if (!Platform.get().isCleartextTrafficPermitted(host)) {
throw new RouteException(new UnknownServiceException(
"CLEARTEXT communication to " + host + " not permitted by network security policy"));
}
} else {
if (route.address().protocols().contains(Protocol.H2_PRIOR_KNOWLEDGE)) {
throw new RouteException(new UnknownServiceException(
"H2_PRIOR_KNOWLEDGE cannot be used with HTTPS"));
}
}
//开始连接
while (true) {
try {
//如果要求隧道模式,建立隧道连接,通常不是这种
if (route.requiresTunnel()) {
connectTunnel(connectTimeout, readTimeout, writeTimeout, call, eventListener);
if (rawSocket == null) {
// We were unable to connect the tunnel but properly closed down our resources.
break;
}
} else {
//一般都走这个逻辑,实际上就是socket的连接
connectSocket(connectTimeout, readTimeout, call, eventListener);
}
//https的建立
establishProtocol(connectionSpecSelector, pingIntervalMillis, call, eventListener);
eventListener.connectEnd(call, route.socketAddress(), route.proxy(), protocol);
break;
} catch (IOException e) {
closeQuietly(socket);
closeQuietly(rawSocket);
socket = null;
rawSocket = null;
source = null;
sink = null;
handshake = null;
protocol = null;
http2Connection = null;
eventListener.connectFailed(call, route.socketAddress(), route.proxy(), null, e);
if (routeException == null) {
routeException = new RouteException(e);
} else {
routeException.addConnectException(e);
}
if (!connectionRetryEnabled || !connectionSpecSelector.connectionFailed(e)) {
throw routeException;
}
}
}
if (route.requiresTunnel() && rawSocket == null) {
ProtocolException exception = new ProtocolException("Too many tunnel connections attempted: "
+ MAX_TUNNEL_ATTEMPTS);
throw new RouteException(exception);
}
if (http2Connection != null) {
synchronized (connectionPool) {
allocationLimit = http2Connection.maxConcurrentStreams();
}
}
}
这里的大体过程:
- 检查连接是否已经建立,若已经建立,则抛出异常,否则继续,连接是否建立由protocol标识,它表示整个连接建立,及可能的协商过程中选择所有要用到的协议。
- 用集合connectionSpecs构造ConnectionSpecSelector
- 如果请求是不安全的请求,会对请求执行一些额外的限制:
3.1 ConnectionSpec集合必须包含ConnectionSpec.CLEARTEXT。也就是说OkHttp用户可以通过OkHttpClient设置不包含ConnectionSpec.CLEARTEXT的ConnectionSpec集合来禁用所有的明文要求。
3.2 平台本身的安全策略允向相应的主机发送明文请求。对于Android平台而言,这种安全策略主要由系统的组件android.security.NetworkSecurityPolicy执行。平台的这种安全策略不是每个Android版本都有的。Android6.0之后存在这种控制。
(okhttp/okhttp/src/main/java/okhttp3/internal/platform/AndroidPlatform.java 里面的isCleartextTrafficPermitted()方法) - 根据请求判断是否需要建立隧道连接,如果建立隧道连接则调用
connectTunnel(connectTimeout, readTimeout, writeTimeout); - 如果不是隧道连接则调用connectSocket(connectTimeout, readTimeout);建立普通连接。
- 通过调用establishProtocol建立协议,主要是https
- 如果是HTTP/2,则设置相关属性。
然后我们来看一下connectSocket方法:
/** Does all the work necessary to build a full HTTP or HTTPS connection on a raw socket. */
private void connectSocket(int connectTimeout, int readTimeout, Call call,
EventListener eventListener) throws IOException {
Proxy proxy = route.proxy();
Address address = route.address();
//根据代理类型来选择socket类型,是代理还是直连
rawSocket = proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.HTTP
? address.socketFactory().createSocket()
: new Socket(proxy);
eventListener.connectStart(call, route.socketAddress(), proxy);
rawSocket.setSoTimeout(readTimeout);
try {
//连接socket,之所以这样写是因为支持不同的平台
//里面实际上是 socket.connect(address, connectTimeout);
Platform.get().connectSocket(rawSocket, route.socketAddress(), connectTimeout);
} catch (ConnectException e) {
ConnectException ce = new ConnectException("Failed to connect to " + route.socketAddress());
ce.initCause(e);
throw ce;
}
// The following try/catch block is a pseudo hacky way to get around a crash on Android 7.0
// More details:
// https://github.com/square/okhttp/issues/3245
// https://android-review.googlesource.com/#/c/271775/
try {
//得到输入,输出流
source = Okio.buffer(Okio.source(rawSocket));
sink = Okio.buffer(Okio.sink(rawSocket));
} catch (NullPointerException npe) {
if (NPE_THROW_WITH_NULL.equals(npe.getMessage())) {
throw new IOException(npe);
}
}
}
//platform.java
public void connectSocket(Socket socket, InetSocketAddress address, int connectTimeout)
throws IOException {
socket.connect(address, connectTimeout);
}
/**
* Connects this socket to the server with a specified timeout value.
* A timeout of zero is interpreted as an infinite timeout. The connection
* will then block until established or an error occurs.
*
* @param endpoint the {@code SocketAddress}
* @param timeout the timeout value to be used in milliseconds.
* @throws IOException if an error occurs during the connection
* @throws SocketTimeoutException if timeout expires before connecting
* @throws java.nio.channels.IllegalBlockingModeException
* if this socket has an associated channel,
* and the channel is in non-blocking mode
* @throws IllegalArgumentException if endpoint is null or is a
* SocketAddress subclass not supported by this socket
* @since 1.4
* @spec JSR-51
*/
public void connect(SocketAddress endpoint, int timeout) throws IOException {
if (endpoint == null)
throw new IllegalArgumentException("connect: The address can't be null");
if (timeout < 0)
throw new IllegalArgumentException("connect: timeout can't be negative");
if (isClosed())
throw new SocketException("Socket is closed");
if (!oldImpl && isConnected())
throw new SocketException("already connected");
if (!(endpoint instanceof InetSocketAddress))
throw new IllegalArgumentException("Unsupported address type");
InetSocketAddress epoint = (InetSocketAddress) endpoint;
InetAddress addr = epoint.getAddress ();
int port = epoint.getPort();
checkAddress(addr, "connect");//检查是否是ipv4或者ipv6
SecurityManager security = System.getSecurityManager();
if (security != null) {
if (epoint.isUnresolved())
security.checkConnect(epoint.getHostName(), port);
else
security.checkConnect(addr.getHostAddress(), port);
}
//进行连接
//impl就是SocksSocketImpl的一个对象,socket是对它的包装
if (!created)
createImpl(true);
if (!oldImpl)
impl.connect(epoint, timeout);
else if (timeout == 0) {
if (epoint.isUnresolved())
impl.connect(addr.getHostName(), port);
else
impl.connect(addr, port);
} else
throw new UnsupportedOperationException("SocketImpl.connect(addr, timeout)");
connected = true;
/*
* If the socket was not bound before the connect, it is now because
* the kernel will have picked an ephemeral port & a local address
*/
bound = true;
}
在上面我们有提到,Connection的状态由ConnectionPool来管理,那么就来看看这个ConnectionPool:
先看官方对它的描述:
Manages reuse of HTTP and HTTP/2 connections for reduced network latency. HTTP requests that
share the same {@link Address} may share a {@link Connection}. This class implements the policy
of which connections to keep open for future use.
管理http和http/2的连接,以便减少网络请求延迟。同一个address将共享同一个connection。该类实现了连接复用。
然后我们来看看它的字段:
//这是一一个用于清除过期连接的线程池,每个线程池最多只能运行一个线程
//并且这个线程池允许被垃圾回收
private static final Executor executor = new ThreadPoolExecutor(0 /* corePoolSize */,
Integer.MAX_VALUE /* maximumPoolSize */, 60L /* keepAliveTime */, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(), Util.threadFactory("OkHttp ConnectionPool", true));
/** The maximum number of idle connections for each address. */
//每个address的最大空闲连接数
private final int maxIdleConnections;
private final long keepAliveDurationNs;
//清理任务
private final Runnable cleanupRunnable = new Runnable() {
@Override public void run() {
while (true) {
long waitNanos = cleanup(System.nanoTime());
if (waitNanos == -1) return;
if (waitNanos > 0) {
long waitMillis = waitNanos / 1000000L;
waitNanos -= (waitMillis * 1000000L);
synchronized (ConnectionPool.this) {
try {
ConnectionPool.this.wait(waitMillis, (int) waitNanos);
} catch (InterruptedException ignored) {
}
}
}
}
}
};
//双向队列
private final Deque<RealConnection> connections = new ArrayDeque<>();
//路由数据库
final RouteDatabase routeDatabase = new RouteDatabase();
//清理任务正在执行的标识
boolean cleanupRunning;
然后看它的构造函数:
//创建一个使用与单个应用程序的新连接池。
//该连接池的参数将在未来的okhttp中发生改变
//目前最多可容纳5个空闲的连接,存活期是5分钟
public ConnectionPool() {
this(5, 5, TimeUnit.MINUTES);
}
public ConnectionPool(int maxIdleConnections, long keepAliveDuration, TimeUnit timeUnit) {
this.maxIdleConnections = maxIdleConnections;
this.keepAliveDurationNs = timeUnit.toNanos(keepAliveDuration);
// Put a floor on the keep alive duration, otherwise cleanup will spin loop.
if (keepAliveDuration <= 0) {
throw new IllegalArgumentException("keepAliveDuration <= 0: " + keepAliveDuration);
}
}
我们可以知道,这个连接池最多维持5个连接,且每个连接最多活5分钟,并且包含一个线程池和一个清理任务。
maxIdelConnections是指每个地址最大空闲连接数,也就是OKHttp只是限制于对一个远程服务器的空闲连接数量,对整体的空闲连接没有限制。
在findConnection
方法中,会调用Internal.instance.get(connectionPool, address, this, route);
去得到一个合适的连接,实际上是调用了ConnectionPool的get方法,所以我们看看ConnectionPool的get方法:
@Nullable RealConnection get(Address address, StreamAllocation streamAllocation, Route route) {
//断言,判断线程是否被自己锁住了
assert (Thread.holdsLock(this));
//遍历连接的集合
for (RealConnection connection : connections) {
//如果connection和需求中的“地址”和“路由”匹配
if (connection.isEligible(address, route)) {
//复用这个连接
streamAllocation.acquire(connection, true);
//返回这个连接
return connection;
}
}
return null;
}
在findConnection
方法中,如果在connectionPool中没有找到可用的连接,就会新建一个,新建之后就需要放入connectionPool中,就会调用Internal.instance.put(connectionPool, result);
,同样是调用ConnectionPool的put
方法:
void put(RealConnection connection) {
assert (Thread.holdsLock(this));
if (!cleanupRunning) {
cleanupRunning = true;
executor.execute(cleanupRunnable);
}
connections.add(connection);
}
put方法更为简单,就是异步触发清理任务,然后将连接添加到队列中,那么我们看看它的清理任务:
private final Runnable cleanupRunnable = new Runnable() {
@Override public void run() {
while (true) {
long waitNanos = cleanup(System.nanoTime());
if (waitNanos == -1) return;
if (waitNanos > 0) {
long waitMillis = waitNanos / 1000000L;
waitNanos -= (waitMillis * 1000000L);
synchronized (ConnectionPool.this) {
try {
ConnectionPool.this.wait(waitMillis, (int) waitNanos);
} catch (InterruptedException ignored) {
}
}
}
}
}
};
其实就是调用cleanup
方法来执行清理,并等待一段时间,持续清理,其中cleanup方法返回的值决定等待的事件长度:
/**
* Performs maintenance on this pool, evicting the connection that has been idle the longest if
* either it has exceeded the keep alive limit or the idle connections limit.
*
* <p>Returns the duration in nanos to sleep until the next scheduled call to this method. Returns
* -1 if no further cleanups are required.
*/
//维护这个pool,如果有连接超过了它的最大存活时间或者超出了空闲连接限制,将清除连接时间最长的那个
long cleanup(long now) {
int inUseConnectionCount = 0;
int idleConnectionCount = 0;
RealConnection longestIdleConnection = null;
long longestIdleDurationNs = Long.MIN_VALUE;
// Find either a connection to evict, or the time that the next eviction is due.
synchronized (this) {
for (Iterator<RealConnection> i = connections.iterator(); i.hasNext(); ) {
RealConnection connection = i.next();
// If the connection is in use, keep searching.
if (pruneAndGetAllocationCount(connection, now) > 0) {
//使用数量
inUseConnectionCount++;
continue;
}
//统计空闲连接数
idleConnectionCount++;
// If the connection is ready to be evicted, we're done.
long idleDurationNs = now - connection.idleAtNanos;
if (idleDurationNs > longestIdleDurationNs) {
//找出空闲事件最长的连接以及对应的空闲时间
longestIdleDurationNs = idleDurationNs;
longestIdleConnection = connection;
}
}
if (longestIdleDurationNs >= this.keepAliveDurationNs
|| idleConnectionCount > this.maxIdleConnections) {
// We've found a connection to evict. Remove it from the list, then close it below (outside
// of the synchronized block).
//如果超出存活时间或者空闲连接超出最大连接数,就清理空闲时间最长的连接
connections.remove(longestIdleConnection);
} else if (idleConnectionCount > 0) {
// A connection will be ready to evict soon.
//不符合清理条件,则返回下次需要执行清理的等待时间,也就是此连接即将到期的时间
return keepAliveDurationNs - longestIdleDurationNs;
} else if (inUseConnectionCount > 0) {
// All connections are in use. It'll be at least the keep alive duration 'til we run again.
/没有空闲的连接,则隔keepAliveDuration(分钟)之后再次执行
return keepAliveDurationNs;
} else {
// No connections, idle or in use.
//清理结束
cleanupRunning = false;
return -1;
}
}
//关闭socket资源
closeQuietly(longestIdleConnection.socket());
// Cleanup again immediately.
//这里是在清理一个空闲时间最长的连接以后会执行到这里,需要立即再次执行清理
return 0;
}
上面代码中,pruneAndGetAllocationCount
可以看出是很重要的,所以我们需要看看它:
/**
* Prunes any leaked allocations and then returns the number of remaining live allocations on
* {@code connection}. Allocations are leaked if the connection is tracking them but the
* application code has abandoned them. Leak detection is imprecise and relies on garbage
* collection.
*/
private int pruneAndGetAllocationCount(RealConnection connection, long now) {
List<Reference<StreamAllocation>> references = connection.allocations;
//遍历所引用列表
for (int i = 0; i < references.size(); ) {
Reference<StreamAllocation> reference = references.get(i);
//如果StreamAllocation被使用则继续循环
if (reference.get() != null) {
i++;
continue;
}
// We've discovered a leaked allocation. This is an application bug.
StreamAllocation.StreamAllocationReference streamAllocRef =
(StreamAllocation.StreamAllocationReference) reference;
String message = "A connection to " + connection.route().address().url()
+ " was leaked. Did you forget to close a response body?";
Platform.get().logCloseableLeak(message, streamAllocRef.callStackTrace);
//若StreamAllocation未被使用则移除引用,这边注释为泄露
references.remove(i);
connection.noNewStreams = true;
// If this was the last allocation, the connection is eligible for immediate eviction.
//如果列表为空则说明此连接没有被引用了,则返回0,表示此连接是空闲连接
if (references.isEmpty()) {
connection.idleAtNanos = now - keepAliveDurationNs;
return 0;
}
}
return references.size();
}
然后我们回到StreamAllcatiom的findHealthyConnection
方法中,在这个方法中,我们会通过findConnection
方法得到一个RealConnection,如果该connection是新的,就直接返回,否则我们会进行再一次检查(除了get请求),调用RealConnection的isHealthy
方法:
/** Returns true if this connection is ready to host new streams. */
public boolean isHealthy(boolean doExtensiveChecks) {
if (socket.isClosed() || socket.isInputShutdown() || socket.isOutputShutdown()) {
return false;
}
if (http2Connection != null) {
return !http2Connection.isShutdown();
}
if (doExtensiveChecks) {
try {
int readTimeout = socket.getSoTimeout();
try {
socket.setSoTimeout(1);
if (source.exhausted()) {
return false; // Stream is exhausted; socket is closed.
}
return true;
} finally {
socket.setSoTimeout(readTimeout);
}
} catch (SocketTimeoutException ignored) {
// Read timed out; socket is good.
} catch (IOException e) {
return false; // Couldn't read; socket is closed.
}
}
return true;
}
我们可以看到其实就是判断socket是否关闭,是否超时等。
然后回到findHealthyConnection
方法中,如果找到的connection不是新的,而且isHealthy
返回false,就会调用onNewStreams
方法,继续寻找:
/** Forbid new streams from being created on the connection that hosts this allocation. */
//将在刚才找到的connection中,禁止承载stream,让此connection释放
public void noNewStreams() {
Socket socket;
Connection releasedConnection;
synchronized (connectionPool) {
releasedConnection = connection;
socket = deallocate(true, false, false);
if (connection != null) releasedConnection = null;
}
closeQuietly(socket);
if (releasedConnection != null) {
eventListener.connectionReleased(call, releasedConnection);
}
}
然后我们回到StreamAllocation的newStream方法中:
当我们找到了一个可用的并且连接好的连接,然后通过resultConnection.newCodec()
方法,得到一个HttpCodec。
HttpCodec是什么呢?在OKHttp中,HttpCodec是网络读写的管理类,也可以理解为解码器,它有对应两个子类,Http1Codec和Http2Codec,分别对应Http/1.1和Http/2.0协议:
/** Encodes HTTP requests and decodes HTTP responses. */
public interface HttpCodec {
/**
* The timeout to use while discarding a stream of input data. Since this is used for connection
* reuse, this timeout should be significantly less than the time it takes to establish a new
* connection.
*/
int DISCARD_STREAM_TIMEOUT_MILLIS = 100;
/** Returns an output stream where the request body can be streamed. */
Sink createRequestBody(Request request, long contentLength);
/** This should update the HTTP engine's sentRequestMillis field. */
void writeRequestHeaders(Request request) throws IOException;
/** Flush the request to the underlying socket. */
void flushRequest() throws IOException;
/** Flush the request to the underlying socket and signal no more bytes will be transmitted. */
void finishRequest() throws IOException;
/**
* Parses bytes of a response header from an HTTP transport.
*
* @param expectContinue true to return null if this is an intermediate response with a "100"
* response code. Otherwise this method never returns null.
*/
Response.Builder readResponseHeaders(boolean expectContinue) throws IOException;
/** Returns a stream that reads the response body. */
ResponseBody openResponseBody(Response response) throws IOException;
/**
* Cancel this stream. Resources held by this stream will be cleaned up, though not synchronously.
* That may happen later by the connection pool thread.
*/
void cancel();
}
- writeRequestHeaders(Request request) :写入请求头
- createRequestBody(Request request, long contentLength) :写入请求体
- flushRequest() 相当于flush,把请求刷入底层socket
- finishRequest() throws IOException : 相当于flush,把请求输入底层socket并不在发出请求
- readResponseHeaders(boolean expectContinue) //读取响应头
- openResponseBody(Response response) //读取响应体
- void cancel() :取消请求
然后我们来看看它的子类:
Http1Codec
在Http1Codec中主要属性是source和sink,它们分别封装了socket的输入和输出。
先看Http1Codec中各种状态的定义:
private static final int STATE_IDLE = 0; // Idle connections are ready to write request headers.
private static final int STATE_OPEN_REQUEST_BODY = 1;
private static final int STATE_WRITING_REQUEST_BODY = 2;
private static final int STATE_READ_RESPONSE_HEADERS = 3;
private static final int STATE_OPEN_RESPONSE_BODY = 4;
private static final int STATE_READING_RESPONSE_BODY = 5;
private static final int STATE_CLOSED = 6;
上面定义了Http1Codec中使用的状态模式,其实就是对象维护它所处的状态,在不同的状态下执行对应的逻辑,并更新状态,在执行逻辑之前通过检查对象状态避免网络请求的若干执行步骤发生错乱。
Http1Codec的属性:
private static final int HEADER_LIMIT = 256 * 1024;
/** The client that configures this stream. May be null for HTTPS proxy tunnels. */
final OkHttpClient client;
/** The stream allocation that owns this stream. May be null for HTTPS proxy tunnels. */
final StreamAllocation streamAllocation;
final BufferedSource source;
final BufferedSink sink;
int state = STATE_IDLE;
private long headerLimit = HEADER_LIMIT;
一般执行网络请求的步骤为,写入请求头,写入请求体,读取响应头,读取响应体。
所以我们先看写请求头:
@Override public void writeRequestHeaders(Request request) throws IOException {
String requestLine = RequestLine.get(
request, streamAllocation.connection().route().proxy().type());
writeRequest(request.headers(), requestLine);
}
/** Returns bytes of a request header for sending on an HTTP transport. */
public void writeRequest(Headers headers, String requestLine) throws IOException {
if (state != STATE_IDLE) throw new IllegalStateException("state: " + state);
sink.writeUtf8(requestLine).writeUtf8("\r\n");
for (int i = 0, size = headers.size(); i < size; i++) {
sink.writeUtf8(headers.name(i))
.writeUtf8(": ")
.writeUtf8(headers.value(i))
.writeUtf8("\r\n");
}
sink.writeUtf8("\r\n");
state = STATE_OPEN_REQUEST_BODY;
}
RequestLine.get就是用StringBuilder的append方法来构建一个HTTP请求首部:请求方法,URL,Http版本(这里为HTTP 1.1)
然后在writeRequest方法中,通过sink写入请求头,最后状态改变为STATE_OPEN_REQUEST_BODY
然后我们看一下写入请求体:
@Override public Sink createRequestBody(Request request, long contentLength) {
if ("chunked".equalsIgnoreCase(request.header("Transfer-Encoding"))) {
// Stream a request body of unknown length.
return newChunkedSink();
}
if (contentLength != -1) {
// Stream a request body of a known length.
return newFixedLengthSink(contentLength);
}
throw new IllegalStateException(
"Cannot stream a request body without chunked encoding or a known content length!");
}
熟悉HTTP协议的同学都知道其实请求和响应体可以分为固定长度和非固定长度的两种,其中非固定长度由头部信息中的Transfer-Encoding=chunked来表示,固定长度则由对应的头部信息表示实体信息的对应长度。
public Sink newChunkedSink() {
if (state != STATE_OPEN_REQUEST_BODY) throw new IllegalStateException("state: " + state);
state = STATE_WRITING_REQUEST_BODY;
return new ChunkedSink();
}
private final class ChunkedSink implements Sink {
private final ForwardingTimeout timeout = new ForwardingTimeout(sink.timeout());
private boolean closed;
ChunkedSink() {
}
@Override public Timeout timeout() {
return timeout;
}
@Override public void write(Buffer source, long byteCount) throws IOException {
if (closed) throw new IllegalStateException("closed");
if (byteCount == 0) return;
sink.writeHexadecimalUnsignedLong(byteCount);
sink.writeUtf8("\r\n");
sink.write(source, byteCount);
sink.writeUtf8("\r\n");
}
@Override public synchronized void flush() throws IOException {
if (closed) return; // Don't throw; this stream might have been closed on the caller's behalf.
sink.flush();
}
@Override public synchronized void close() throws IOException {
if (closed) return;
closed = true;
sink.writeUtf8("0\r\n\r\n");
detachTimeout(timeout);
state = STATE_READ_RESPONSE_HEADERS;
}
}
这里使用一个内部类ChunkedSink来封装Sink,这个我们知道其中的三个重要的方法,在write方法中,用sink将数据写入。然后刷新和关闭逻辑很简单,其中关闭时注意更新状态。
对于固定长度的请求体,其封装的sink逻辑是类似的,其中需要传入一个bytesRemaining,保证写数据结束时保证数据长度是正确的。
public Sink newFixedLengthSink(long contentLength) {
if (state != STATE_OPEN_REQUEST_BODY) throw new IllegalStateException("state: " + state);
state = STATE_WRITING_REQUEST_BODY;
return new FixedLengthSink(contentLength);
}
private final class FixedLengthSink implements Sink {
private final ForwardingTimeout timeout = new ForwardingTimeout(sink.timeout());
private boolean closed;
private long bytesRemaining;
FixedLengthSink(long bytesRemaining) {
this.bytesRemaining = bytesRemaining;
}
@Override public Timeout timeout() {
return timeout;
}
@Override public void write(Buffer source, long byteCount) throws IOException {
if (closed) throw new IllegalStateException("closed");
checkOffsetAndCount(source.size(), 0, byteCount);
if (byteCount > bytesRemaining) {
throw new ProtocolException("expected " + bytesRemaining
+ " bytes but received " + byteCount);
}
sink.write(source, byteCount);
bytesRemaining -= byteCount;
}
@Override public void flush() throws IOException {
if (closed) return; // Don't throw; this stream might have been closed on the caller's behalf.
sink.flush();
}
@Override public void close() throws IOException {
if (closed) return;
closed = true;
if (bytesRemaining > 0) throw new ProtocolException("unexpected end of stream");
detachTimeout(timeout);
state = STATE_READ_RESPONSE_HEADERS;
}
}
sink,source对象以及封装的sink和source对象,代表http1Codec中的sink和source对象。即封装了socket的输出和输入流,而封装的sink和source对象则是构建的固定长度和非固定长度的输出输入流,其实它们只是对http1Codec成员变量的·中的sink和source的一种封装,其实就是装饰者模式,封装以后的sink和source对象可以用在外部请求和构建ReponseBody。
当写完请求头和请求体之后,需要完成,这时候会调用finishReqeust()方法
@Override public void finishRequest() throws IOException {
sink.flush();
}
其实这一步其实很简单,只有一行代码,就是执行流的刷新。
注意这一步是不需要检查状态的,因为此时的状态有可能是STATE_OPEN_REQUEST_BODY(没有请求体的情况)或者STATE_READ_RESPONSE_HEADERS(已经完成请求体写入的情况)。这一步只是刷新新流,所以什么情况都不会造成影响,所以没有必要检查状态,也没有更新状态,保持之前的状态即可。
接下来是读取响应头:
@Override public Response.Builder readResponseHeaders(boolean expectContinue) throws IOException {
if (state != STATE_OPEN_REQUEST_BODY && state != STATE_READ_RESPONSE_HEADERS) {
throw new IllegalStateException("state: " + state);
}
try {
//readHeaderLiner()方法是读取response的响应头,然后进行转换:Http协议(这里是Http1.1或Http 1.0),
//code,message
StatusLine statusLine = StatusLine.parse(readHeaderLine());
Response.Builder responseBuilder = new Response.Builder()
.protocol(statusLine.protocol)
.code(statusLine.code)
.message(statusLine.message)
.headers(readHeaders());//readHeaders()读取响应头,返回Headers
if (expectContinue && statusLine.code == HTTP_CONTINUE) {
return null;
//HTTP_CONTINUE == 100
} else if (statusLine.code == HTTP_CONTINUE) {
state = STATE_READ_RESPONSE_HEADERS;
return responseBuilder;
}
state = STATE_OPEN_RESPONSE_BODY;
return responseBuilder;
} catch (EOFException e) {
// Provide more context if the server ends the stream before sending a response.
IOException exception = new IOException("unexpected end of stream on " + streamAllocation);
exception.initCause(e);
throw exception;
}
}
private String readHeaderLine() throws IOException {
String line = source.readUtf8LineStrict(headerLimit);
headerLimit -= line.length();
return line;
}
此时所处的状态有可能为STATE_OPEN_REQUEST_BODY和STATE_READ_RESPONSE_HEADERS两种
然后是读取响应体:
@Override public ResponseBody openResponseBody(Response response) throws IOException {
streamAllocation.eventListener.responseBodyStart(streamAllocation.call);
String contentType = response.header("Content-Type");
//如果没有body
if (!HttpHeaders.hasBody(response)) {
Source source = newFixedLengthSource(0);
return new RealResponseBody(contentType, 0, Okio.buffer(source));
}
//如果是非固定长度的
if ("chunked".equalsIgnoreCase(response.header("Transfer-Encoding"))) {
Source source = newChunkedSource(response.request().url());
return new RealResponseBody(contentType, -1L, Okio.buffer(source));
}
long contentLength = HttpHeaders.contentLength(response);
if (contentLength != -1) {
Source source = newFixedLengthSource(contentLength);
return new RealResponseBody(contentType, contentLength, Okio.buffer(source));
}
return new RealResponseBody(contentType, -1L, Okio.buffer(newUnknownLengthSource()));
}
public Source newFixedLengthSource(long length) throws IOException {
if (state != STATE_OPEN_RESPONSE_BODY) throw new IllegalStateException("state: " + state);
state = STATE_READING_RESPONSE_BODY;
return new FixedLengthSource(length);
}
public Source newChunkedSource(HttpUrl url) throws IOException {
if (state != STATE_OPEN_RESPONSE_BODY) throw new IllegalStateException("state: " + state);
state = STATE_READING_RESPONSE_BODY;
return new ChunkedSource(url);
}
public static BufferedSource buffer(Source source) {
return new RealBufferedSource(source);
}
这里和写入请求体的地方十分类似,响应体也是分为固定长度和非固定长度两种,除此以外,为了代码的健壮性okhttp还定义了UnknownLengthSource(位置长度Source,代表意外情况)。
对于Http2Codec就不分析了。
然后我们回到SreamAllcation的newStream方法,这里是调用resultConnection.newCodec()
返回了HrrpCodec:
public HttpCodec newCodec(OkHttpClient client, Interceptor.Chain chain,
StreamAllocation streamAllocation) throws SocketException {
if (http2Connection != null) {
return new Http2Codec(client, chain, streamAllocation, http2Connection);
} else {
socket.setSoTimeout(chain.readTimeoutMillis());
source.timeout().timeout(chain.readTimeoutMillis(), MILLISECONDS);
sink.timeout().timeout(chain.writeTimeoutMillis(), MILLISECONDS);
return new Http1Codec(client, streamAllocation, source, sink);
}
}
其实也就是根据Http2Connection是否为null,创建不停的HttpCodec。
当StreamAllocation的newStream方法返回后,会执行下一个拦截器:
CallServerInterceptor
上面我们已经成功连接到了服务器,那我们就要做请求,发数据了。
@Override public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
HttpCodec httpCodec = realChain.httpStream();
StreamAllocation streamAllocation = realChain.streamAllocation();
RealConnection connection = (RealConnection) realChain.connection();
Request request = realChain.request();
long sentRequestMillis = System.currentTimeMillis();
realChain.eventListener().requestHeadersStart(realChain.call());
//写入请求头
httpCodec.writeRequestHeaders(request);
realChain.eventListener().requestHeadersEnd(realChain.call(), request);
Response.Builder responseBuilder = null;
if (HttpMethod.permitsRequestBody(request.method()) && request.body() != null) {
// If there's a "Expect: 100-continue" header on the request, wait for a "HTTP/1.1 100
// Continue" response before transmitting the request body. If we don't get that, return
// what we did get (such as a 4xx response) without ever transmitting the request body.
//如果请求头里面含有Expect的值等于100-continue,就不需要写入请求体了,直接读取响应头
//如果客户端有POST数据要上传,可以考虑使用100-continue协议。
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
httpCodec.flushRequest();
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(true);//true表示希望继续发送
}
//写入请求体
if (responseBuilder == null) {
// Write the request body if the "Expect: 100-continue" expectation was met.
realChain.eventListener().requestBodyStart(realChain.call());
long contentLength = request.body().contentLength();
CountingSink requestBodyOut =
new CountingSink(httpCodec.createRequestBody(request, contentLength));
BufferedSink bufferedRequestBody = Okio.buffer(requestBodyOut);
request.body().writeTo(bufferedRequestBody);
bufferedRequestBody.close();
realChain.eventListener()
.requestBodyEnd(realChain.call(), requestBodyOut.successfulCount);
} else if (!connection.isMultiplexed()) {
// If the "Expect: 100-continue" expectation wasn't met, prevent the HTTP/1 connection
// from being reused. Otherwise we're still obligated to transmit the request body to
// leave the connection in a consistent state.
streamAllocation.noNewStreams();
}
}
//请求完成
httpCodec.finishRequest();
//读取响应头
if (responseBuilder == null) {
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(false);
}
Response response = responseBuilder
.request(request)
.handshake(streamAllocation.connection().handshake())
.sentRequestAtMillis(sentRequestMillis)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
//读取响应体
int code = response.code();
if (code == 100) {
// server sent a 100-continue even though we did not request one.
// try again to read the actual response
//服务器返回100表示已收到请求的第一部分,正在等待其余部分。
//重新读取响应头
responseBuilder = httpCodec.readResponseHeaders(false);
response = responseBuilder
.request(request)
.handshake(streamAllocation.connection().handshake())
.sentRequestAtMillis(sentRequestMillis)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
code = response.code();
}
realChain.eventListener()
.responseHeadersEnd(realChain.call(), response);
if (forWebSocket && code == 101) {
// Connection is upgrading, but we need to ensure interceptors see a non-null response body.
// 101 (切换协议) 请求者已要求服务器切换协议,服务器已确认并准备切换。
response = response.newBuilder()
.body(Util.EMPTY_RESPONSE)
.build();
} else {
response = response.newBuilder()
.body(httpCodec.openResponseBody(response))
.build();
}
if ("close".equalsIgnoreCase(response.request().header("Connection"))
|| "close".equalsIgnoreCase(response.header("Connection"))) {
streamAllocation.noNewStreams();
}
if ((code == 204 || code == 205) && response.body().contentLength() > 0) {
throw new ProtocolException(
"HTTP " + code + " had non-zero Content-Length: " + response.body().contentLength());
}
return response;
}