1. Abnormal emergent
In this ordinary day, I wrote a general code, but suddenly received no general alarm
javax.net.ssl.SSLHandshakeException: server certificate change is restrictedduring renegotiationView log access requests to pay all xx error, emergency contact each other, know each other to replace the server certificate. Since the connection pool will cache the connection, the old connection can not be released in time, the line has continued to alarm, eventually restart the server, all business was back to normal.
2. questioned
Although the system returned to normal, but there are several issues remained in my mind:
- Why is there this exception?
- HttpClient is how https request?
- In addition to restart the machine, whether there are other ways to deal with?
3. On the Https
This article from the source point of view, explore how HttpClient Https request, and how Java SSL connection is not involved SSL / TLS protocol specific content.
3.1 Https Basics
Https basics predecessors have a very good summary, recommended several blog posts, you can learn.
To better understand the following, two reference herein SSL / TLS handshake process, taken HTTPS depth understanding
FIG authentication server handshake process 1
Pictures from the network
(1) by the client Client Hellosends it supports SSL version, cryptographic algorithms, key exchange algorithm, the MAC algorithm information to the SSL server message.
(2) determine the version server and cipher suite used in the current communication, by Server Helloa notification message to the client. If the server allows the client to reuse this session in a later communication, the server-based sessions assigned a session ID, by Server Hellosending a message to the SSL client.
(3) The server will carry their own digital certificate public key information by Certificatesending a message to the client.
(4) server sends a Server Hello Donemessage to notify the client version and cipher suite negotiated end to start the key exchange.
(5) the authentication server of the client certificate valid, the public key encryption using a certificate client randomly generated premaster secret, and by Client Key Exchangesending a message to the SSL server.
(6) The client sends Change Cipher Speca message, the server notifies the subsequent packets using the negotiated key and cipher suite for encryption and MAC calculation.
(7) handshake message (except the client computing interactions have been Change Cipher Specall messages have message interaction outer) Hash value using the negotiated key and cipher suite processing Hashvalue (calculated value and MAC, encryption, etc.), and by the Finishedmessage sent to the server. Hash value calculation server has a method of interaction using the same handshake message, and compares the decryption result Finished message, if they are identical, and the MAC value verification is successful, the key and cipher suite negotiation success.
(8) in the same manner, the SSL server sends a Change Cipher Specmessage to notify the client subsequent packets using the negotiated key and cipher suite for encryption and MAC calculation.
(9) the server computing handshake messages interaction Hashvalues, using the negotiated key and cipher suite processing Hash value (calculated value and MAC, encryption, etc.), and by Finishedsending a message to the client. Hash value is the client computing interaction by the same method handshake message, and with the Finishedcomparison result of decryption of the message, if they are identical, and the MAC value verification is successful, the key and cipher suite negotiation success.
(10) the client receives the server sends Finishedback a message, if the decryption is successful, the server can determine the owner of a digital certificate that the server authentication is successful, because only with the private key from the server to Client Key Exchangeget the decrypted messages premaster secret, thereby indirectly to achieve the client authentication to the server.
FIG reuse handshake session 2
Negotiation session parameters, the process of establishing a session, you need to use asymmetric key algorithm to encrypt the key, to verify the identity of the peer communication, the large amount of calculation, taking up a lot of system resources. To simplify the SSL handshake process, SSL allows the session has been negotiated reuse, the specific process is:
(1) The client sends Client Helloa message, the session ID in the message is provided to reuse the session ID.
(2) If the server allows the reuse of the session, through the Server Hellosame session ID to the response message. In this way, the client and server can use the original key and cipher suite of the session, you do not have to be renegotiated.
(3) the server sends Change Cipher Speca message to notify the client subsequent packets using the key and cipher suite original session encryption and MAC calculation.
(4) server computing handshake messages interaction Hashvalue by processing the original key and cipher suite session Hashvalues, and by Finishedsending a message to the client, so that the SSL client determines whether the correct key and cipher suite.
(5) The client sends Change Cipher Speca message, the server notifies SSL subsequent packets using the key and cipher suite original session encryption and MAC calculation.
(6) the client computing handshake messages interaction Hashvalue by processing the original key and cipher suite session Hashvalues, and by Finishedsending a message to the SSL server to the SSL server determines whether the correct key and cipher suite.
3.2 HttpClient how to deal with https / http request
Look at the small section of code, which is a simple request code twice koala home. After the code can be seen by, during the second request, and will not reestablish the connection.
public static void main(String[] args) throws IOException {
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("https://www.kaola.com");
CloseableHttpResponse response1 = httpclient.execute(httpget);
CloseableHttpResponse response2 = httpclient.execute(httpget);
}Creating CloseableHttpClient examples
First look at the CloseableHttpClient create a default instance of what had been done thing, where they talk about some of the core attributes. Entryorg.apache.http.impl.client.HttpClientBuilder#build
ClientExecChain
Execution chain, HttpClient uses the chain of responsibility pattern, the request through a series of actuators finally get a result, each actuator has its own responsibilities. The following is the order of the execution chain default:
- RedirectExec: responsible for handling requests redirected
- RetryExec: a determination because the request is responsible for I / O exception whether to retry failed
- ProtocolExec: Http protocol responsible for internal use
HttpProcessorto build the necessary Http request header, and the processing Http response header Session update state toHttpClientContextthe - MainClientExec: performing the last link chain, responsible for performing acquisition request Response, using the
HttpRequestExecutortransmission request
HttpClientConnectionManager
When the connection manager, the default implementation usedPoolingHttpClientConnectionManager, it registered when creating a support different protocols to create a Socket factory, which corresponds to the https protocol isSSLConnectionSocketFactory
PoolingHttpClientConnectionManager poolingmgr = newPoolingHttpClientConnectionManager (RegistryBuilder. < ConnectionSocketFactory> create () .register ( "http", PlainConnectionSocketFactory .getSocketFactory ()) .register ( "https ", sslSocketFactory) .build ());HttpRoutePlanner
The default implementationDefaultRoutePlanner, which determines a request ofHttpRouterouting information, including domain name, port, protocol. Its implementation must be thread-safe.
Https execution request
HttpGet httpget = new HttpGet("https://www.kaola.com");
CloseableHttpResponse response1 = httpclient.execute(httpget);Finally, a direct view of the execution chain ring, just to illustrate some of the key code:
- Code entry
org.apache.http.impl.execchain.MainClientExec#execute
public CloseableHttpResponse execute(final HttpRoute route,final HttpRequestWrapper request,
final HttpClientContext context,final HttpExecutionAware execAware) throws IOException, HttpException {
//...
//获取连接请求，这里没有真正建立连接
final ConnectionRequest connRequest = connManager.requestConnection(route, userToken);
//...
final RequestConfig config = context.getRequestConfig();
//获取连接，这里没有真正建立连接
final HttpClientConnection managedConn;
try {
final int timeout = config.getConnectionRequestTimeout();
managedConn = connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS);
}//... 异常处理
//...
final ConnectionHolder connHolder = new ConnectionHolder(this.log, this.connManager, managedConn);
try {
//...
HttpResponse response;
for (int execCount = 1;; execCount++) {
//...
if (!managedConn.isOpen()) {
try {//2. 建立连接
establishRoute(proxyAuthState, managedConn, route, request, context);
} //异常处理
}
//执行
response = requestExecutor.execute(request, managedConn, context);
//...
}
//释放回连接池
}//...异常处理
}- Establish a connection
org.apache.http.impl.conn.HttpClientConnectionOperator#connectestablishRoutemethod where, eventually callingconnectmethod
public void connect(final ManagedHttpClientConnection conn,final HttpHost host,
final InetSocketAddress localAddress,final int connectTimeout,
final SocketConfig socketConfig,final HttpContext context) throws IOException {
//根据协议获取对应的Socket工厂
final Lookup<ConnectionSocketFactory> registry = getSocketFactoryRegistry(context);
//因为是https协议，这里获取到SSLConnectionSocketFactory
final ConnectionSocketFactory sf = registry.lookup(host.getSchemeName());
//这里会从dns查到多个IP地址，如果连接失败，会尝试连接下一个IP
final InetAddress[] addresses = host.getAddress() != null ?
new InetAddress[] { host.getAddress() } : this.dnsResolver.resolve(host.getHostName());
final int port = this.schemePortResolver.resolve(host);
for (int i = 0; i < addresses.length; i++) {
final InetAddress address = addresses[i];
final boolean last = i == addresses.length - 1;
Socket sock = sf.createSocket(context);//注意，这里创建的一个普通的socket连接
//...
conn.bind(sock);
//...
try {//3. 建立TLS连接
sock = sf.connectSocket(connectTimeout, sock, host, remoteAddress, localAddress, context);
conn.bind(sock);
return;
}//处理异常
}- Establish a TLS connection
org.apache.http.conn.ssl.SSLConnectionSocketFactory#connectSocket
to note here, Https protocol is based on SSL / TLS protocol, SSL / TLS is based on the TCP protocol, so we need to establish a TCP connection, establishes a SSL / TLS connection, and finally transmitted over SSL / TLS Http messages.
public Socket connectSocket(final int connectTimeout,final Socket socket,
final HttpHost host,final InetSocketAddress remoteAddress,
final InetSocketAddress localAddress,final HttpContext context) throws IOException {
//...
try {
sock.connect(remoteAddress, connectTimeout);//建立TCP连接
} //异常处理
//...
//建立TLS连接
return createLayeredSocket(sock, host.getHostName(), remoteAddress.getPort(), context);
}org.apache.http.conn.ssl.SSLConnectionSocketFactory#createLayeredSocket
public Socket createLayeredSocket(final Socket socket,final String target,
final int port,final HttpContext context) throws IOException {
//基于socket创建SSLScoket
final SSLSocket sslsock = (SSLSocket) this.socketfactory.createSocket(socket,target,port,true);
//...
prepareSocket(sslsock);//空实现，预留的hook
sslsock.startHandshake();//开始握手
verifyHostname(sslsock, target);
return sslsock;
}HttpClient request part of this is over, will the normal data exchange (https some special treatment) After a successful connection, the following look at how to build Java SSL / TLS connection
3.3 Java establish SSL / TLS connection
From here there is no source code, part of the variable name is filled according to their own understanding, but self-explanatory JDK is very good, most of them can be understood
Perform the initialization handshake
sun.security.ssl.SSLSocketImpl#performInitialHandshake
private void performInitialHandshake() throws IOException {
synchronized(this.handshakeLock) {
if(this.getConnectionState() == 1) {//连接状态初始化，如果复用连接不会走到这
this.kickstartHandshake();//1.开始握手，发送Client Hello消息
//...2. 读取服务端返回数据
this.readRecord(this.inrec, false);
this.inrec = null;
}
}
}- Client Hello send a message
sun.security.ssl.Handshaker#kickstart
1 herein correspond to the (1) in FIG.
void kickstart() throws IOException {
if(this.state < 0) {
HandshakeMessage messge = this.getKickstartMessage();//构造消息体
//发送消息
messge.write(this.output);
this.output.flush();
this.state = messge.messageType();//握手消息 messageType=22
}
}Client Hello message body configurationsun.security.ssl.ClientHandshaker#getKickstartMessage
HandshakeMessage getKickstartMessage() throws SSLException {
SessionId sessionId = SSLSessionImpl.nullSession.getSessionId();
CipherSuiteList cipherSuiteList = this.getActiveCipherSuites();
this.maxProtocolVersion = this.protocolVersion;
//取session，这里是造成异常的原因，之后分析
this.session = ((SSLSessionContextImpl)this.sslContext.engineGetClientSessionContext()).get(this.getHostSE(),this.getPortSE());
//...
if(this.session != null) {
//从sessino中还原信息
}
if(this.session == null && !this.enableNewSession) {
throw new SSLHandshakeException("No existing session to resume");
} else {
//获取可支持的加密套件
if(!isNegotiable) {
throw new SSLHandshakeException("No negotiable cipher suite");
} else {
//这里会把sessionId添加到ClientHello消息
ClientHello clientHello = new ClientHello(this.sslContext.getSecureRandom(), this.maxProtocolVersion, sessionId, cipherSuiteList);
//...
return clientHello;
}
}
}- The server receives data
sun.security.ssl.SSLSocketImpl#readRecord
The cycle has been read server data until the handshake is complete
private void readRecord(InputRecord inputRecord, boolean var2) throws IOException {
synchronized(this.readLock) {
while(true) {
int var3;
if((var3 = this.getConnectionState()) != 6 && var3 != 4 && var3 != 7) {
try {
inputRecord.setAppDataValid(false);
//读取服务器返回
inputRecord.read(this.sockInput, this.sockOutput);
} //异常处理
//解码
synchronized(this) {
switch(inputRecord.contentType()) {//根据不同的消息类型进行处理
case 20://change_cipher_spec 服务端通知更换密钥 图1 (8)
//...
this.changeReadCiphers();
this.expectingFinished = true;
continue;
case 21://alert
this.recvAlert(inputRecord);
continue;
case 22://handshake 握手相关消息
this.initHandshaker();//初始化ClientHandshaker
//...
//处理数据
this.handshaker.process_record(inputRecord, this.expectingFinished);
this.expectingFinished = false;
//... 完成握手,保存信息,退出
continue;
case 23://application_data
//...
break;
default:
//...
}
return;
}
inputRecord.close();
return;
}
}
}See process_recordcall chain method, will eventually find a way handshake message processing sun.security.ssl.ClientHandshaker#processMessage, different processing performed by the different message types, the control can be appreciated that FIG.
void processMessage(byte handshakeType, int length) throws IOException {
if(this.state >= handshakeType && handshakeType != 0) {
//... 异常
} else {
label105:
switch(handshakeType) {
case 0://hello_request
this.serverHelloRequest(new HelloRequest(this.input));
break;
//...
case 2://sever_hello 图1 (2)
this.serverHello(new ServerHello(this.input, length));
break;
case 11:///certificate 图1 (3)
this.serverCertificate(new CertificateMsg(this.input));
this.serverKey = this.session.getPeerCertificates()[0].getPublicKey();
break;
case 12://server_key_exchange 该消息并不是必须的，取决于协商出的key交换算法
//...
case 13: //certificate_request 客户端双向验证时需要
//...
case 14://server_hello_done 图1 (4)
this.serverHelloDone(new ServerHelloDone(this.input));
break;
case 20://finished 图1 (9)
this.serverFinished(new Finished(this.protocolVersion, this.input, this.cipherSuite));
}
if(this.state < handshakeType) {//握手状态
this.state = handshakeType;
}
}
}In the sun.security.ssl.ClientHandshaker#serverHelloDonemethod, the client returns a cipher suite determined encryption, different configurations Client Key Exchange message according to the service side, for example RSAClientKeyExchange, DHClientKeyExchange, ECDHClientKeyExchange corresponding to FIG. 1 (5).
After sending the ClientKeyExchange, followed by sendChangeCipherAndFinishthe method Change Cipher Spec message is sent and the Finished message, corresponding to FIG. 1 (6) (7).
After Change Cipher Spec message after receiving the service side of FIG. 1 (8), the key switch is completed, a Finished message waiting in FIG. 1 (9), if this time is not the recovery session, session stored in the cache met
So far successfully establish a connection, because of space limitations, every step of sending and receiving messages do not dwell on specific, follow the above analysis can clearly find all entrances.
3.4 Multiplexing handshake session
Multiplexing is included in the session process in the above-described process, only part of the node determines whether there is the session, whether the session recovery actions for different
- Client Hello message carrying the Session FIG. 2 (1)
It would already exists session (key = ip: port) when viewing cached in the handshake if the session exists in the Client Hello message will bring information sessionsun.security.ssl.ClientHandshaker#getKickstartMessage
this.sslContext.engineGetClientSessionContext()).get(this.getHostSE(),this.getPortSE());- Carry Sessin Information Server Hello 2 (2)
In the sun.security.ssl.ClientHandshaker#serverHellomethod determines whether the same client and server came sessionId
if(this.session.getSessionId().equals(severHello.sessionId)) {
//从session中恢复信息
}- Change Cipher Spec message receiving FIG. 2 (3)
In the sun.security.ssl.SSLSocketImpl#changeReadCiphersprocessing method
- Finished message received in FIG. 2 (4)
After receiving the Finished message, to establish different first connection, if it is determined to restore the session, Change Cipher Spec message will be issued and the Finished message, corresponding to FIG. 2 (5) (6)
if(this.resumingSession) {
this.input.digestNow();
this.sendChangeCipherAndFinish(true);
}3.6 data transmission
After the SSL / TLS connection is established, Http how to send packets?
For it does not need to focus on the upper SSL / TLS layer, the data is encrypted by SSLSocketImpl, decryption. Transmitting packets inlet Http
org.apache.http.protocol.HttpRequestExecutor#execute, Using the underlying AppOutputStreamoutput stream. By the final sun.security.ssl.SSLSocketImpl#writeRecordInternaloutput data
private void writeRecordInternal(OutputRecord outputRecord, boolean var2) throws IOException {
outputRecord.addMAC(this.writeMAC);
outputRecord.encrypt(this.writeCipher);//数据加密
//...
outputRecord.write(this.sockOutput, var2, this.heldRecordBuffer);//输出
//...
}4. The resolution of the question
What happened? Why is there an exception?
After replacing the server certificate, the client set up SSL / TLS connection does not fail, simplified handshake process using the cache sessionId when shaking hands, thereby triggering an exception.
How HttpClient be Https request?
HttpClient create a connection using a different Socket factory according to different protocols, for use https SSLConnectionSocketFactorySpecific SSL / TLS connection establishment procedure referred to SSLSocketImpltreatment.
After the connection is established HttpClient will not need to focus on the underlying data encryption, you will be responsible for reading and writing SSLSocketImpl data.
How to deal with?
Restart the machine
Because of the connection pool, waiting for the connection error to re-establish a new connection is not a good choice, which could cause the system to continue abnormalities. If there are no other measures restart Dafa welcome you.
Disable session
After establishing a connection invalidate the cache can be avoided using a simplified handshake process, but a greater impact performance, do not advocate
SSLSocket.getSession().invalidate();Failure connection
Since it is because there is a connection pool to restart the machine, then we can clear the connection pool. In connection pool manager PoolingHttpClientConnectionManagerthere are ways to clean up the idle connection.
void closeIdleConnections(long idletime, TimeUnit tunit);We can idletime set is very small, you can turn off most connected. However, this approach is somewhat rude, may cause injury.
Connection pooling can be customized as needed to customize their desired features such as remote connection to empty the pool, some of the more sophisticated, according to ip + port clean-specified connection.
https://zhuanlan.zhihu.com/p/44786952