TLS/SSL protocol

1. The working principle of the TLS protocol

TSL design purpose:

• Authentication
• Confidentiality
• Completeness

Record protocol

• Symmetric encryption

Handshake agreement

• Verify the identity of the communicating parties
• Security Suite for Exchange Encryption and Decryption
• Negotiate encryption parameters

2. The working principle of symmetric encryption

1), XOR and filling

The plaintext P and the key K are based on the AES encryption function to generate the ciphertext C, the ciphertext C is transmitted to the receiver through the network, and the receiver gets the ciphertext C based on the same key K and AES decryption function to get the plaintext P

The reason why symmetric encryption can realize both encryption and decryption with the same key is based on the exclusive OR operation of symmetric encryption and XOR

The key sequence 1010 and the plaintext 0110 are XORed to get the ciphertext 1100, and the ciphertext 1100 and the key sequence 1010 are XORed to get the plaintext 0110.

2), working mode

1) Group working mode

Allow the use of the same block cipher key to encrypt more than one piece of data and ensure its security

2) ECB (Electronic codebook) mode

Directly decompose the plaintext into multiple blocks, and encrypt each block independently

Problem: Unable to hide data characteristics

3）CBC（Cipher-block chaining）模式

The first group of plaintext is encrypted to obtain the ciphertext, the first group of ciphertext and the second group of plaintext are XORed first and then encrypted, the second group of ciphertext and the third group of plaintext are XORed first and then encrypted

Each plaintext block is XORed with the previous ciphertext block before being encrypted

Problem: The encryption process is serialized, and the second and third steps cannot be performed without completing the first step

4) CTR (Counter) mode

Generate a continuous key stream by incrementing an encryption counter

The ciphertext generated by the counter, key, and plaintext has no data characteristics, because each introduces different variables and can be operated in parallel.

Problem: Cannot provide ciphertext message integrity check

5) Verify integrity: MAC (Message Authentication Code)

After the sender generates the ciphertext, it uses the hash algorithm to generate the MAC value of the ciphertext and key. The ciphertext and MAC value are transmitted to the receiver. After the receiver receives the message, it first generates the MAC based on the same MAC algorithm and the same key. The value, compared with the transmitted MAC value, is it equal? ​​Equality indicates that the message is complete, and you don’t want to wait to indicate that there is an error

6）GCM（Galois/Counter Mode）=CTR+GMAC

3. Explain the AES symmetric encryption algorithm in detail

AES (Advanced Encryption Standard) encryption algorithm

• Commonly used filling algorithm: PKCS7
• Common group working mode: GCM

Three key lengths of AES :

AES encryption steps :

1. Split the plaintext into several plaintext blocks according to 128bit (16 bytes), each plaintext block is a 4*4 matrix
2. Fill the last plaintext block according to the selected filling method
3. Each plaintext block is encrypted into a ciphertext block using AES encryptor and key
4. Join all the ciphertext blocks to become the final ciphertext result

1) AddRoundKey steps

The key here is not the original key, it is a key expansion algorithm

Key extension is to generate a different new key used in the next round from the key used in each round

Each round of the key is a 4*4 16-byte matrix. First, every four bytes form a word, each word is 4 bytes, and the matrix becomes an array of 4 elements, based on the g function and the exclusive Or generate the next round of keys

2) SubBytes step

3) ShiftRows steps

4) MixColumns step

4. Asymmetric encryption and RSA algorithm

Generate a pair of keys according to the asymmetric encryption algorithm: public key and private key, the public key is disclosed to the other party, and the private key is only used by yourself

When encrypting, use the other party’s public key to encrypt the message, when decrypting, use your own private key to decrypt

5. Verification of RSA based on openssl

1) Use openssl to generate public and private keys based on the RSA algorithm

Generate private key :openssl genrsa -out private.pem

hanxiantaodeMBP:tls hanxiantao$ openssl genrsa -out private.pem
Generating RSA private key, 2048 bit long modulus
........................+++
.................................+++
e is 65537 (0x10001)
hanxiantaodeMBP:tls hanxiantao$ cat private.pem 
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAy5bTnsjJXtBMuafMmATFdAAX0jwTCpuuhclvF5y1+5gZ/uRM
dgRVwqnc3aFxHJbXv3rzwy0ehCKkOGue6Qw3FGUTWph573lNbFTI45VhsnllEvM3
cWkdljZ09LLbbOi+iuHrJV+xwauoH/UddiZ40Ih33Wg8wNSqe7KnSgBWK8ABr7Ke
YeimVrwO7hFqnOkjonD5lSamGOvEn5uqkQ5IQfc0gnPRYHrHXaw9xIZ5IRtww8pz
f+zO7XkAziWhLJ9kUtaSkL5Mifbw9kv8Gb+9epcWlBRGgFB/vPdJ+K4I59KMqAgf
zCKNEmGPT3sxBOzxpl5aV8LXFV8Da92nY0f86wIDAQABAoIBAEWF4AZdMsb6Avlz
X96Z4oPWdEwKz8XTnCl7vEAn981PB7GPbLzwhgjP0OiudN36dPqilhOUmNMusT3D
IqUa0sRYL9/EKf+pQNM5sNBm9tHnuqhZ/hjweHYPaqkVWvE6Gbd7pr1AjIdCg0tG
fSUXxjIQKD6nlfeTqBRN0ernaoXNk9K23DYVuBw4rtt4VpLlrABcX8klU1EEDy4s
lv9ZkzYV1F2brVxteaoXUqp7e4TguXLq/OpLC3fKGgn3pIpgILckB/8A3Rx6iH1/
lbj5IBxvy8tm4XtnFGt4nohkiYFlIN9wjiFYHbZ7x5i5Jgw7I1D5lIY5AyEMWyHl
35bcKuECgYEA6KO7UBZ8aYaCKcIpPbAsx6w3oY+tEUJla5G230ZPafT0UxmKLvVJ
/Q6guD850f8P/c5Qp4wZGXADRARKUCB8dqO2qE0IHlhjMq8tUf2s/mMR6lvKkh9q
KKHqiWezPvQAa0DuJ3zcWy1xliDgyOS6oE+CSCwOF+S6kZ3LEi7tGWcCgYEA4AhS
GKimUkvNPL6+j4krdx37HXuxiN5+Su+m9yv92AlIIDE6Z3BrhBRgKO/50CvVYrhL
0I8e0ZjZar+uCTwI0AsJx4wh10oPaIhGNLISXjXbX23WKasnZ/XLEhck9BIgWIf7
qX9iBln2Q4dEoUIWaYrffEd2OtezRGPdCEUyGd0CgYAUyKPwaMHer5yrXGRQ1Y96
m0ExFuPwWc0zygXbdq2bmr3FOs/kmBdvG0Jyk3t37mCgXTFJdrO7WQ2Boxx8ghp8
gu3LpW4nP+BE4++Zlp9A7trn5CF54oKadLS+Z9xUsHnlGxzrvDT3lFzEe9V5PS4L
Km9KQV1U9yNP5RgCXNzj2wKBgQCWXL9NEZRf08RyKsuXZscncZXjGev0IvC8ttBn
QL1kzAX+pUu/tTJUOaC4mSgf0eusEGnCFuzmXCJAhVn3lLWNfHsZ1TG5X1msHRqR
r5qoZJlSGVQOL3OJUOz0vVfuAGR5RvtfrcFK5gJlHFqxqLuGJtEMhqIRqEgHdMcb
D+YqeQKBgQC1Q1+2IikZSlc/Gw6rLeLcAyUerABygb+g6na27I4txpbBQWAk+D2n
rKdSWsk/9eTY5Th72T+zbxnyjaz0oCicyGsGLO0WrI0OxmpTCl5FfDbTBekb8Sps
YjCvQE22mket/sA6cYsCjtj2I4/2aNPIestlzZOeBmg22nDtLQtI9A==
-----END RSA PRIVATE KEY-----

Extract the public key from the private key :openssl rsa -in private.pem -pubout -out public.pem

hanxiantaodeMBP:tls hanxiantao$ openssl rsa -in private.pem -pubout -out public.pem
writing RSA key
hanxiantaodeMBP:tls hanxiantao$ cat public.pem 
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAy5bTnsjJXtBMuafMmATF
dAAX0jwTCpuuhclvF5y1+5gZ/uRMdgRVwqnc3aFxHJbXv3rzwy0ehCKkOGue6Qw3
FGUTWph573lNbFTI45VhsnllEvM3cWkdljZ09LLbbOi+iuHrJV+xwauoH/UddiZ4
0Ih33Wg8wNSqe7KnSgBWK8ABr7KeYeimVrwO7hFqnOkjonD5lSamGOvEn5uqkQ5I
Qfc0gnPRYHrHXaw9xIZ5IRtww8pzf+zO7XkAziWhLJ9kUtaSkL5Mifbw9kv8Gb+9
epcWlBRGgFB/vPdJ+K4I59KMqAgfzCKNEmGPT3sxBOzxpl5aV8LXFV8Da92nY0f8
6wIDAQAB
-----END PUBLIC KEY-----

2), use RSA public and private keys to encrypt and decrypt

Find an original file with the following content

hanxiantaodeMBP:tls hanxiantao$ cat hello.txt 
hello world

Encrypt the file to generate hello.en

hanxiantaodeMBP:tls hanxiantao$ openssl rsautl -encrypt -in hello.txt -inkey public.pem -pubin -out hello.en

Decrypt the file and generate hello.de

hanxiantaodeMBP:tls hanxiantao$ openssl rsautl -decrypt -in hello.en -inkey private.pem -out hello.de
hanxiantaodeMBP:tls hanxiantao$ cat hello.de 
hello world

6. Asymmetric password application: PKI certificate system

Registered users use the RSA algorithm to generate a pair of public and private keys, the private key is saved by themselves, and the public key and personal identity information are applied for and sent to the CA certification authority. After the CA certification agency verifies the identity information, it uses the CA agency’s own private key to encrypt the information. After the encryption is completed, the certificate is called a public key certificate.

Signature process :

Use the hash function to generate a hash value for the transmitted data data, encrypt the hash value with the private key of the CA organization, and get a string of cipher text

Sign verification process :

Use the same hash function to generate a hash value for the transmitted data, and decrypt the cipher text with the public key of the CA organization. If the decryption is successful, it proves that the information is indeed encrypted by the CA organization and verifies the authenticity of the CA organization's identity. Then check whether the two hash values ​​match, if the matching verification is successful

A smaller CA can be signed and certified by a larger CA, but at the end of the chain, the Root CA, you can only prove yourself. This is called a self-signed certificate or a root certificate.

With this certificate system, the operating system and browser have built-in the root certificates of major CAs. As long as the server sends its certificate when surfing the Internet, the signature in the certificate can be verified and verified layer by layer along the certificate chain. Until the root certificate is found, it can be determined that the certificate is trustworthy, so that the public key inside is also trustworthy

The certificate subscriber generates a pair of public and private keys, and sends the personal identity information and public key to the CA registration authority. The CA registration authority verifies the identity of the subscriber. After the verification is passed, it sends the verification signature application to the CA organization. The CA organization uses its own private The certificate is issued after the key is signed, and finally passed to the certificate subscriber, who deploys the certificate on the Web server

After the user establishes the connection, the TLS handshake is performed. During the TLS handshake, the certificate is requested first. The server sends the public key certificate to the relying party, and the relying party (browser) verifies whether the certificate is trustworthy according to the certificate chain

7. Asymmetric cipher application: DH key exchange protocol

RSA key exchange has no forward secrecy. Before cracking the private key, save the ciphertext. After cracking the private key, you can decrypt all the previously saved ciphertexts.

DH key exchange allows both parties to create a key through an insecure channel without any prior information from the other party. Each time the key is generated in real time and has forward secrecy.

Server first generates public and private keys private key1 and public key1, public key public key1 is sent to Client, Client also generates a pair of public and private keys private key2 and public key2, public key public key2 is sent to Server, Client is based on private key2 and public key1 based on one Algorithm generates keys. Server generates keys based on private key1 and public key2. The keys generated by Client and Server are exactly the same, so they can be used for subsequent symmetric encryption.

Problems with the DH key exchange protocol :

Man-in-the-middle forgery attack

• Pretend to Alice that she is Bob, and perform a DH key exchange
• Pretend to Bob that he is Alice and perform a DH key exchange

Solving man-in-the-middle forgery attacks: identity verification

8. ECDH protocol in TLS1.2 and TLS1.3

The ECDH protocol is a combination of ECC algorithm and DH for key negotiation (both parties can negotiate a key without sharing any secrets). This key exchange algorithm is called ECDH. The ECDH protocol uses the ECC algorithm to calculate faster than the DH key exchange protocol

TLS1.2 communication process:

1. Client tells Server which security suites it supports
2. Server selects the security suite and informs the Client
3. Server sends a certificate for Client to verify identity
4. Pass the public key generated by the server to the client
5. Tell the server that the above steps are completed
6. The client generates the corresponding public key and sends it to the server
7. Server and Client generate keys based on their private key and the public key provided by the other party, and the keys generated on both sides are the same
8. Encrypted communication based on the generated key

Support for TLS1.3 in openssl 1.1.1 version :

Ciphersuites security suite

• TLS13-AES-256-GCM-SHA384
• TLS13-CHACHA20-POLY1305-SHA256
• TLS13-AES-128-GCM-SHA256
• TLS13-AES-128-CCM-8-SHA256
• TLS13-AES-128-CCM-SHA256

In TLS1.3, the number of supported security suites is greatly reduced, especially some insecure security suite algorithms are cancelled.

Client Hello and Server Hello choose the security suite to cause TLS1.2 to have an unnecessary delay of RTT

In TLS1.3, the client generates a public and private key in advance before the handshake, and sends the public key to the server during the first handshake. The server also generates the public and private key. The client and the server use their own private key and the public key provided by the other party. Generate the key, and then start communication

TLS1.3 supports 5 kinds of encryption suites, so which one is used for communication between Client and Server ?

Client will generate all the public and private keys of these 5 types of encryption suites, and then send all the 5 types of public keys to the Server. The server will select the encryption suite and inform the Client of the public key selected by itself.

9. Optimization of handshake: session cache, ticket and 0-RTT of TLS1.3

1), session cache

After the first handshake, the server will generate a session ID and send it to the client. Within a certain period of time, the client will access the server again with the session ID, and the server will retrieve the encryption key corresponding to the session ID from the cache, and the server will tell the client to use the last session. The encryption key used is sufficient, no need to generate the key again

Require Client and Server to cache session ID and encryption key

Existing problems :

• The server caches the session ID in the memory. When the server has multiple application instances, the stored session ID cannot be shared
• Cache session ID in memory is memory consumption

2）、session ticket

Each server does not need to consume memory to store the session ID, but based on the password shared in the cluster, use the password to encrypt the relevant encrypted information and send it to the client. The client sends the relevant encrypted information to the server during the next handshake. , After decryption, Server gets the key after the last successful handshake

3), 0RTT handshake of TLS1.3

The 0RTT of TLS1.3 means that the relevant data is carried during the handshake, and the response can be received after one RTT. No additional RTT is occupied during the handshake, so it is called 0RTT

0RTT can also be completed in the second handshake. During the first handshake, the Client and Server cache the relevant key information. The second handshake is sent to the Server based on the information cached last time. If the relevant information of the key has not expired, you can decrypt the key and send the corresponding response to the Client

The Client sends a POST request, which uses the key cached last time to encrypt, and finally changes the content in the Server database. If the message is intercepted, the interceptor does not need to decrypt the message, but only needs to resend the message. Can keep changing the contents of the Server database

So 0RTT, session ticket, session ID need to set a reasonable expiration time

10. Summarize the working mode of HTTPS

The following uses the flowchart in "Interesting Network Protocol" to summarize the working mode of HTTPS:

The client sends a Client Hello message to the server to transmit TLS version information, cipher suite candidate list, compression algorithm candidate list and other information in plain text. In addition, there will be a random number, which is used when negotiating a symmetric key

The server returns a Server Hello message, telling the client that the server chooses the protocol version, encryption suite, compression algorithm, etc., and a random number for subsequent key negotiation

Server sends the server-side certificate, and then says: Server Hello Done, I have this information here

The client uses the public key in the CA's certificate to decrypt the server's certificate from the CA warehouse it trusts. If it succeeds, it means that the Server is trusted

After the certificate is verified, it feels that the server is credible, so the client calculates and generates a random number Pre-master, sends the Client Key Exchange, encrypts it with the public key in the certificate, and sends it to the server. The server can decrypt it with the private key

So far, whether it is Client or Server, there are three random numbers: own, peer, and just generated Pre-Master random numbers. Through these three random numbers, the same symmetric key can be generated on the Client and Server

With the symmetric key, the client can say: Change Cipher Spec. In the future, the negotiated communication key and encryption algorithm will be used for encrypted communication.

Then send an Encrypted Handshake Message to encrypt the parameters that have been negotiated, etc., using the negotiated key, and send it to the server for data and handshake verification

Similarly, the Server can also send Change Cipher Spec, saying: No problem, in the future, the negotiated communication key and encryption algorithm will be used for encrypted communication, and also try to send Encrypted Handshake Message. When the handshake between the two parties is over, the encrypted transmission can be carried out through the symmetric key.

Reference :

"Detailed Web Protocol and Packet Capture": https://time.geekbang.org/course/intro/100026801

"HTTPS Protocol: The process of ordering food is so complicated": https://time.geekbang.org/column/article/9492