In-depth analysis of the principle and practice of RSA keys

I. Introduction

After experiencing many darkest moments in your life, you read this article, you will regret and even angry: Why didn't you write this article earlier? !

Your darkest moments include:

1. Your project needs to be connected to the bank, and the other party needs you to provide an encryption certificate. You only have a CET-6 English certificate, and you are not sure whether this meets the needs of the other party. Because you have been unable to provide the correct certificate, the project was postponed, the salary increase plan was ruined, the monthly payment was cut off, and your girlfriend broke up. You feel like your life is over.

2. You have been 2 months old and finally understand the .crt format certificate. Join the new project, the project is undergoing certificate trusteeship transformation. Haha, I will do this, which is to upload the certificate file to the hosting system. You shout to the project team members, let go of those certificates, and let me come! Squeeze in and see that it is an old project. There is no certificate at all. At that time, public and private keys were used. How do public and private keys become certificates... Because you have been unable to provide the correct certificate, the project was postponed , The salary increase plan was ruined, the monthly payment was cut off, and the girlfriend broke up. You feel like your life is over.

3. You have spent 3 months trying to understand the ins and outs of the SSL certificate. Feeling confident about joining a new project, you have told the project manager a painful history of blood and tears, after this battle, you have grown into an expert in safety certification. The project manager was overjoyed. It happened that the project was undergoing data security transformation. The database needed to be enabled with SSL. It was time to come. Don't worry, just provide a few key files before get off work tomorrow. The more tomorrow, half an hour before the end of get off work, you slowly walk to the project manager, "The goods you want have arrived", and three certificates are released. This is the key file, this is the public key file, and this is the certificate file. The project manager nodded and shook his head. What I want is a JKS file. You said it will be available tomorrow. The more tomorrow, half an hour before get off work, you hand over the JKS format file to the project manager. The project manager nodded and shook his head. What about the password? What if there is no password? Because you have been unable to provide the correct certificate, the project was postponed, the salary increase plan was ruined, the monthly payment was cut off, and your girlfriend broke up. You feel like your life is over.

This article will reveal the little-known secrets of RSA key files from the following parts:

• Mathematical foundation of RSA algorithm
• Six-layer model of RSA key system
• RSA tool usage
• RSA key usage scenarios

Note: Although the key and the certificate are not identical in the strict sense, for the convenience of presentation, the term key in this article covers the concepts of public key, private key, certificate, etc., unless otherwise specified.

2. Mathematical basis of RSA algorithm

The RSA algorithm is based on number theory. The complexity of the RSA algorithm is based on the fact that the prime number decomposition of a large number is an NP problem, which is very difficult to crack. Mathematical concepts related to the RSA algorithm:

For any number x, y can be calculated:

Through y, x can be calculated:

In other words, after x generates y through the number pair (m, e), you can restore y to x through the number pair (m, d).

Here, we actually demonstrate the mathematical process of RSA encryption and decryption. Through formula (1), the process of calculating y from x is encryption, and through formula (2), the process of calculating x from y is decryption.

In practical applications, the RSA algorithm uses the public key to encrypt and the private key to decrypt, so the number of pairs (m, e) is the public key, and (m, d) is the private key.

In fact, in order to increase the speed of private key decryption, the private key saves some intermediate results, such as p, q, e, and so on.

Therefore, in practical applications, the public key can be derived from the private key.

Three, RSA key six-layer model

In order to facilitate the understanding of the principle of RSA keys, I creatively invented the concept of the six-layer model of RSA keys. Each layer defines its own responsibilities and boundaries. The lower the level, the more abstract and theoretical the content it represents; the higher the level, the more it represents practical applications.

• Data: The data layer defines the mathematical concepts of RSA keys (m, e, p, q, etc.) or participating entities (subject, issuer, etc.).
• Serialization: The serialization layer defines a method for serializing complex data structures.
• Structure: The structure layer defines the data organization form of RSA keys in different formats.
• Text: The text layer defines the method of converting the binary key into text.
• Presentation: Presentation layer, which defines the presentation form of the text format key.
• Application: The application layer defines various scenarios for the use of RSA keys.

Each layer will be explained in detail below.

3.2 Data layer

From the above, the secret key is a data structure, each structure contains 2 or more members (the public key contains m and e, and the private key contains m, d, e and some other intermediate results). In order to save these data structures in a file, you need to define a format to serialize the secret key.

3.3 Serialization layer

Currently, common formats for defining data structures include text formats such as JSON and XML.

For example, theoretically we can define the public key as a JSON:

JSON format key

{
    "m":"15",
    "e":"3"
}

Or, you can define the private key as an XML:

<?xml>
<key>
    <module>15</module>
    <e>3</e>
    <d>3</d>
    <p>3</p>
    <q>5</q>
<key>

But when RSA was invented, neither of these formats existed yet. Therefore, the scientists chose ASN.1, the more popular syntax format at the time.

3.3.1 ASN.1

The full name of ASN.1 is Abstract Syntax Notation dot one, (the first edition of Abstract Syntax Notation). The number 1 is added to the back of the ASN by ISO in order to maintain the openness of the ASN, allowing the more powerful ASN to be named ASN.2 in the future, but it has not yet appeared.

ASN.1 describes a data format for representing, encoding, transmitting and decoding data. It provides a set of formal formats for describing the structure of objects, regardless of how the language is executed and the specific reference of these data, and it does not need to care about what kind of application it is.

3.3.2 ASN.1 encoding rules

The specific syntax of ASN.1 can refer to Wikipedia (https://zh.wikipedia.org/wiki/ASN.1 ), here is only a brief description.

The data type representation in ASN.1 is in the form of TLV: the first 2 bytes represent the data type, the next 2 bytes represent the byte length, and V represents the specific value. Common basic types include Integer, UTF8String, and composite structures include SEQUENCE, SET. Both the secret key and the certificate are of SEQUENCE type, and the type of SEQUENCE is 0x30, and the length is greater than 127, so the second byte is 0x82. The data represented by ASN.1 encoding is binary data, which is usually converted into a string by BASE64 and stored in a pem file, and after BASE64 encoding, 0x3082 is the string MI, so the first two characters of the secret key stored in all PEM files It is MI.

BER, CER, DER are ASN.1 encoding rules. Among them, DER (Distinguish Encode Rules) is an unambiguous encoding rule to ensure that the serialization results produced by the same data structure are also the same.

ASN.1 only defines the serialization method of abstract data, but the specific encoding needs further definition.

Strictly speaking, ASN.1 is not a format for defining data, but a grammatical standard. According to this standard, various formats can be formulated.

3.4 Structural layer

According to the different purposes of the key file, the following standards define different structures to encode the key data in ASN.1. Generally speaking, different formats of keys imply different structures.

• pkcs#1 is  used to define RSA public key and private key structure
• pkcs#7 is  used to define the certificate chain
• pkcs#8 is  used to define the public and private keys of any algorithm
• pkcs#12 is  used to define the private key certificate
• X.509  defines public key certificates

For the specific differences between these formats, see 3.5.2 below

3.5 Presentation layer

You can see that ASN.1 and its encoding rules (BER, CER, DER) define binary rules, which are also in binary format when stored in a file. Since the email standard at that time did not support the transmission of binary content, if the secret key file is transmitted by email, the binary file needs to be converted into a text file. This is the origin of PEM (Privacy-Enhanced Mail). Therefore, the secret key content saved in the PEM file is the binary content generated by ASN.1 encoding, and then the base64 encoded text.

In addition, in order to facilitate the user to identify the format, a line of text indicating identity is added to the beginning and end of the Chinese file. A PEM file generally contains three parts: the first line of labels, BASE64 encoded text data, and the last line of labels.

-----BEGIN <label>-----
<BASE64 ENCODED DATA>
-----END <label>-----

For different formats, the value of <label> is different.

3.5.2 Summary of PEM file format

3.6 Application layer

In actual use, it is not only necessary to use public and private keys to encrypt and decrypt data, but also to solve key distribution and verification according to different usage scenarios. Section 5 lists some common usage scenarios of RSA keys.

Four, tools

4.1 openssl

Note: The -RSAPublicKey_in and -RSAPublicKey_out options in the following commands need to be supported by openssl1.0 or higher. If an error is reported, please check the openssl version.

4.1.1 Create a key file

# 生成 pkcs#1 格式2048位的私钥
openssl genrsa -out private.pem 2048

#从私钥中提取 pkcs#8 格式公钥
openssl rsa -in private.pem -out public.pem -pubout

#从私钥中提取 pkcs#1 格式公钥
openssl rsa -in private.pem -out public.pem -RSAPublicKey_out

4.1.2 Secret key file format conversion

#pkcs#1 公钥转换成 pkcs#8 公钥
openssl rsa -in public.pem -out public-pkcs8.pem -RSAPublicKey_in

#pkcs#8 公钥转换成 pkcs#1 公钥
openssl rsa -in public-pkcs8.pem -out public-pkcs1.pem -pubin -RSAPublicKey_out

#pkcs#1 私钥转换成 pkcs#8 私钥
openssl pkcs8 -in private.pem -out private-pkcs8.pem -topk8

#pkcs#8 私钥转换成 pkcs#1 私钥
openssl rsa -in private-pkcs8.pem -out private-pkcs1.pem

4.1.3 View key file information

#查看公钥信息
openssl rsa -in public.pem -pubin -text -noout

#查看私钥信息
openssl rsa -in private.pem -text -noout

4.1.4 Certificate

RSA certificate

#从现有私钥创建 CSR 文件
openssl req -key private.pem -out request.csr -new

#从现有 CSR 文件和私钥中创建证书,有效期365天
openssl x509 -req -in request.csr -signkey private.pem -out cert.crt -days 365

#生成全新证书和私钥
openssl req -nodes -newkey rsa:2048 -keyout root.key -out root.crt -x509 -days 365

#通 过 现 有 证 书 和 私 钥 （作 为CA ） 为 其 他 CSR 文 件 签 名
openssl x509 -req -in child.csr -days 365 -CA root.crt -CAkey root.key -set_serial 01 -out child.crt

#查看证书信息
openssl x509 -in child.crt -text -noout

#从证书中提取公钥
openssl x509 -pubkey -noout -in child.crt  > public.pem

4.1.5 JKS

#将CA证书转换成JKS格式
keytool -importcert -alias Cacert -file ca.crt  -keystore truststoremysql.jks -storepass password123

#将client.crt和client.key转换成PKCS#12格式
openssl pkcs12 -export -in client.crt -inkey client.key -name "mysqlclient" -passout pass:mypassword -out client-keystore.p12

#将PKCS#12格式转换成JKS格式
keytool -importkeystore -srckeystore client-keystore.p12 -srcstoretype pkcs12 -srcstorepass mypassword -destkeystore clientstore.jks -deststoretype JKS -deststorepass password456

Five, RSA key usage scenarios

5.1 HTTPS one-way authentication

Since the HTTP protocol is a plaintext transmission, in order to ensure that the HTTP message is not leaked and tampered with, HTTPS encrypts and decrypts the HTTP message through the SSL/TLS protocol.

To put it simply, the HTTPS protocol requires that during the process of establishing a connection between the client and the server, a session key is exchanged first, and then the session key is used to encrypt and decrypt the communication message. The entire communication process is as follows:

1. The server creates the RSA certificate server.crt and private key server.key through the method shown in 4.1.4, and configures them in the WEB server.

2. The client establishes a connection with the server, and the server sends the certificate server.crt to the client.

3. The client verifies the server certificate and randomly generates a session key, encrypts the session key with the server certificate, and transmits it to the server.

4. The server decrypts the encrypted session key through server.key to obtain the original text of the session key.

5. The client encrypts the HTTP message with the session key and transmits it to the server.

6. The server decrypts the HTTP encrypted message using the session key to obtain the original text of the HTTP message.

7. The server encrypts the HTTP response message with the session key and returns it to the client.

8. The client uses the session key to decrypt the HTTP response message to obtain the original text of the HTTP response message.

(Figure 1. HTTPS one-way authentication)

5.2 HTTPS mutual authentication

The HTTPS scenario described in Section 5.1 is a general scenario. The entire process is only the client verifies the server, that is, after the client obtains the server certificate, it will verify the validity of the certificate, such as whether it is signed by a CA and whether it is still within the validity period. Wait inside. This one-way verification is not a problem in scenarios such as browser access, because this service is designed to provide services to tens of thousands of users. However, in some scenarios, such as only providing services to specific companies and merchants, the server needs to verify the client, and the trusted client that passes the verification can be normal.

When accessing the server, HTTPS two-way authentication is required.

The process of HTTPS two-way authentication is to enhance the authentication of the server to the client on the basis of HTTPS one-way authentication. The idea of ​​the solution is that the client saves the client certificate client.crt, but the client certificate is not signed by the client itself or signed by the CA, but signed by the root.key of the server. During the HTTPS two-way authentication process, the client needs to send the client certificate client.crt to the server, and the server uses root.key to verify that it is correct before proceeding with subsequent communication; otherwise, the client is an untrusted client. The server refuses to provide follow-up services.

The specific communication process is as follows:

1. The server creates the RSA certificate server.crt and private key server.key through the method shown in 4.1.4, and configures them in the WEB server.

2. The client establishes a connection with the server, and the server sends the certificate server.crt to the client.

3. The client verifies the server certificate, and the subsequent process continues after the verification is passed; the connection is disconnected if the verification fails, and the process ends.

4. The server sends a message to the client requesting the client to send a client certificate.

5. The client sends the client certificate to the server.

6. The server verifies the client certificate through root.key, and the verification is correct for the subsequent process; otherwise, the connection is disconnected and the process ends.

7. The client randomly generates a session key, encrypts the session key with the server certificate, and transmits it to the server.

8. The server decrypts the encrypted session key through server.key to obtain the original text of the session key.

9. The client encrypts the HTTP message with the session key and transmits it to the server.

10. The server decrypts the HTTP encrypted message using the session key to obtain the original text of the HTTP message.

11. The server encrypts the HTTP response message with the session key and returns it to the client.

12. The client uses the session key to decrypt the HTTP response message to obtain the original text of the HTTP response message.

It can be seen that, compared to the HTTPS one-way authentication process, the HTTPS two-way authentication process will increase the client to send the client certificate client to the server after the client verifies the server certificate and before sending the encrypted session key to the server. .crt, the process of verifying the certificate by the server.

(Figure 2. HTTPS two-way authentication)

5.3 MySQL enable SSL

The principle that MySQL provides SSL is similar to HTTPS. The difference is that the service provided by MySQL will not target thousands of ordinary users, so the demand for CA is not high.

Therefore, the actual CA certificate is usually generated by the server itself.

Similar to HTTPS, MySQL provides two forms of SSL authentication mechanisms: one-way authentication and two-way authentication.

5.3.1 MySQL one-way SSL authentication

(1) Server configuration files: ca.crt, server.crt, server.key, where server.crt is generated by the signature of ca.crt.

(2) Client configuration files: ca.crt, ca.crt is the same as ca.crt on the server side.

(3) Client generates JKS file

keytool -importcert -alias Cacert -file ca.crt -keystore truststoremysql.jks -storepass password123

(4) Configure SSL options and JKS files through jdbc string

verifyServerCertificate=true&useSSL=true&requireSSL=true&trustCertificateKeyStoreUrl=file:./truststoremysql.jks&trustCertificateKeyStorePassword=password123

5.3.2 MySQL two-way SSL authentication

(1) Server configuration files: ca.crt, server.crt, server.key, where server.crt is generated by ca.crt signature.

(2) Client configuration files: ca.crt, client.crt, client.key, where ca.crt is the same as ca.crt on the server side, and client.crt is generated by the signature of ca.crt.

(3) The client generates the trustKeyStore file

keytool -importcert -alias Cacert -file ca.crt -keystore truststore.jks -storepass password123

(4) The client generates the clientKeyStore file

keytool -importcert -alias Cacert -file ca.crt -keystore clientstore.jks -storepass password456

(5) Configure SSL options and JKS files through jdbc string

verifyServerCertificate=true&useSSL=true&requireSSL=true&trustCertificateKeyStoreUrl=file:./truststore.jks&trustCertificateKeyStorePassword=password123&clientCertificateKeyStoreUrl=file:./clientstore.jks&clientCertificateKeyStorePassword=password456

For more details about MySQL's SSL certification, please refer to:

• https://www.howtoforge.com/tutorial/how-to-enable-ssl-and-remote-connections-for-mysql-on-centos-7/

• https://www.cnblogs.com/ccgood/p/13034426.html

Appendix A ASN.1 encoding in different formats

A.1 pkcs#1

A.1.1 Public key

RSAPublicKey ::= SEQUENCE {
    modulus INTEGER , -- n
    publicExponent INTEGER -- e
}

A.1.2 Private key

RSAPrivateKey ::= SEQUENCE {
    version Version ,
    modulus INTEGER , -- n
    publicExponent INTEGER , -- e
    privateExponent INTEGER , -- d
    prime1 INTEGER , -- p
    prime2 INTEGER , -- q
    exponent1 INTEGER , -- d mod (p-1)
    exponent2 INTEGER , -- d mod (q-1)
    coefficient INTEGER , -- (inverse of q) mod p
    otherPrimeInfos OtherPrimeInfos OPTIONAL
}

A.2 pkcs#8

A.2.1 pkcs#8 public key

PublicKeyInfo ::= SEQUENCE {
    algorithm AlgorithmIdentifier ,
    PublicKey BIT STRING
}
AlgorithmIdentifier ::= SEQUENCE {
    algorithm OBJECT IDENTIFIER ,
    parameters ANY DEFINED BY algorithm OPTIONAL
}

A.2.2 pkcs#8 private key

OneAsymmetricKey ::= SEQUENCE {
    version Version ,
    privateKeyAlgorithm PrivateKeyAlgorithmIdentifier ,
    privateKey PrivateKey ,
    attributes [0] Attributes OPTIONAL ,
    ...,
    [[2: publicKey [1] PublicKey OPTIONAL ]],
    ...
}
PrivateKey ::= OCTET STRING
    -- Content varies based on type of key. The
    -- algorithm identifier dictates the format of
    -- the key.

A.3 X.509

A.3.1 X.509 certificate

Certificate ::= SEQUENCE {
    tbsCertificate TBSCertificate ,
    signatureAlgorithm AlgorithmIdentifier ,
    signatureValue BIT STRING
}

TBSCertificate ::= SEQUENCE {
    version [0] EXPLICIT Version DEFAULT v1,
    serialNumber CertificateSerialNumber ,
    signature AlgorithmIdentifier ,
    issuer Name,
    validity Validity ,
    subject Name,
    subjectPublicKeyInfo SubjectPublicKeyInfo ,
    issuerUniqueID [1] IMPLICIT UniqueIdentifier OPTIONAL ,
        -- If present , version MUST be v2 or v3
    subjectUniqueID [2] IMPLICIT UniqueIdentifier OPTIONAL ,
        -- If present , version MUST be v2 or v3
     extensions [3] EXPLICIT Extensions OPTIONAL
        -- If present , version MUST be v3
}

Version ::= INTEGER { v1(0), v2(1), v3(2) }

CertificateSerialNumber ::= INTEGER

Validity ::= SEQUENCE {
    notBefore Time,
    notAfter Time
}

Time ::= CHOICE {
    utcTime UTCTime ,
    generalTime GeneralizedTime
}

UniqueIdentifier ::= BIT STRING

SubjectPublicKeyInfo ::= SEQUENCE {
    algorithm AlgorithmIdentifier ,
    subjectPublicKey BIT STRING
}

Extensions ::= SEQUENCE SIZE (1..MAX) OF Extension

Extension ::= SEQUENCE {
    extnID OBJECT IDENTIFIER ,
    critical BOOLEAN DEFAULT FALSE ,
    extnValue OCTET STRING
        -- contains the DER encoding of an ASN.1 value
        -- corresponding to the extension type identified
        -- by extnID
}

Author: Zhu Ran, from Internet technology team vivo