[Android] Multi-channel packaging and signing mechanism

[Android] Multi-channel packaging and signing mechanism

multi-channel packaging

When we release an APP, we often need to generate multiple channel packages for uploading to different application markets.

And each channel package can contain its own channel information. When the APP interacts with the background or reports data, it can bring its own channel information. In this way, we can count the sales of each distribution market. Key information such as the number of downloads and the number of users.

Multi-Channel Packaging Approach

ProductFlavor

As Android developers, we must know that Android provides us with such a Gradle plugin library by default:

classpath "com.android.tools.build:gradle:4.0.1"

Then we will introduce specific plugins in build.gradle in the module, such as:

apply plugin: 'com.android.application'

And ProductFlavor is a configuration in this plugin, or an API provided by this plugin.

So how to use ProductFlavor for multi-channel packaging?

In fact, it is very simple, we only need to write the following code in the build.gradle in the module

apply plugin: 'com.android.application'
android {
    
    
    // ...
    flavorDimensions "default"
    file("channel.txt").readLines().each {
    
     channel ->
        // 基于channel，使用productFlavors.create方法，创建一个productFlavor（变种）
        productFlavors.create(channel, {
    
    
            dimension "default"
            // 填充AndroidManifest中的my_channel的值
            manifestPlaceholders = [my_channel: channel]
        })
    }
}
// ...

The above code is groovy code, let me explain its general meaning:

1. First use the relative path to find the channel.txt file in the same directory as build.gradle

2. Read the content of each line in the channel.txt file and use the channel variable to represent it

3. productFlavors.createCreate a productFlavor (also called a variant) based on the channel variable and usage method

4. For each variant of AndroidManifest, fill in the value of my_channel in AndroidManifest

In order to fill the value of my_channel in AndroidManifest, we also need to write the following code in AndroidManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.ycx.demo">

    <application
        // ...
        <meta-data
            android:name="CHANNEL_VALUE"
            android:value="${my_channel}" />
    </application>

</manifest>

In this way, every time the app is built, the my_channel in the AndroidManifest will be replaced with the channel value of the corresponding variant.

And channel.txtthe file, we can write like this:

Through the above operations, our app has the ability to generate multi-channel packages. When we click the Build Variants button of AS, we will see the following variant information:

Visible, there are 6 variants available. But channel.txtthe file clearly only wrote 3, why are there 6 variants?

That's because by default, our app will contain two variants by default, Debug and Release ,

So we have productFlavors.createcreated N variants, then our application will actually have N * 2a variant (N Debug variants + N Release variants)

Use channel information

Now that multiple channel packages can be packaged, let's use the channel information of each channel below!

When we select the variant of xiaomiDebug for packaging, the packaged APK AndroidManifest.xmlwill look like this:

It can be seen that it has generated a channel information, namely xiaomi.

Now that AndroidManifest.xmlthe channel information can already be obtained, it is easy to obtain the channel information in the java code.

With this channel information, we can report this information to the business background, so that we can count key information such as the number of downloads in each application market.

How to Optimize Multi-Channel Packaging

Using the official Product Flavor for multi-channel packaging is actually quite time-consuming, because for each channel package, a complete APK packaging process is required . If one package takes 5 minutes, then 100 packages will take 500 minutes. , which is a bit exaggerated.

So is there a better multi-channel packaging solution?

I am very sure of that. Plug-ins such as Meituan's Wall and Tencent's VasDolly are good choices.

And these plug-ins do not need to package all the channel packages once when packaging. Their general principle is to add a file recording channel information to the signature folder in the APK, so as to inject the channel information into the apk ( V1 signature).

And if we want to modify the files in the APK, we must prevent the signature of the APK from being broken. Once we break the signature of the APK, the APK will not be able to be installed.

Therefore, the time-consuming optimization of multi-channel packaging is transformed into the following problems:

如何在不破坏APK签名的前提下，往APK中注入渠道信息？

Before talking about multi-channel optimization, we need to have a deep understanding of Android's signature mechanism.

Android's signature mechanism

Why does Android need signatures?

If you understand the communication process of HTTPS, you should know that when communicating messages, two problems must be solved:

• Ensure the authenticity of sources
• Ensure messages cannot be tampered with by third parties

When installing the APK, it is also necessary to ensure the authenticity of the source of the APK and that the APK has not been tampered with by a third party.

To understand how signatures are implemented in Android, you need to understand several concepts: message digests, digital signatures, and digital certificates.

message digest

The message digest is to execute a one-way Hash function on the message data to generate a fixed-length Hash value , which is the message digest.

The message digest has the following characteristics. No matter how long the input message data is, the length of the calculated message digest is always fixed . In addition, the message digest function is a one-way function , that is to say, it can only perform forward information digest. Unable to recover any messages from the digest. In general, no two messages appear, and their corresponding message digests are the same.

It is precisely because of the above characteristics of the message digest that the message digest algorithm is widely used in 数字签名the field .

Common message digest algorithms include MD5algorithm , SHA-1, SHA-256and so on. The default algorithm currently used by Android signatures is SHA-256.

Note that the message digest can only guarantee the integrity of the message, and cannot guarantee the immutability of the message .

So, how to ensure that the message cannot be tampered with?

This is another concept to talk about - digital signature .

Before talking about digital signatures, we need to briefly understand several related concepts: 公钥密码体制, 对称加密算法, 非对称加密算法.

public key cryptography

The public key cryptosystem is divided into three parts, the public key, the private key, and the encryption and decryption algorithm. Its encryption and decryption process is as follows:

• Encryption: Encrypt the plaintext by sum 加密算法and get the ciphertext公钥
• Decryption: Decrypt the ciphertext by sum 解密算法and get the plaintext私钥

That is to say, the content encrypted by the public key can only be decrypted by the private key. If the private key is not known, the ciphertext cannot be decrypted.

In actual use, people in need will generate a pair of public key and private key, publish the public key for others to use, and keep the private key for themselves.

Symmetric encryption algorithm

In symmetric encryption algorithms, the same key is used for both encryption and decryption.

Therefore, if the security of the symmetric encryption algorithm is to be ensured, the key must be kept secret, and it can only be known by the user and cannot be disclosed to the outside world.

Asymmetric encryption algorithm

In an asymmetric encryption algorithm, the key used for encryption and the key used for decryption are different.

digital signature

When the private key is used for encryption and the public key for decryption, the process is called签名 ,

How is it different from encryption?

Because the public key is public, anyone who holds the public key can decrypt the ciphertext encrypted by the private key, so this process does not guarantee the security of the message, but it can guarantee the accuracy and non-repudiation of the source of the message sex ,

In other words, if a ciphertext can be decrypted normally using the public key, it can be proved that the ciphertext must have been issued by the private key holder , not by other third parties, and the private key holder cannot Denies that he ever posted the message. Hence the process is called 签名.

The conventional signature scheme should be a combination of digest algorithm and digital signature and encryption algorithm. Its process is as follows:

• Digest the data with the digest algorithm to get the digest value
• Then encrypt the digest value with the private key of the source

The message obtained through the above two steps is the digital signature of the original message .

To sum up, digital signature is actually a combination of 非对称加密技术+ 消息摘要技术.

digital certificate

Through digital signature technology, the problem of reliable communication can be solved.

But have you noticed that the digital signature method mentioned above has a prerequisite, that is, the recipient of the message must obtain the correct public key in advance .

If the public key has been tampered with from the beginning, the data decrypted by this public key will not be the data sent by the source.

So how to ensure the security and trustworthiness of the public key? This is solved by digital certificates .

A digital certificate is a digitally 证书授权中心signed file containing information about the owner of the public key and the public key . It contains the following:

• Certificate issuing authority
• Certificate Validity
• Certificate owner's public key
• certificate owner's name
• The signature algorithm used by the certificate
• The digital signature of the certificate issued by the certificate issuer

It can be seen that the digital certificate itself also uses digital signature technology, but the content of the signature is the entire certificate.

Different from ordinary digital signatures, the signer of digital certificates is not just any ordinary organization, but a CA organization.

Generally speaking, the root certificates of these CA institutions have been pre-installed on your device before the device leaves the factory.

Therefore, a digital certificate can ensure that the public key in the certificate is indeed the owner of the certificate, or the certificate can be used to confirm the identity of the other party.

It can be seen that digital certificates are mainly used to solve the problem of safe distribution of public keys.

Android's signature mechanism

If we are familiar with the apk packaging process, we must know that in the penultimate step of the packaging process, the apk will be signed. There are two tools for signing apk in Android: and, their signature algorithms are the same, mainly for jarsignersignature apksigneruse files are different.

• jarsigner: The signature tool that comes with jdk, which can sign the jar and use the keystore file to sign. The generated signature file is named by the alias of the keystore by default
• apksignersdk: A signing tool specifically for Android applications . The generated signature files are named uniformly CERT.

The process of signature verification

Android provides us with three signature versions, namely V1, V2, and V3. Among them, the V2 signature only supports Android 7.0 and later versions, and the V3 signature supports Android 9.0 and above versions.

If an apk is installed on a system above Android 9.0, it will first determine whether the apk uses a V3 signature. If it does not use a V3 signature, it will determine whether it uses a V2 signature. If it still does not use a V2 signature, it will determine whether the apk uses a V1 signature. , then the V1 signature will be used for verification. If the verification fails, the installation will be refused, and if the verification is successful, the installation will be installed.

This is a flowchart of an apk verification signature officially provided by Android:

It can be seen that an apk can use the signatures of three versions at the same time. If the signatures of the three versions are used at the same time, the signature of V3 will be used for verification first.

It should be noted that, in the case of overwriting installation, signature verification only supports upgrading, not downgrading. That is to say, if an APK signed with V1 is installed on the device, it can be overwritten with an APK signed with V2, and vice versa.

In Android studio, we can choose to package V2 and V1 signatures, but currently AS does not support V3 signatures.

If we want to use the V3 signature, we can use the command line to operate the apksigner tool to perform the V3 signature, and we will not give specific examples.

V1 signature mechanism

If we use the V1 signature in the apk , we will find that in the apk, there will be an extra META-INFfolder, and META-INFthere will be three sub-files in the folder:

• MANIFEST.MF
• CERT.SF
• CERT.RSA

These three files are the products of the V1 signature, and I will briefly introduce them below.

1. MANIFEST.MF : apkEach file in the pair uses SHA-256an algorithm to extract the message digest of the file and then performs base64 encoding, and the obtained encoded value SHA1-Digestis written into a block in the MANIFEST.MF file as the value of the attribute, and every A block has an Nameattribute whose value is the path of the file in the APK package, as follows:

2. CERT.SF : Align apkeach file and MANIFEST.MF文件use SHA-256an algorithm to extract the message digest of each file and then base64 encode it. It can be understood that the SF file is actually used for protection MANIFEST.MF文件.

3. CERT.RSA : CERT.SFThe file using the private key to calculate the signature , and then the signature and the digital certificate containing the public key information will be written CERT.RSAinto saved.

In fact, to put it bluntly, it MANIFEST.MF文件is used to protect all the files in the apk, and CERT.S文件it is used for protection MANIFEST.MF文件, CERT.RSAfor protection CERT.SF 文件, such a chain of links to ensure the security of the apk.

Notice,

The V1 signature protects the existing files in the apk from being modified .

In other words, if you decompress the apk, add a file of your own, and then compress it into an apk, then the apk can still be installed successfully, because the newly added file itself is not within the protection scope of the V1 signature.

So, if the file we added is a file related to channel information, is it possible to inject channel information into the APK without destroying the APK signature ?

V2 signature mechanism

In fact, we can find that there are two places where the V1 signature can be improved:

• Signature verification is slow: During the verification process, digest calculations are required for all files in the apk. Signature verification will take a long time on machines with a lot of APK resources and poor performance, resulting in slow installation speed.
• Insufficient integrity guarantee: the META-INF directory is used to store signatures, so the directory itself is not included in the signature verification process

In order to solve these two problems, in the Android 7.0 version, the V2 signature was introduced.

The V1 signature protects the existing files in the ZIP, while the V2 signature protects the byte data of the entire APK .

When a file is added to the apk, the byte data corresponding to the apk will change, so the V2 signature cannot META-INFinject a new file into the folder in the apk like the V1 signature for multi-channel packaging.

So under the V2 signature, how to inject channel information into the APK without destroying the APK signature?

First of all, we need to know that apk is actually a compressed package in zip format, so the data format of apk is the same as the compressed package in zip format.

According to the ZIP file format, insert a signature block area to record the signature information.

Before signing, the ZIP data format is divided into three parts, namely blue, green and purple. After signing, one is inserted between blue and green, as shown in the following figure 签名分块: APK Signing Block:

So, what is the data format of this signature block ?

In the entire signature block, the first size of blockdata block, which occupies 8 bytes, records the length of the entire signature block except itself.

id-valueLook at the blue part again, the key-value pairs are recorded here , and this is the place to record channel information . In the key-value pair, there is an id of 0x7109871a, and its corresponding value is the signature data of the V2 signature.

After all key-value pairs are recorded, there is another size of blockdata block, which also occupies 8 bytes, and it also records the length of the entire signature block except itself.

Finally, there is a magicdata block, also called a magic number, which occupies 16 bytes, which is used to mark the file format,

Some people may ask: Isn't the suffix of the file also able to mark the file format?

In fact, the essence of the suffix of the file is to make it easier for the program to identify the format of the file, so as to choose different processing methods to process the file.

However, there will also be cases where someone manually modifies the suffix of a file, such as a file in jpg format, and manually modifies its suffix to .png. This operation cannot fundamentally modify the file type. In other words, even manually Modified to a .png file, it is still essentially a .jpg file.

What is really used to mark the file format is the magic number.

How to optimize multi-channel packaging under V2 signature?

According to the description on the official website, the content protected by the V2 signature is the three areas of blue, green, and purple, and the data with the id in the second part (signature block) 0x7109871a, while other data in the signature block will not be guaranteed of.

So when we insert an id-value key-value pair into the signature block to record channel information, it will not destroy the signature of the apk.

According to this idea, under the V2 signature, channel information can be injected into the APK without destroying the APK signature.

The process of injecting channel information is as follows:

1. Analyze the apk to determine whether the V2 signature is used, and if so, locate the signature block area corresponding to V2

2. Add the channel information in the signature blockid-value键值对

3. Copy the original apk and modify the signature block data to generate an apk with channel information

V3 signature mechanism

The V3 signature is actually very similar to the V2 signature. It still uses the verification method of checking the entire compressed package.

The difference is that an Attr block is added to the signature block inserted after the V2 signature. In this new block, our previous signature information and new signature information will be recorded.

Use the key wheel scheme to replace and upgrade signatures. This means that as long as the old signing certificate is in hand, we can use it to change the signature in the new APK file.

It can be seen that the scheme of optimizing multi-channel packaging under V3 signature is the same as that of V2 signature.