Some summary of update.zip package in OTA upgrade

1. The directory structure of update.zip package
          |----boot.img
          |----system/
          |----recovery/
                `|----recovery-from-boot.p
                ` |----etc/
                        `|----install-recovery.sh
          |---META-INF/
              `|CERT.RSA
              `|CERT.SF
              `|MANIFEST.MF
              `|----com/
                     ` |----google/
                             `|----android/
                                    `|----update-binary
                                    `|----updater-script
                             `|----android/
                                    `|----metadata
2.
boot.img contains kernel and ramdisk, which are used to update the files required by the boot partition.
After the system content is upgraded, it is placed in the system partition of the system. It mainly updates
the recovery in the recovery/ directory of some apk and so libraries. -from-boot.p is the patch of boot.img and recovery.img, which is mainly used to update the recovery partition. The install-recovery.sh in the etc/ directory is the update script
update-binary is a binary file, which is quite For a script interpreter, it can recognize the operation described in updater-script. The name of the file in the specific update package is determined by
      the value of ASSUMED_UPDATE_BINARY_NAME in bootable/recovery/install.c in the source code.
updater-script: This file is A script file that describes the update process in detail. Before that, the script can be written according to the specific situation to meet our specific needs. The name of the file is determined by   
      the value of SCRIPT_NAME in the bootable/recovery/updater/updater.c file in the source code
metadata The file is the metadata that describes the device information and environment variables. It mainly includes some compilation options, signature public key, timestamp, device model and other
uesrdata directories. It is used to update the user data part of the system. This part of the content will be stored after the update. In the data directory of the system, but the data partition is encrypted by default after Android M, and the data partition cannot be mounted during OTA.

The update.zip package needs to be signed after it is generated, otherwise an error message of authentication failure will appear during the upgrade. And the signature uses the same encryption public key as the target version. The specific path of the encryption public key and the three files required for encryption after compiling the Android source code is:
out/host/linux-x86/framework/signapk.jar
build/target/product/security/testkey.x509.pem
build/ target/product/security/testkey.pk8
specific encryption method: $ java -jar out/host/linux-x86/framework/signapk.jar -w build/traget/product/security/tesetkey.x509.pem build/target/ product/security/testkey.pk8 update.zip update_signed.zip
MANIFEST.MF : This manifest file defines data related to the composition structure of the package, similar to the manifest.xml file for Android applications
CERT.RSA: The signature associated with the signature file The program block file, which stores the public signature
CERT.SF used to sign the jar file: this is the signature file of the jar file, where the prefix CERT represents the signer.
During the specific upgrade, the update.zip package will be roughly divided into three when checking. Step: 1. Verify that the SF file matches the RSA file. 2. Check whether MANIFEST.MF is consistent with the digest in the signature file. 3. Verify that the files in the package are consistent with those described in MANIFEST.
3. Analysis of the generation process of the Android upgrade package update.zip
Use make otapackage command to generate the process analysis of update.zip.
            Executing the make otapackage command in the source root directory to generate the update.zip package is mainly divided into two steps. The first step is to compile and generate an update original package (zip format) according to the Makefile. The second step is to run a python script with the zip package prepared in the previous step as input, and finally generate the upgrade package we need. These two processes are further analyzed below. (To put it simply, first generate the cota package, and then use the cota package to generate the differential package and the entire package) The
            first step: Compile the Makefile. The location of the corresponding Makefile file: build/core/Makefile. A zip package will be generated from this file, which will eventually be used to make an OTA package or filesystem image.
According to the Makefile, the generation process of this package can be analyzed:
            first, create a root_zip root directory, and then create the following other directories in this directory in turn. ①Create
            a RECOVERY directory and fill in the contents of this directory, including the kernel image and recovery root file. Mirror of the system. This directory is ultimately used to generate recovery.img.
            ②Create and populate the BOOT directory. Contains kernel and cmdline and pagesize size, etc., this directory is finally used to generate boot.img.
            ③ Fill the system image to the SYSTEM directory.
            ④ Fill data image to DATA.
            ⑤ Additional content required for generating the OTA package. It mainly includes some bin commands.
            ⑥Create a META directory and add some text files to the directory, such as apkcerts.txt (describe the authentication certificate used in the apk file), misc_info.txt (describe the block size of Flash memory and partitions such as boot, recovery, system, userdata, etc.) size information).
            ⑦ Compress the root_zip directory we got above using the keep connection option.
            ⑧ Use fs_config (build/tools/fs_config) to configure the permission owner and other information of all system files (directories and files under system/) in the above zip package. fs_config includes a header file #include "private/android_filesystem_config.h". In this header file, the permissions and owners of each file in the system directory are hard-coded. After the configuration is executed, the configured information will be output to META/filesystem_config.txt in text mode. and zip again into the original package we will eventually need.
Step 2: The zip package above is just a raw package generated during compilation. This original zip package has two functions in the actual compilation process, one is to generate an OTA update upgrade package, and the other is to generate a system image. During the compilation process, if the OTA update upgrade package is generated, a Python script named ota_from_target_files will be called (specifically located in lines 1037 to 1058 of the Makefile), located in /build/tools/releasetools/ota_from_target_files. The role of this script is to take the original zip package generated in the first step as input, and finally generate a usable OTA upgrade zip package.
Usage---Usage: ota_from_target_files [flags] input_target_files output_ota_package
                        -b Obsolete.
                        -k key used for signing
                        -i This option is used when generating incremental OTA packages. We will use this option later to generate OTA incremental packages.
                        -w Whether to clear the userdata partition
                        -n Whether to not check the timestamp when upgrading, the default is to check, that is, by default, it can only be upgraded based on the old version.
                        -e Whether there are additional scripts to run
                        -m The format required to generate the script (updater-script) during the execution process, there are currently two types: amend and edify. Different interpreters will be used when upgrading to the previous two versions.
        By default , scripts in both formats are generated at the same time.
                        -p defines the paths to some executable files used by the script.
                        -s defines paths to additionally run scripts.
                        -x defines possible key-value pairs for additional scripts to be run.
                        -v Print out the executed command during execution.
                        -h command help
Let 's analyze how the ota_from_target_files python script generates the final zip package.
The main function main is the entry function of python. Let's start with the main function, and roughly look at the process in the main function (the last of the script) to know the execution process of the script.
                       ① At the beginning of the main function, first store the option option set by the user into the OPTIONS variable, which is a class in python. Then judge whether there are additional scripts, and if so, read them into the OPTIONS variable.
                       ② Unzip the input zip package, which is the original zip package we generated above. Then determine whether to use device-specific extensions (device extensions), and if so, read them into the OPTIONS variable.
                       ③ Determine whether to sign, and then determine whether there is an incremental source of new content, and if so, decompress the incremental source package and put it into a temporary variable (source_zip). Since then, all preparations have been completed, and then the main function in the script, WriteFullOTAPackage(input_zip, output_zip) will be called.
                       ④ The processing process of the WriteFullOTAPackage function is to obtain the generator of the script first. The default format is edify. Then get the metadata metadata, which comes from some Android environment variables. Then get the device configuration parameters such as the version of the api function. Then determine whether to ignore the timestamp.
                       ⑤ After the WriteFullOTAPackage function completes the preparations, it starts to generate the updater-script for the upgrade. After generating the script file, write the metadata obtained in the previous step to the output package out_zip.
                       ⑥At this point, a complete update.zip upgrade package is generated. Copy the upgrade package to the SD card and it can be used for upgrade.
Fourth, the generation of Android OTA incremental package update.zip The update.zip upgrade package generated
in the above process is the upgrade package of the entire system, and in the actual upgrade, we only hope to be able to upgrade the part of the content we changed. This requires the use of incremental packages to upgrade. The process of generating incremental packages also requires the participation of the ota_from_target_files.py mentioned above.
         Below is the process of making the update.zip incremental package.
          ① Execute the following commands in sequence in the source root directory
           $ .build/envsetup.sh
           $ lunch XXX
           $ make
           $ make otapackage After
           executing the above command, our first system upgrade package will be generated under out/target/product/tcc8800/. We first name it A.zip
          ② Modify the parts we need to change in the source code, such as modifying the kernel configuration, adding new drivers, and so on. Execute the above command again after modification. The second update.zip upgrade package generated after our modification will be generated. Name it B.zip.
          ③ We saw the help of the ota_from_target_files.py script above, where the option -i is used to generate differential incremental packages. The usage method is to take the above A.zip and B.zip packages as input and update.zip package as output. The generated update.zip is the last incremental package we need.
              The specific usage method is: copy the above two packages to the root directory of the source code, and then execute the following commands.
              $ ./build/tools/releasetools/ota_from_target_files -i A.zip B.zip update.zip. (You can also add other parameters, such as -x, -k, -p, etc.)
              When executing the above command, there will be an error that recovery_api_version is not found. The reason is that when the above script is executed, if option i is used, WriteIncrementalOTAPackage will be called, and misc_info.txt will be searched from the META directory in package A and package B to read the value of recovery_api_version. However, there is no such directory in the update.zip package generated when the make otapackage command is executed, and there is no such document.
              At this point we need to use the original zip package generated by executing make otapackage. The location of this package is in the out/target/product/XX/obj/PACKAGING/target_files_intermediates/ directory (cota package). It is an intermediate product after using the command make otapackage and is the most original upgrade package. We rename the two compiled packages to A.zip and B.zip respectively, and copy them to the root directory of the SD card and execute the above command repeatedly:
               $ ./build/tools/releasetools/ota_form_target_files -i A.zip B.zip update.zip.
             
In addition:
the call to generate the differential package is the WriteIncrementalOTA method in the file ./build/tools/releasetools/ota_from_target_files. When calling, the two versions of the differential resource package need to be passed in as parameters, such as:
./build/tools/releasetools/ ota_from_target_files –n –i ota_v1.zip ota_v2.zip update.zip
Among them, the parameter n means to ignore the timestamp; i means to generate incremental packets (ie differential packets); ota_v1.zip and ota_v2.zip respectively represent the difference between the two versions before and after Resource package; and update.zip represents the final generated differential package.
The WriteIncrementalOTA function will calculate the difference between the two input differential resource packages and write them into the differential package; at the same time, add the updater and the generated script file udpate-script to the upgrade package.