Day945. Version management, compilation and automated testing of Android system development - system reconstruction practice

Version management, compilation and automated testing of Android system development

Hi, I am 阿昌, what I learned to record today is about Android系统开发的版本管理、编译与自动化测试content.

From application to system development, the amount of code has grown from hundreds of thousands of lines to tens of millions of lines, and the development framework and compilation environment are different from application development. So if you want to learn Android system development , you need to understand the corresponding development framework and tool chain first.

1. Repo & Gerrit code management

Code management . Due to the huge amount of code in the Android source code, multiple are used Git 仓库to manage the code.

You can view the corresponding warehouses through GoogleSource , there are about 3000 warehouses.

If there is a requirement development that involves modification across multiple warehouses, how to maintain code submission and synchronize work?

In order to solve this problem, the official provides a multi-Git warehouse code management tool - Repo .

According to the official website, Repo will not replace Git, the purpose is to help use Git more easily in the Android environment.

Repo summarizes multiple Git projects into one Manifest file, and it is very convenient to use the Repo command to concurrently operate code submission and code synchronization of multiple Git repositories.

---

Some commonly used commands in Repo are sorted out in a table:

The basic workflow of using Repo for Android development is shown in the figure below, including creating a branch, modifying files, submitting staging, submitting changes, and finally uploading to the review server.

In addition, under normal circumstances, the code review tools used for application development are similar to the GitLab platform, which pulls the Merge Request through the temporary branch to submit the code review, and after the review is passed, the code is stored in the warehouse. For code review of multiple warehouses, another code review tool— Gerrit is officially provided .

Gerrit is a web-based code review system for projects using Git. Google originally designed Gerrit for managing Android projects.

The difference from the code review of Merge Request is that Gerrit uses +1 +2 scoring to control the integration of code, which can be understood in combination with the following screenshots.

So how does Gerrit associate the code submission records of multiple warehouses?

This is also different from tools such as GitLab. Gerrit provides a standard "commit-msg" hook to generate Change-Id.

Through Change-Id, it can be associated with the submission of multiple code warehouses, which is convenient for managing code submissions across warehouses.

---

2. Soong Compilation System

As can be seen from the code warehouse management, the Android system has nearly 3,000 warehouses. How to manage the compilation and construction of so many codes and finally generate the img image is naturally a very complicated problem. To this end, Google introduced the Sooong build system in Android 7.0 (Nougat), designed to replace Make.

It leverages the Kati GNU Make clone tool and Ninja build system components to speed up Android builds.

---

A schematic diagram to sort out the relationship between Make, Kati, Soong, Ninja and other tools.

The difference between Android.bp and Ninja is that the target object of Android.bp is developers, and developers write scripts based on the syntax rules of bp. The goal of Ninja is to become an assembler. By organizing compilation tasks in parallel, the construction speed is greatly improved. .

As can be seen from the figure above, Soong defines and describes the construction of a module through the Android.bp file. Android.bp files are simple, they do not contain any conditional statements nor control flow statements.

---

Take the desktop Android.bp file as an example to show you the basic bp grammar rules, the code is as follows.

//模块类型，定义构建产物的类型，例如这里的android_app就是定义生成APK类型
android_app {
    
    
    //应用名称
    name: "Launcher3",
    //编译所依赖的静态库
    static_libs: [
        "Launcher3CommonDepsLib",
    ],
    //编译源码路径
    srcs: [
        "src/**/*.java",
        "src/**/*.kt",
        "src_shortcuts_overrides/**/*.java",
        "src_shortcuts_overrides/**/*.kt",
        "src_ui_overrides/**/*.java",
        "src_ui_overrides/**/*.kt",
        "ext_tests/src/**/*.java",
        "ext_tests/src/**/*.kt",
    ],
    //编译资源路径
    resource_dirs: [
        "ext_tests/res",
    ],
    //配置混淆
    optimize: {
    
    
        proguard_flags_files: ["proguard.flags"],
        // Proguard is disable for testing. Derivarive prjects to keep proguard enabled
        enabled: false,
    },
    //配置编译相关的SDK版本号
    sdk_version: "current",
    min_sdk_version: min_launcher3_sdk_version,
    target_sdk_version: "current",
    //... ...
}

So how to trigger the execution of compilation?

Soong supports whole machine compilation and specified module compilation.

Let's take a look at how to complete the compilation of the whole machine.

After downloading the entire AOSP source code, enter the root directory of AOSP, and enter the following command to initialize the compilation environment.

source build/envsetup.sh

Next, you need to select the target to be built through the lunch command. Lunch is a command defined in envsetup.sh, which is used to allow the user to select the compilation target. As shown in the figure below, after selecting the corresponding build target, it can be triggered by the m command compile.

What if you only need to compile the desktop app?

As mentioned earlier, Soong also supports compiling a single module, we can trigger the compilation through the command to compile a single module, the code is as follows.

// 进入桌面应用所在的目录
cd packages/apps/Launcher3
// 编译当前目录下的模块，不编译依赖模块
mm

In addition, the latter two methods can also trigger an independent compilation of a module.

• mma - Build all modules and their dependencies in the current directory.
• mmma - builds all modules and their dependencies in the provided directories.

In particular, Google plans to migrate the Android build system to Bazel over a period of several years.

Bazel will replace all existing build systems and build configuration systems in AOSP (Make, Kati, Soong, Make-based product configuration).

For this obvious technical trend, if the team has the conditions, it can be considered to prepare in advance.

---

3. Automated testing

Most manufacturers will extend custom code based on the Android system. In order to ensure that the extended code will not affect the functions of the original system framework and meet the compatibility requirements, Google provides CTS and VTS test suites.

• CTS (Compatibility Test Suite) Chinese is a compatibility test suite, mainly used to test the compatibility of App and framework.

• VTS (Vendor Test Suite) is a vendor test suite in Chinese, which mainly automatically executes HAL and operating system kernel tests, as shown in the figure below.

Taking CTS as an example, the latest Android 13 CTS has about 1068 test modules and about 2.69 million test cases.

It can be seen from this that Google's investment in automated testing is still very large, which also reflects the importance of automated testing.

The code of CTS is in the cts directory of the AOSP source code. If you are interested, you can learn about the official test design and writing. The tests in CTS mainly use Instrumentation and Junit Test, which is similar to the application test writing described above.

Using Gradle in application development, tests can be triggered by commands such as testDUT and testCAT.

---

As mentioned earlier, the compilation system of the Android source code uses Soong, so if a module in the Android system is added with a test, how should the test be executed?

First, the android.bp configuration file for the test module can be defined.

Here also take the desktop application as an example, the bp configuration file code is as follows.

//配置文件模块为androidTest
android_test {
    
    
//测试模块名称
    name: "Launcher3Tests",
//测试目录
    srcs: [
        ":launcher-tests-src",
    ],
//依赖库
    static_libs: ["Launcher3TestLib"],
    libs: [
        "android.test.base",
        "android.test.runner",
        "android.test.mock",
    ],
//测试Launcher3模块
    instrumentation_for: "Launcher3",
    manifest: "AndroidManifest.xml",
    //... ...
}

Corresponding tests can be written, which is consistent with the way applications are written. Let's take a look at a simple test case in the desktop test module.

@SmallTest
@RunWith(AndroidJUnit4.class)
public class IntSetTest {
    
    
    @Test
    public void shouldBeEmptyInitially() {
    
    
        IntSet set = new IntSet();
        assertThat(set.size()).isEqualTo(0);
    }

    @Test
    public void oneElementSet() {
    
    
        IntSet set = new IntSet();
        set.add(2);
        assertThat(set.size()).isEqualTo(1);
        assertTrue(set.contains(2));
        assertFalse(set.contains(1));
    }
}

The last problem to be solved is how to run these test cases.

Google officially provides Atest, a tool for running tests. Atest is a command-line tool that allows users to build, install and run Android tests locally, and can greatly speed up the speed of re-running tests.

If you need to run the use case of the entire desktop test module, you can directly execute the following command.

atest Launcher3Tests

But if you only want to run a single class in the module, you can use the Module:Class method, the command is as follows.

atest Launcher3Tests：IntSetTest 

For more information about Android system development, if you are interested, you can refer to the Android Open Source Project on the official website . This website is similar to the official website for application development .

---

Four. Summary

Compared with application development, some infrastructure tools for Android system development are more complicated.

In order to solve the problem of multi-warehouse development, Repo and Gerrit tools are officially provided.

• Repo helps us to operate multiple Git repositories in batches, which greatly simplifies the workload of code submission synchronization when we make cross-repository modifications.

• The Gerrit tool also helps us associate the submission records of multiple warehouses in the form of Change-ID, which is convenient for us to do CodeReview.

• In addition, in order to manage the compilation of the whole machine and the compilation of independent modules, Android introduces Soong's compilation system. Soong defines and describes the construction of a module through the Android.bp file, and finally converts it into a Ninja file to compile the final target product.

• Regarding automated testing, Google officially designs compatibility suites such as CTS and VTS to ensure the compatibility and stability of the framework.

• For automated testing of a single module, the Atest test suite is also provided to help quickly execute tests within the module.

The tools and methods used in application development and system development are summarized into a table for review reference.

---