[Android] make and Soong of the compilation system

Introduction to make

make is an automated build tool, make automatically builds the source code into executable programs and library files by Makefilereading the files.

The Makefile defines the dependencies of the target program and the relevant rules for generating the target program.

In the early days, make was included in Unixthe system , and as GNU/Linuxit was Unixderived and carried forward, GNU/Linuxit retained and expanded the original make, adding many built-in functions and automatic variables, etc., and formed GNU make.

In the early days, make was mainly used to build projects developed in C language, and later gradually developed and widely used to build projects developed in various languages such as C, C++, and Java.

The system source code of Android 6.0 and below is built Makefile（Android.mk）using , and later in Android 8.0 and later, Google tried its best to promote it Android.bp, but in fact there are still many files Android9.0of each original factory.Android10.0Android.mk

Commonly used build tools in Java projects are ant, maven, gradle, and they all have their own command tools, build rules, and configuration files, for example,

The command tool of ant is ant, and the configuration file is an xml file
The command tool of gradle is gradlew, and the configuration file isbuild.gradle

Similarly, makeas the grandfather of automated construction, it also has its own command tools, construction rules, and configuration files.

The command tool of make is make, and the configuration file isMakefile

In the Makefile file, the construction rules of the project are described, and the rules are interpreted makeby . When make executes, it needs to read at least one Makefile.

Makefile composition

A complete Makefile generally contains 4 elements:

Indicators : Indicators include a series of keywords, built-in functions, automation variables, and environment variables . Indicators indicate an action to be performed when the make program reads the makefile
Rules : It describes how to update one or more Makefiles under what circumstances
Variable : Use a variable name to represent a variable value. After a variable is defined, the Makefile can use this variable name in the place where it needs to use the variable value.
Note : The content after “#”the character is regarded as the content of the comment

For example, keywords in Makefile are as follows:

define : used to define variables
endef : define the terminator of the variable, generally speaking, define and endef are used in pairs
ifdef : Determine whether the variable has been defined
ifndef : Determine whether the variable is undefined
ifeq : Determines whether two variables are equal
ifneq : checks if two variables are not equal
else : branch processing of conditional statements
endif : the terminator of the conditional statement
include : used to include other Makefiles
sinclude : Equivalent to include (for compatibility with non-GNU make)
override : used to overload variables
export : Add a variable and its value to the environment variable of the current job
unexport : opposite to export

There are also many built-in functions, environment variables, and automation variables in the Makefile, and the space is limited, so I will not give examples here.

Let's focus on the application of Makefile in Android, Android.mkwhich is .

The role of Android.mk

Android.mk is a kind of Makefilefile , which is GUN makefilea part of and will be parsed one or more times by the compilation system, so we should declare variables Android.mkin and don't assume that anything will not be defined during the parsing process.

The Android.mk file mainly tells the compilation system what rules to use to compile our source code and generate the corresponding target files. The target files can be divided into the following types:

apk file, a normal Android application
jar file, java class library
c\c++ application, executable c\c++ application
c\c++ static library, packaged into .aa file
c\c++ dynamic library, packaged into .soa file

Note: Only dynamic libraries can be installed or copied to apk, while static libraries can be linked into dynamic libraries.

It is used to specify such as compiled and generated so library name, referenced header file directory, .cor .cppfiles and .astatic library files that need to be compiled.

Android.mk file syntax allows us to sourcepackage into one modules, and this modulescan be a static library or a dynamic library . We can define one or more Android.mkin modulesor sourceadd to multiple modules.

The Build System handles a lot of details for us and we don't need to care about it anymore, for example: we don't need to list header files and external dependency files Android.mkin .

Android.mk case study

Let's look at the Android.mksimplest :

# 每个 Android.mk 文件必须以定义 LOCAL_PATH 为开始，它用于在开发 tree 中查找源文件
# 宏 my-dir 则由 Build System 提供，返回包含 Android.mk 的目录路径
LOCAL_PATH := $(call my-dir)

# CLEAR_VARS 变量由 Build System 提供，并指向一个指定的 GNU Makefile，
# 它负责清理很多 LOCAL_xxx，例如：LOCAL_MODULE, LOCAL_SRC_FILES, LOCAL_STATIC_LIBRARIES 等等。但不清理LOCAL_PATH，
# 这个清理动作是必须的，因为所有的编译控制文件由同一个 GNU Makefile 解析和执行，其变量是全局的，所以清理后才能避免相互影响
include $(CLEAR_VARS)

# LOCAL_MODULE 模块必须定义，以表示 Android.mk 中的每个模块，名字必须唯一且不包含空格
# Build System 会自动添加适当的前缀和后缀
# 例如，foo，要产生动态库，则生成 libfoo.so，但请注意：如果模块名被定为：libfoo，则生成 libfoo.so. 不再加前缀
LOCAL_MODULE := hello

# LOCAL_SRC_FILES 变量必须包含将要打包如模块的 C/C++ 源码，我们不必列出头文件，build System 会自动帮我们找出依赖文件
# 缺省的 C++ 源码的扩展名为 .cpp，也可以通过 LOCAL_CPP_EXTENSION 修改。
LOCAL_SRC_FILES := hello.c

# BUILD_SHARED_LIBRARY 是 Build System 提供的一个变量，指向一个 GNU Makefile Script
# 它负责收集自从上次调用 include $(CLEAR_VARS) 后的所有 LOCAL_XXX 信息，并决定编译为什么
# BUILD_STATIC_LIBRARY：编译为静态库。
# BUILD_SHARED_LIBRARY：编译为动态库
# BUILD_EXECUTABLE：编译为 Native C 可执行程序
# BUILD_PREBUILT：Android 8.1 以及之后的版本使用
include $(BUILD_SHARED_LIBRARY)

In this example, the purpose is to use Android.mkGenerate sofile, the meaning of each line of code is clearly written in the comment,

Another thing to note is: Android 8.1in , the two macros PREBUILT_STATIC_LIBRARY and PREBUILT_SHARED_LIBRARY have been discarded , BUILD_PREBUILT is used for precompilation, and the compiled file type is specified by LOCAL_MODULE_CLASS . like:

LOCAL_MODULE_CLASS := STATIC_LIBRARIES
# LOCAL_MODULE_CLASS := SHARED_LIBRARIES
# LOCAL_MODULE_CLASS := APPS
include $(BUILD_PREBUILT)

So, how to write Android.mk to compile an apk?

You can refer to the following example:

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Test
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(LOCAL_MODULE).apk
LOCAL_MODULE_CLASS := APPS
LOCAL_MODULE_SUFFIX := $(COMMON_ANDROID_PACKAGE_SUFFIX)
LOCAL_CERTIFICATE := PRESIGNED # LOCAL_CERTIFICATE := platform (使用平台签名)
# 可选项，如果不添加此变量，则预装到 system/app 下，此 apk 将不能被卸载，
# 添加后，被安装到 data/app 目录下，可卸载
LOCAL_MODULE_PATH := $(TARGET_OUT_DATA_APPS)
include $(BUILD_PREBUILT)

If the apk also contains the local so library, it should be written like this:

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Test
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(LOCAL_MODULE).apk
LOCAL_MODULE_CLASS := APPS
LOCAL_MODULE_SUFFIX := $(COMMON_ANDROID_PACKAGE_SUFFIX)
LOCAL_CERTIFICATE := PRESIGNED # LOCAL_CERTIFICATE := platform (使用平台签名)

# 引入本地 so 库
LOCAL_PREBUILT_JNI_LIBS := \
	/lib/so1.so \
	/lib/so2.so \
	/lib/so3.so
LOCAL_DEX_PREOPT := true

# 可选项，如果不添加此变量，则预装到 system/app 下，此 apk 将不能被卸载，
# 添加后，被安装到 data/app 目录下，可卸载
LOCAL_MODULE_PATH := $(TARGET_OUT_DATA_APPS)
include $(BUILD_PREBUILT)

Soong build system

Let's first look at the development and evolution process of Android system compilation and construction :

Prior to Android 7.0 use GNU MakeandAndroid.mk
Android 7.0 introduces ninjaandkati
Android 8.0 Android.bpuses instead Android.mkand introduces the Soongbuild system
Android 9.0 makes it mandatory to useAndroid.bp

What do the nouns mentioned above mean? Here's a brief introduction:

ninja: It is a compilation framework, which will be compiled according to the configuration file in the corresponding ninjaformat . ninjaGenerally, the file will not be modified manually, Android.bpbut ninjacompiled by converting the file into a format file
Android.bp: It appears to replace the Android.mk file, but it does not support conditional statements, so in actual projects, if the built script must contain conditional statements, it is recommended to use Android.mk or use the Go language
Soong: It is a tool designed for Android system compilation. It is similar to the make compilation system. It can be considered that it targets the make compilation system. Soong is mainly responsible for semantically Android.bpanalyzing and converting it into ninjafiles. Soong will also compile and generate A androidmkcommand that Android.mkconverts a file to Android.bpa file
Blueprint: It is a tool for generating and parsing Android.bp, which is part of Soong. Blueprint is only responsible for parsing the file format
Kati: A tool based on Golangand , the main function is to convert files into filesC++Android.mkninja

The conversion relationship between Android.bp, Android.mk, and ninja is as follows:

insert image description here

So why should Google gradually abandon it GNU Makeand use Soongit?

The reason is that using GNU Makecompilation , at the Android level slowly becomes slow, error-prone, non-scalable, and difficult to test, and the Soong build system provides exactly the flexibility Android system builds require.

Introduction to Android.bp

The syntax of Android.bp is simpler than Android.mk in design, but it does not support conditional statements, so in actual projects, if the script to be built must contain conditional statements, it is recommended to use Android.mk or use Go language. A module in an Android.bp file 模块类型begins with , followed by a set of key-value pair attributes: name: value,

Android.bp common module types

In Android.bp, we will build what we need based on the module type. The commonly used types are as follows:

android_app

It is used to build apk, which has the same function as Android.mk BUILD_PACKAGE.

java_library

The file used to build and link the source code into the .jardevice .

By default, java_librarythere is only one variant, which produces a bundle containing .classfiles .jar. The resulting jar is not suitable for direct installation on the device, rather it will be used as a static_libsdependency .

If specified “installable:true” will generate a fileclasses.dex containing the file , suitable for installation on the device. .jarSpecifying 'host_supported:true'will produce two variables, one compiled according to the bootclasspath of the device, and the other compiled according to the bootclasspath of the host.

java_library_static

Equivalent to java_library, but java_library_staticis deprecated and deprecated

android_library

Build and link the source code with the Android resource files into the device's .jarfile .

android_library has a separate variant that produces a file containing .classfiles .jar, and aapt2a .package-res.apkfile containing Android resources compiled using . The generated apk file cannot be installed directly on the device, but can be used as android_appa module 's static_libsdependency.

cc_library

Create static or shared libraries for device or host.

By default, cc_librarythere is a single variant for the device. Specifying host_supported:truealso creates a host-targeted library.

cc_libraryThe related module types include cc_library_shared, cc_library_headers, and cc_library_staticso on.

Here is a simple example:

// 表示该模块用于构建一个 apk
android_app {
    
    
	// 模块都必须具有 name 属性，且值是唯一的
    name: "Test", 
    
    // 以字符串列表的形式指定用于构建模块的源文件
    srcs: [
        "src/**/*.java",
        "com/example/xxx/*.aidl",
    ],
    
    // 引入静态依赖库
    static_libs: [
        "androidx.cardview_cardview",
        "androidx.recyclerview_recyclerview",
        "TestLib",
    ],
    
    // 引入java库
    libs: ["android.car"],
    
    // 指定资源文件的位置
    resource_dirs: ["res"],
    
    // 设定 apk 安装路径为 priv-app
    privileged: true,
    
    // 是否启用代码优化，android_app 中默认为 true，java_library 中默认为 false
    optimize: {
    
    
        enabled: false,
    },
    
    // 是否预先生成 dex 文件，默认为 true。
    // 该属性会影响应用的首次启动速度以及 Android 系统的启动速度
    dex_preopt: {
    
    
        enabled: false,
    },
    
    // 设置该标记后会使用 sdk 的 hide 的 api 來编译，
    // 如果编译的 APK 中需要使用系统级 API，必须设定该值，
    // 和 Android.mk 中的 LOCAL_PRIVATE_PLATFORM_APIS 的作用相同
    platform_apis: true,
    
    // 表示生成的 apk 会被安装在系统的 product 分区，
    // 和 Android.mk 中 LOCAL_PRODUCT_MODULE 作用相同
    product_specific: true,
    
    // 用于指定 APK 的签名方式
    certificate: "platform",
}

// 表示该模块用于构建一个 Lib
android_library {
    
    
    name: "TestLib",
    
    srcs: [
    	"xxx/**/*.java", 
    	"xxx/**/*.kt",
    ],
    
    resource_dirs: ["res"],
    
    // 用于指定 Manifest 文件
    manifest: "AndroidManifest-withoutActivity.xml",
    
    platform_apis: true,
    
    optimize: {
    
    
        enabled: false,
    },
    
    dex_preopt: {
    
    
        enabled: false,
    },
    
    static_libs: [
        "androidx.cardview_cardview",
        "androidx.recyclerview_recyclerview",
        "androidx.palette_palette",
        "car-assist-client-lib",
        "android.car.userlib",
        "androidx-constraintlayout_constraintlayout"
    ],
    
    libs: ["android.car"], 
}

In this example, in Testthe module , TestLibthe module is introduced as a static dependency library. If make Testthe command , Test.apkthe file will be generated.

It should be noted that certificateit is used to specify the signature method of the APK, and there are four signature methods in Android:

testkey: common apk, the signature is used by default
platform: If the apk needs to access the folders existing in the system, etc., or needs to complete some core functions of the system, then use the signature change. The uid of the process where the apk compiled in this way is system
shared: If the apk needs to share data with home/contactsthe process , use this signature
media: If the apk is a part of media/downloadthe system , use this signature