Compilation environment initialization-Android10.0 compilation system (2)

Summary: This section mainly focuses on the environment initialization of the Android10.0 compilation system.

1 Overview

  The previous section gave a general description of the Android compilation system, and this section gives a detailed demonstration of the initialization of the compilation environment.

 

2 Initialization of compilation environment

  Initialization command:

  source build/envsetup.sh

  envsetup.sh mainly does the following things:

  envsetup.sh build code:

  ...
 
  validate_current_shell
 
  source_vendorsetup
 
  addcompletions

2.1 hmm View supported interfaces

  Enter hmm to see some interfaces supported by envsetup

Order	illustrate
lunch	lunch <product_name>-<build_variant> Select <product_name> as the product to build and <build_variant> as the variant to build, and store these selections in the environment for subsequent calls to "m" etc. to read.
tapas	Interaction method: tapas [<App1> <App2> ...] [arm|x86|mips|arm64|x86_64|mips64] [eng|userdebug|user]
croot	Change directories to the top of the tree or its subdirectories.
m	Compile the entire source code without switching to the root directory
mm	Compile the source code in the current directory, excluding their dependent modules
mmm	Compile all modules in the specified directory, excluding their dependent modules. For example: mmm dir/:target1,target2.
mma	Compile the source code in the current directory, including their dependent modules
mmma	Compile all modules in the specified directory, including their dependent modules
provision	Flash device with all required partitions. Options will be passed to fastboot.
cgrip	Execute grep command on all local C/C++ files in the system
ggrep	Execute grep command on all local Gradle files in the system
grab	Execute grep command on all Java files local to the system
grip	Execute the grep command on the xml files in all res directories locally on the system
mangrep	Execute grep command on all AndroidManifest.xml files local to the system
mgr	Execute grep command on all local Makefiles files in the system
sepgrep	Execute the grep command on all sepolicy files local to the system
grip	Execute grep command on all source files local to the system
is raised	Search the entire directory according to the parameter file name after godir, and switch directories
allmod	list all modules
gomod	Go to the directory containing the module
pathmod	Get the directory containing the module
refreshmod	Refresh the module list of allmod/gomod

 

2.2 validate_current_shell

  Determine the current shell environment and establish shell commands

function validate_current_shell() {
    
    
    local current_sh="$(ps -o command -p $$)"
    case "$current_sh" in
        *bash*)
            function check_type() {
    
     type -t "$1"; }
            ;;
        *zsh*)
            function check_type() {
    
     type "$1"; }
            enable_zsh_completion ;;
        *)
            echo -e "WARNING: Only bash and zsh are supported.\nUse of other shell would lead to erroneous results."
            ;;
    esac
}

2.3 source_vendorsetup

  Search vendorsetup.sh from directories such as device\vendor\product and source it.

function source_vendorsetup() {
    
    
    allowed=
    for f in $(find -L device vendor product -maxdepth 4 -name 'allowed-vendorsetup_sh-files' 2>/dev/null | sort); do
        if [ -n "$allowed" ]; then
            echo "More than one 'allowed_vendorsetup_sh-files' file found, not including any vendorsetup.sh files:"
            echo "  $allowed"
            echo "  $f"
            return
        fi
        allowed="$f"
    done
 
    allowed_files=
    [ -n "$allowed" ] && allowed_files=$(cat "$allowed")
    for dir in device vendor product; do
        for f in $(test -d $dir && \
            find -L $dir -maxdepth 4 -name 'vendorsetup.sh' 2>/dev/null | sort); do
 
            if [[ -z "$allowed" || "$allowed_files" =~ $f ]]; then
                echo "including $f"; . "$f"
            else
                echo "ignoring $f, not in $allowed"
            fi
        done
    done
}

example:

1. Create a directory: /vendor/ingres/build

2. Create a vendorsetup.sh

  Write a log: echo "vendor build test."

3. Execute source build/envsetup.sh

Print after source:

including vendor/ingres/build/vendorsetup.sh
vendor build test.

2.4 addcompletions

function addcompletions()
{
    
    
    local T dir f
 
    # Keep us from trying to run in something that's neither bash nor zsh.
        # 检测shell版本字符串BASH_VERSION 或ZSH_VERSION长度为0时，返回
    if [ -z "$BASH_VERSION" -a -z "$ZSH_VERSION" ]; then
        return
    fi
 
    # Keep us from trying to run in bash that's too old.
    # 检测bash主版本低于3时返回
    if [ -n "$BASH_VERSION" -a ${
    
    BASH_VERSINFO[0]} -lt 3 ]; then
        return
    fi
 
        # 指定bash文件目录并检查是否存在
    local completion_files=(
      system/core/adb/adb.bash
      system/core/fastboot/fastboot.bash
      tools/asuite/asuite.sh
    )
    # Completion can be disabled selectively to allow users to use non-standard completion.
    # e.g.
    # ENVSETUP_NO_COMPLETION=adb # -> disable adb completion
    # ENVSETUP_NO_COMPLETION=adb:bit # -> disable adb and bit completion
        #*.bash文件列表，并将这些*.bash文件包含进来
    for f in ${completion_files[*]}; do
        if [ -f "$f" ] && should_add_completion "$f"; then
                 # 对*.bash文件执行'.'操作
            . $f
        fi
    done
 
    if should_add_completion bit ; then
        complete -C "bit --tab" bit
    fi
    if [ -z "$ZSH_VERSION" ]; then
        # Doesn't work in zsh.
        complete -o nospace -F _croot croot
    fi
    complete -F _lunch lunch   # _lunch命令提供lunch命令的补全操作
 
    complete -F _complete_android_module_names gomod
    complete -F _complete_android_module_names m
}

 

3. lunch aosp_arm_eng

3.1 lunch description

  After the environment variables are initialized, we need to select a compilation target. The main function of lunch is to set environment variables related to specific products based on the product name entered or selected by the user.

  If you don't know what target you want to compile, just execute a lunch command and all targets will be listed. Just press Enter and the aosp_arm-eng target will be used by default.

  Execute command: lunch 1, you can see some configured environment variables

  The meanings of these environment variables are as follows:

lunch results	illustrate
PLATFORM_VERSION_CODENAME=REL	Indicates the name of the platform version
PLATFORM_VERSION=10	Android platform version number
TARGET_PRODUCT=aosp_arm	Compiled product name
TARGET_BUILD_VARIANT=userdebug	Type of product being compiled
TARGET_BUILD_TYPE=release	Compilation type, debug and release
TARGET_ARCH=arm	Represents the CPU architecture of the compilation target
TARGET_ARCH_VARIANT=armv7-a-neon	Indicates the CPU architecture version of the compilation target
TARGET_CPU_VARIANT=generic	Represents the CPU codename of the compilation target
HOST_ARCH=x86_64	Represents the architecture of the compilation platform
HOST_2ND_ARCH=x86
HOST_OS=linux	Represents the operating system of the compilation platform
HOST_OS_EXTRA=Linux-4.15.0-112-generic-x86_64-Ubuntu-16.04.6-LTS	Extra information outside the build system
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=QQ1D.200205.002	BUILD_ID will appear in the version information and can be used
OUT_DIR=out	Path to compile result output

  After lunch aosp_arm-eng is completed, create an out folder and generate some intermediate files as shown in the figure below:

 

3.2 lunch()

  The lunch command is used to set environment variables such as TARGET_PRODUCT, TARGET_BUILD_VARIANT, TARGET_PLATFORM_VERSION, TARGET_BUILD_TYPE, TARGET_BUILD_APPS, etc.

  The lunch operation process is as follows:

  1. Get the parameters of the lunch operation. If the parameters are not empty, the parameters specify the device model and compilation type to be compiled; if the parameters are empty, print_lunch_menu will be called to display the Lunch menu items, read the user's input, and store the answer

  2. If answer is empty, that is, it was used in the lunch menu before, and the user only pressed Enter. The default option will be changed to aosp_arm-eng, and the result will be stored in selection

  3. If the input obtained by the lunch operation is a number, convert the number into a string in LUNCH_MENU_CHOICES, and store the result in selection

  4. Parse the value of selection, get product = aosp_arm and variant = eng, and save them to TARGET_PRODUCT and TARGET_BUILD_VARIANT respectively.

  5.根据前面的设置，调用build_build_var_cache 来更新编译环境相关变量

  6.export 编译选项TARGET_PRODUCT, TARGET_BUILD_VARIANT和TARGET_BUILD_TYPE三元组

  7.调用set_stuff_for_environment 来设置其他环境变量，如PROMPT_COMMAND，编译toolchain和tools相关的路径等

  8.调用printconfig 来输出当前的设置选项

 
function lunch()
{
    
    
    local answer
        # 获取lunch操作的参数
    if [ "$1" ] ; then
        answer=$1
    else
           # lunch操作不带参数，则先显示lunch menu，然后读取用户输入
        print_lunch_menu
        echo -n "Which would you like? [aosp_arm-eng] "
        read answer
    fi
 
    local selection=
       # lunch操作得到的结果为空（例如用户直接在lunch要求输入时回车的情况）
    # 则将选项默认为"aosp_arm-eng"
    if [ -z "$answer" ]
    then
        selection=aosp_arm-eng
        # lunch操作得到的输入是数字，则将数字转换为LUNCH_MENU_CHOICES中的字符串
    elif (echo -n $answer | grep -q -e "^[0-9][0-9]*$")
    then
        local choices=($(TARGET_BUILD_APPS= get_build_var COMMON_LUNCH_CHOICES))
        if [ $answer -le ${
    
    #choices[@]} ]
        then
            # array in zsh starts from 1 instead of 0.
            if [ -n "$ZSH_VERSION" ]
            then
                selection=${choices[$(($answer))]}
            else
                selection=${choices[$(($answer-1))]}
            fi
        fi
    else
        selection=$answer
    fi
 
    export TARGET_BUILD_APPS=
 
    local product variant_and_version variant version
 
    product=${selection%%-*} # Trim everything after first dash
    variant_and_version=${selection#*-} # Trim everything up to first dash
    if [ "$variant_and_version" != "$selection" ]; then
        variant=${variant_and_version%%-*}
        if [ "$variant" != "$variant_and_version" ]; then
            version=${variant_and_version#*-}
        fi
    fi
 
    if [ -z "$product" ]
    then
        echo
        echo "Invalid lunch combo: $selection"
        return 1
    fi
 
        # 设置TARGET_PRODUCT和TARGET_BUILD_VARIANT
    TARGET_PRODUCT=$product \
    TARGET_BUILD_VARIANT=$variant \
    TARGET_PLATFORM_VERSION=$version \
 
    # 根据前面的设置，更新编译环境相关变量
    build_build_var_cache     #参考[3.1.1]
    if [ $? -ne 0 ]
    then
        return 1
    fi
 
        # export 编译选项TARGET_PRODUCT, TARGET_BUILD_VARIANT和TARGET_BUILD_TYPE三元组
    export TARGET_PRODUCT=$(get_build_var TARGET_PRODUCT)
    export TARGET_BUILD_VARIANT=$(get_build_var TARGET_BUILD_VARIANT)
    if [ -n "$version" ]; then
      export TARGET_PLATFORM_VERSION=$(get_build_var TARGET_PLATFORM_VERSION)
    else
      unset TARGET_PLATFORM_VERSION
    fi
    export TARGET_BUILD_TYPE=release
 
    echo
 
    set_stuff_for_environment # 设置其他环境变量，如PROMPT_COMMAND，编译toolchain和tools相关的路径等
    printconfig        # 输出当前的设置选项
    destroy_build_var_cache
}

3.1.1 build_build_var_cache()

  根据前面的设置，更新编译环境相关变量

  主要通过执行 "build/soong/soong_ui.bash --dumpvars-mode" 完成

  最终执行的是 "./out/soog_ui  --dumpvars-mode"

 
function build_build_var_cache()
{
    
    
    local T=$(gettop)
    # Grep out the variable names from the script.
    cached_vars=(`cat $T/build/envsetup.sh | tr '()' '  ' | awk '{for(i=1;i<=NF;i++) if($i~/get_build_var/) print $(i+1)}' | sort -u | tr '\n' ' '`)
    cached_abs_vars=(`cat $T/build/envsetup.sh | tr '()' '  ' | awk '{for(i=1;i<=NF;i++) if($i~/get_abs_build_var/) print $(i+1)}' | sort -u | tr '\n' ' '`)
    # Call the build system to dump the "<val>=<value>" pairs as a shell script.
    build_dicts_script=`\builtin cd $T; build/soong/soong_ui.bash --dumpvars-mode \
                        --vars="${cached_vars[*]}" \
                        --abs-vars="${cached_abs_vars[*]}" \
                        --var-prefix=var_cache_ \
                        --abs-var-prefix=abs_var_cache_`
    local ret=$?
    if [ $ret -ne 0 ]
    then
        unset build_dicts_script
        return $ret
    fi
    # Execute the script to store the "<val>=<value>" pairs as shell variables.
    eval "$build_dicts_script"
    ret=$?
    unset build_dicts_script
    if [ $ret -ne 0 ]
    then
        return $ret
    fi
    BUILD_VAR_CACHE_READY="true"
}

soong_ui 由build/soong/cmd/soong_ui/main.go编译生成

[build/soong/cmd/soong_ui/main.go]
func main() {
    
    
...
    if os.Args[1] == "--dumpvar-mode" {
    
    
        dumpVar(buildCtx, config, os.Args[2:])
    } else if os.Args[1] == "--dumpvars-mode" {
    
    
        dumpVars(buildCtx, config, os.Args[2:])
    } else {
    
    
        ...
    }
...
}
[build/soong/cmd/soong_ui/main.go]
func dumpVars(ctx build.Context, config build.Config, args []string) {
    
    
       varData, err := build.DumpMakeVars(ctx, config, nil, allVars)
}

最后调用到了ckati执行-f build/make/core/config.mk

[/build/soong/ui/build/dumpvars.go]
func dumpMakeVars(ctx Context, config Config, goals, vars []string, write_soong_vars bool) (map[string]string, error) {
    
    
    ctx.BeginTrace(metrics.RunKati, "dumpvars")
    defer ctx.EndTrace()
 
    cmd := Command(ctx, config, "dumpvars",
        config.PrebuiltBuildTool("ckati"),
        "-f", "build/make/core/config.mk",
        "--color_warnings",
        "--kati_stats",
        "dump-many-vars",
        "MAKECMDGOALS="+strings.Join(goals, " "))
    cmd.Environment.Set("CALLED_FROM_SETUP", "true")
    if write_soong_vars {
    
    
        cmd.Environment.Set("WRITE_SOONG_VARIABLES", "true")
    }
    cmd.Environment.Set("DUMP_MANY_VARS", strings.Join(vars, " "))
    cmd.Sandbox = dumpvarsSandbox
    output := bytes.Buffer{
    
    }
    cmd.Stdout = &output
    pipe, err := cmd.StderrPipe()
    if err != nil {
    
    
        ctx.Fatalln("Error getting output pipe for ckati:", err)
    }
    cmd.StartOrFatal()
    // TODO: error out when Stderr contains any content
    status.KatiReader(ctx.Status.StartTool(), pipe)
    cmd.WaitOrFatal()
 
    ret := make(map[string]string, len(vars))
    ...
 
    return ret, nil
}

下面我们单独研究一下config.mk。

 

4.config.mk

 说明：config.mk首先加载了build/make/common 中的core.mk、math.mk、strings.mk、json.mk 用来配置一些shell环境、math函数、string和json的一些支持函数。最主要的操作还是加载build/make/core中的envsetup.mk和dumpvar.mk

    ...
 
#配置两个目录的变量，供之后的mk使用
BUILD_SYSTEM :=$= build/make/core
BUILD_SYSTEM_COMMON :=$= build/make/common
 
#加载core.mk， 只使用ANDROID_BUILD_SHELL来包装bash。
include $(BUILD_SYSTEM_COMMON)/core.mk
 
 
#设置make中使用的有效数学函数。
include $(BUILD_SYSTEM_COMMON)/math.mk
 
include $(BUILD_SYSTEM_COMMON)/strings.mk
 
include $(BUILD_SYSTEM_COMMON)/json.mk
 
# 避免硬件解码路径被覆盖的调用pathmap.mk建立硬解映射
include $(BUILD_SYSTEM)/pathmap.mk
 
# 允许项目定义自己的全局可用变量
include $(BUILD_SYSTEM)/project_definitions.mk
 
 
# ###############################################################
# Build system internal files
# ###############################################################
# 构建系统内部文件（写Android.mk时会调用include头文件，也就是这些makefile文件）
 
BUILD_COMBOS:= $(BUILD_SYSTEM)/combo
 
CLEAR_VARS:= $(BUILD_SYSTEM)/clear_vars.mk
BUILD_HOST_STATIC_LIBRARY:= $(BUILD_SYSTEM)/host_static_library.mk
BUILD_HOST_SHARED_LIBRARY:= $(BUILD_SYSTEM)/host_shared_library.mk
BUILD_STATIC_LIBRARY:= $(BUILD_SYSTEM)/static_library.mk
 
...
 
BUILD_NOTICE_FILE := $(BUILD_SYSTEM)/notice_files.mk
BUILD_HOST_DALVIK_JAVA_LIBRARY := $(BUILD_SYSTEM)/host_dalvik_java_library.mk
BUILD_HOST_DALVIK_STATIC_JAVA_LIBRARY := $(BUILD_SYSTEM)/host_dalvik_static_java_library.mk
 
BUILD_HOST_TEST_CONFIG := $(BUILD_SYSTEM)/host_test_config.mk
BUILD_TARGET_TEST_CONFIG := $(BUILD_SYSTEM)/target_test_config.mk
 
 
 
#定义大多数全局变量。这些是特定于用户的构建配置的。
include $(BUILD_SYSTEM)/envsetup.mk
 
#构建系统为在哪里找到内核公开了几个变量
#(1)TARGET_DEVICE_KERNEL_HEADERS是为当前正在构建的设备自动创建的。
#它被设置为$（TARGET_DEVICE_DIR）/kernel headers，
#例如DEVICE/samsung/tuna/kernel headers。此目录不是由任何人显式设置的，生成系统总是添加此子目录。
TARGET_DEVICE_KERNEL_HEADERS := $(strip $(wildcard $(TARGET_DEVICE_DIR)/kernel-headers))
 
#(2)TARGET_BOARD_KERNEL_HEADERS由BoardConfig.mk允许包含其他目录的文件。
#如果有一些常见的地方为一组设备保留了一些报头，那么这很有用。
#例如，device/<vendor>/common/kernel头可以包含一些<vendor>设备的头。
TARGET_BOARD_KERNEL_HEADERS := $(strip $(wildcard $(TARGET_BOARD_KERNEL_HEADERS)))
TARGET_BOARD_KERNEL_HEADERS := $(patsubst %/,%,$(TARGET_BOARD_KERNEL_HEADERS))
$(call validate-kernel-headers,$(TARGET_BOARD_KERNEL_HEADERS))
#(3)TARGET_PRODUCT_KERNEL_头由产品继承图生成。
#这允许体系结构产品为使用该体系结构的设备提供报头。
TARGET_PRODUCT_KERNEL_HEADERS := $(strip $(wildcard $(PRODUCT_VENDOR_KERNEL_HEADERS)))
TARGET_PRODUCT_KERNEL_HEADERS := $(patsubst %/,%,$(TARGET_PRODUCT_KERNEL_HEADERS))
$(call validate-kernel-headers,$(TARGET_PRODUCT_KERNEL_HEADERS))
 
 
# 选择一个Java编译器
include $(BUILD_SYSTEM)/combo/javac.mk
 
# A list of SEPolicy versions, besides PLATFORM_SEPOLICY_VERSION, that the framework supports.
#框架支持的SEPolicy版本列表，除了PLATFORM_SEPOLICY_VERSION
PLATFORM_SEPOLICY_COMPAT_VERSIONS := \
    26.0 \
    27.0 \
    28.0 \
 
ifeq ($(CALLED_FROM_SETUP),true)
include $(BUILD_SYSTEM)/ninja_config.mk
include $(BUILD_SYSTEM)/soong_config.mk
endif
 
#加载dumpvar.mk,用来生成make目标
include $(BUILD_SYSTEM)/dumpvar.mk

4.1 build/make/core/envsetup.mk

  envsetup.mk 主要加载了product_config.mk和board_config.mk，用来得到TARGET_DEVICE和其他变量。

...
#设置host和target编译链相关的变量
include $(BUILD_SYSTEM)/combo/select.mk
#(1)阅读产品规格，这样我们就可以得到TARGET_DEVICE和其他变量，我们需要找到输出文件
include $(BUILD_SYSTEM)/product_config.mk
 
include $(BUILD_SYSTEM)/board_config.mk
...

4.2  build/make/core/product_config.mk

  阅读产品规格，这样我们就可以得到TARGET_DEVICE和其他变量，我们需要找到输出文件。

...
# ---------------------------------------------------------------
# Include the product definitions.
# We need to do this to translate TARGET_PRODUCT into its
# underlying TARGET_DEVICE before we start defining any rules.
#
include $(BUILD_SYSTEM)/node_fns.mk
include $(BUILD_SYSTEM)/product.mk
include $(BUILD_SYSTEM)/device.mk
 
...
 
#############################################################################
# Sanity check and assign default values
TARGET_DEVICE := $(PRODUCT_DEVICE)
...

4.3  build/make/core/board_config.mk

  板级可以在$（SRC_TARGET_DIR）/board/$（TARGET_DEVICE）下定义，也可以在vendor/*/$（TARGET_DEVICE）下定义。

  在这两个地方搜索，但要确保只存在一个。真正的板级应始终与OEM vendor相关联。

# Boards may be defined under $(SRC_TARGET_DIR)/board/$(TARGET_DEVICE)
# or under vendor/*/$(TARGET_DEVICE).  Search in both places, but
# make sure only one exists.
# Real boards should always be associated with an OEM vendor.
ifdef TARGET_DEVICE_DIR
  ifneq ($(origin TARGET_DEVICE_DIR),command line)
    $(error TARGET_DEVICE_DIR may not be set manually)
  endif
  board_config_mk := $(TARGET_DEVICE_DIR)/BoardConfig.mk
else
  board_config_mk := \
    $(strip $(sort $(wildcard \
      $(SRC_TARGET_DIR)/board/$(TARGET_DEVICE)/BoardConfig.mk \
      $(shell test -d device && find -L device -maxdepth 4 -path '*/$(TARGET_DEVICE)/BoardConfig.mk') \
      $(shell test -d vendor && find -L vendor -maxdepth 4 -path '*/$(TARGET_DEVICE)/BoardConfig.mk') \
    )))
  ifeq ($(board_config_mk),)
    $(error No config file found for TARGET_DEVICE $(TARGET_DEVICE))
  endif
  ifneq ($(words $(board_config_mk)),1)
    $(error Multiple board config files for TARGET_DEVICE $(TARGET_DEVICE): $(board_config_mk))
  endif
  TARGET_DEVICE_DIR := $(patsubst %/,%,$(dir $(board_config_mk)))
  .KATI_READONLY := TARGET_DEVICE_DIR
endif
include $(board_config_mk)

5.总结

  至此，envsetup.sh 和lunch()的初始化流程基本上理清了，主要就是加载了环境变量，并选择了编译目标，后面只要执行一下make就能够进行启动编译，下一节让我们一起看看敲下make后到底发生了什么。

摘要：本节主要来进行Android10.0 编译系统的环境初始化