Android build中的envsetup.sh详解

源码基于：Android R

0. 前言

今天在编译项目的时候，想看看 envsetup.sh 中变化了些什么，才想起来编译专栏中好像没有详解该脚本，索性现在空余时间比较多，整理一下方便以后查看。

Android envsetup.sh 为编译前的准备工作，提供 lunch、m、mm等命令函数定义，是整个Android 编译系统的第一步。该文件一共1700 行左右，简单总结出两件事情：

• 函数定义
• 生成编译配置列表

本文会结合代码对 envsetup.sh 进行详细的剖析，函数比较多，会进行长期地补充和维护。

1. source envsetup.sh

当运行该脚本的时候终端提示：

including vendor/xxx/common/vendorsetup.sh
clang compdb: ln -s out/soong/development/ide/compdb/compile_commands.json $ANDROID_BUILD_TOP
including vendor/qqq/opensource/core-utils/vendorsetup.sh
including vendor/qqq/proprietary/common/vendorsetup.sh

那执行该脚本的时候到底做了些什么呢？

build/envsetup.sh

validate_current_shell
source_vendorsetup
addcompletions

执行该程序共运行三个函数，分别是 validate_current_shell、source_vendorsetup、addcompletions。下面来看下这三个函数做了些什么事情？

1.1 validate_current_shell

function validate_current_shell() {
    local current_sh="$(ps -o command -p $$)"
    case "$current_sh" in
        *bash*)
            function check_type() { type -t "$1"; }
            ;;
        *zsh*)
            function check_type() { type "$1"; }
            enable_zsh_completion ;;
        *)
            echo -e "WARNING: Only bash and zsh are supported.\nUse of other shell would lead to erroneous results."
            ;;
    esac
}

主要是测试下当前的shell 环境是否可用，对于 zsh 环境需要特殊处理，其他环境会其实warning。

扫描二维码关注公众号，回复： 15266198 查看本文章

1.2 source_vendorsetup

function source_vendorsetup() {
    unset VENDOR_PYTHONPATH
    allowed=
    for f in $(find -L device vendor product -maxdepth 4 -name 'allowed-vendorsetup_sh-files' 2>/dev/null | sort); do
        if [ -n "$allowed" ]; then
            echo "More than one 'allowed_vendorsetup_sh-files' file found, not including any vendorsetup.sh files:"
            echo "  $allowed"
            echo "  $f"
            return
        fi
        allowed="$f"
    done

    allowed_files=
    [ -n "$allowed" ] && allowed_files=$(cat "$allowed")
    for dir in device vendor product; do
        for f in $(test -d $dir && \
            find -L $dir -maxdepth 4 -name 'vendorsetup.sh' 2>/dev/null | sort); do

            if [[ -z "$allowed" || "$allowed_files" =~ $f ]]; then
                echo "including $f"; . "$f"
            else
                echo "ignoring $f, not in $allowed"
            fi
        done
    done
}

• 首先，查找 device、vendor、product 目录下最大深度为4 的子目录中，是否存在 allowed-vendorsetup_sh-files 文件并排序，但这个文件只能1个，不然会报错退出；
• 接着，在device、vendor、product 中同样查找最大深度为 4 的子目录中，是否存在 vendorsetup.sh，打印 includeing ... ，然后运行该 vendorsetup.sh 脚本，这些脚本大多是配置一些环境变量；

例如上面的打印 including vendor/xxx/common/vendorsetup.sh，就是在 vendor/xxx/common 目录下找到了 vendorsetup.sh，运行的时候会配置一些环境变量，并打印：

clang compdb: ln -s out/soong/development/ide/compdb/compile_commands.json 

1.3 addcompletions

function addcompletions()
{
    local T dir f

    #避免不在 bash 或 zsh 的环境中运行
    #需要提前指定这两个环境变量，如果没有指定，该函数return
    if [ -z "$BASH_VERSION" -a -z "$ZSH_VERSION" ]; then
        return
    fi

    #避免运行在太久的版本中，这里是小于 3 的版本
    if [ -n "$BASH_VERSION" -a ${BASH_VERSINFO[0]} -lt 3 ]; then
        return
    fi

    local completion_files=(
      system/core/adb/adb.bash
      system/core/fastboot/fastboot.bash
      tools/asuite/asuite.sh
    )
    
    #确认上面几个脚本是否存在，通过should_add_completion确认该脚本是否在白名单中
    for f in ${completion_files[*]}; do
        if [ -f "$f" ] && should_add_completion "$f"; then
            . $f
        fi
    done

    if should_add_completion bit ; then
        complete -C "bit --tab" bit
    fi
    if [ -z "$ZSH_VERSION" ]; then
        # Doesn't work in zsh.
        complete -o nospace -F _croot croot
    fi
    complete -F _lunch lunch

    complete -F _complete_android_module_names dumpmod
    complete -F _complete_android_module_names pathmod
    complete -F _complete_android_module_names gomod
    complete -F _complete_android_module_names m
}

详细的 complete 命令可以另外查找，最后就是把一些命令进行补齐的设置。

2. 辅助函数

2.1 hmm

hmm 函数输出 envsetupsh.sh 的帮助说明，执行 build/envsetup.sh 后可以调用的操作总结：

function hmm() {
cat <<EOF

Run "m help" for help with the build system itself.

Invoke ". build/envsetup.sh" from your shell to add the following functions to your environment:
- lunch:      lunch <product_name>-<build_variant>
              Selects <product_name> as the product to build, and <build_variant> as the variant to
              build, and stores those selections in the environment to be read by subsequent
              invocations of 'm' etc.
- tapas:      tapas [<App1> <App2> ...] [arm|x86|mips|arm64|x86_64|mips64] [eng|userdebug|user]
- croot:      Changes directory to the top of the tree, or a subdirectory thereof.
- m:          Makes from the top of the tree.
- mm:         Builds and installs all of the modules in the current directory, and their
              dependencies.
- mmm:        Builds and installs all of the modules in the supplied directories, and their
              dependencies.
              To limit the modules being built use the syntax: mmm dir/:target1,target2.
- mma:        Same as 'mm'
- mmma:       Same as 'mmm'
- provision:  Flash device with all required partitions. Options will be passed on to fastboot.
- cgrep:      Greps on all local C/C++ files.
- ggrep:      Greps on all local Gradle files.
- gogrep:     Greps on all local Go files.
- jgrep:      Greps on all local Java files.
- resgrep:    Greps on all local res/*.xml files.
- mangrep:    Greps on all local AndroidManifest.xml files.
- mgrep:      Greps on all local Makefiles and *.bp files.
- owngrep:    Greps on all local OWNERS files.
- sepgrep:    Greps on all local sepolicy files.
- sgrep:      Greps on all local source files.
- godir:      Go to the directory containing a file.
- allmod:     List all modules. or List all modules inside certain directory.
- gomod:      Go to the directory containing a module.
- dumpmod:    Get all info of specific modules from \$ANDROID_PRODUCT_OUT/module-info.json
- pathmod:    Get the directory containing a module.
- refreshmod: Refresh list of modules for allmod/gomod.
- ninja-build: Bypass ninja generate procedure, directly use ninja to build a target or module, also build droid is support too by no param given.
- ninja-query: Query input and output of a ninja target, this is the most powerful tool when digging the compile steps.
- ninja-commands: Print all build commands, like a --just-print version of ninja-build, also known as the --dry-run purpose
- mod:        Analyze $OUT/module-info.json with parameters, see mod -h for help

Environment options:
- SANITIZE_HOST: Set to 'address' to use ASAN for all host modules.
- ANDROID_QUIET_BUILD: set to 'true' to display only the essential messages.

Look at the source to view more functions. The complete list is:
EOF
    local T=$(gettop)
    local A=""
    local i
    for i in `cat $T/build/envsetup.sh | sed -n "/^[[:blank:]]*function /s/function \([a-z_]*\).*/\1/p" | sort | uniq`; do
      A="$A $i"
    done
    echo $A
}

最后还列出了完整的函数名，通过：

sed -n "/^[[:blank:]]*function /s/function \([a-z_]*\).*/\1/p"

但其操作有一个 bug，用于匹配函数的正则表达式 function \([a-z_]*\).* 会漏掉函数 is64bit()；

将匹配模式从 function \([a-z_]*\).*/  修改为 function \([a-z_]\w*\).*  就可以匹配文件中的所有函数了。

2.2 gettop

gettop 函数，从指定的 $TOP 目录或者当前目录开始查找 build/make/core/envsetup.mk，并将能找到该文件的目录返回个调用函数作为操作的根目录： 

function gettop
{
    local TOPFILE=build/make/core/envsetup.mk

    #如果编译环境已经设置了 $TOP,就检查 $TOP/build/make/core/envsetup.mk文件是否存在
    if [ -n "$TOP" -a -f "$TOP/$TOPFILE" ] ; then
        # The following circumlocution ensures we remove symlinks from TOP.
        #跳转到$TOP 目录，并pwd将$TOP 目录指向的真实路径存放到PWD中
        (cd $TOP; PWD= /bin/pwd)
    else
        #如果当前路径下能找到 build/make/core/envsetup.mk文件，则将当前目录的真实路径存放到PWD中
        if [ -f $TOPFILE ] ; then
            PWD= /bin/pwd
        else
            #如果当前目录下无法找到build/make/core/envsetup.mk文件，则不断返回到外层目录查找，
            #直至到根目录为止
            local HERE=$PWD
            local T=
            while [ \( ! \( -f $TOPFILE \) \) -a \( $PWD != "/" \) ]; do
                \cd ..
                T=`PWD= /bin/pwd -P`
            done
            #查找完后回到之前操作的路径
            \cd $HERE
            #如果目录$T包含了build/make/core/envsetup.mk，则说明$T是编译的根目录
            if [ -f "$T/$TOPFILE" ]; then
                echo $T
            fi
        fi
    fi
}

2.3 croot

croot 命令用以将当前目录切换到当前编译环境的根目录。

当然croot 之后可以跟一个参数标记切到根目录之后的下一级目录，例如 croot device 命令。

详细可以查看代码：

function croot()
{
    local T=$(gettop)
    if [ "$T" ]; then
        if [ "$1" ]; then
            \cd $(gettop)/$1
        else
            \cd $(gettop)
        fi
    else
        echo "Couldn't locate the top of the tree.  Try setting TOP."
    fi
}

2.4 cproj

cproj 命令用于切换到当前模块的编译目录下（含Android.mk的目录下）

function cproj()
{
    local TOPFILE=build/make/core/envsetup.mk
    local HERE=$PWD  #临时保存当前目录
    local T=

    #当前目录下build/make/core/envsetup.mk不存在，即当前不是编译根目录，且
    #当前目录不是系统根目录
    while [ \( ! \( -f $TOPFILE \) \) -a \( $PWD != "/" \) ]; do
        T=$PWD
        if [ -f "$T/Android.mk" ]; then  #如果该目录下有Android.mk文件，则cd过去
            \cd $T
            return
        fi
        \cd ..
    done
    \cd $HERE   #恢复之前的目录
    echo "can't find Android.mk"
}

2.5 getprebuilt

function getprebuilt
{
    get_abs_build_var ANDROID_PREBUILTS
}

function get_abs_build_var()
{
    if [ "$BUILD_VAR_CACHE_READY" = "true" ]
    then
        eval "echo \"\${abs_var_cache_$1}\""
    return
    fi

    local T=$(gettop)
    if [ ! "$T" ]; then
        echo "Couldn't locate the top of the tree.  Try setting TOP." >&2
        return
    fi
    (\cd $T; build/soong/soong_ui.bash --dumpvar-mode --abs $1)
}

主要是通过函数 get_abs_build_var() 中查找变量值，这里查找的是 ANDROID_PREBUILTS 这个绝对路径为 $TOP/prebuilt/linux-x86

2.6 setpaths

主要是配置一些环境变量，在lunch 命令时调用，或者是在 choosecombo 函数中调用：

function setpaths()
{
    local T=$(gettop)
    if [ ! "$T" ]; then
        echo "Couldn't locate the top of the tree.  Try setting TOP."
        return
    fi

    ##################################################################
    #                                                                #
    #              Read me before you modify this code               #
    #                                                                #
    #   This function sets ANDROID_BUILD_PATHS to what it is adding  #
    #   to PATH, and the next time it is run, it removes that from   #
    #   PATH.  This is required so lunch can be run more than once   #
    #   and still have working paths.                                #
    #                                                                #
    ##################################################################

    # Note: on windows/cygwin, ANDROID_BUILD_PATHS will contain spaces
    # due to "C:\Program Files" being in the path.

    # out with the old
    if [ -n "$ANDROID_BUILD_PATHS" ] ; then
        export PATH=${PATH/$ANDROID_BUILD_PATHS/}
    fi
    if [ -n "$ANDROID_PRE_BUILD_PATHS" ] ; then
        export PATH=${PATH/$ANDROID_PRE_BUILD_PATHS/}
        # strip leading ':', if any
        export PATH=${PATH/:%/}
    fi

    # and in with the new
    local prebuiltdir=$(getprebuilt)
    local gccprebuiltdir=$(get_abs_build_var ANDROID_GCC_PREBUILTS)

    # defined in core/config.mk
    local targetgccversion=$(get_build_var TARGET_GCC_VERSION)
    local targetgccversion2=$(get_build_var 2ND_TARGET_GCC_VERSION)
    export TARGET_GCC_VERSION=$targetgccversion

    # The gcc toolchain does not exists for windows/cygwin. In this case, do not reference it.
    export ANDROID_TOOLCHAIN=
    export ANDROID_TOOLCHAIN_2ND_ARCH=
    local ARCH=$(get_build_var TARGET_ARCH)
    local toolchaindir toolchaindir2=
    case $ARCH in
        x86) toolchaindir=x86/x86_64-linux-android-$targetgccversion/bin
            ;;
        x86_64) toolchaindir=x86/x86_64-linux-android-$targetgccversion/bin
            ;;
        arm) toolchaindir=arm/arm-linux-androideabi-$targetgccversion/bin
            ;;
        arm64) toolchaindir=aarch64/aarch64-linux-android-$targetgccversion/bin;
               toolchaindir2=arm/arm-linux-androideabi-$targetgccversion2/bin
            ;;
        mips|mips64) toolchaindir=mips/mips64el-linux-android-$targetgccversion/bin
            ;;
        *)
            echo "Can't find toolchain for unknown architecture: $ARCH"
            toolchaindir=xxxxxxxxx
            ;;
    esac
    if [ -d "$gccprebuiltdir/$toolchaindir" ]; then
        export ANDROID_TOOLCHAIN=$gccprebuiltdir/$toolchaindir
    fi

    if [ "$toolchaindir2" -a -d "$gccprebuiltdir/$toolchaindir2" ]; then
        export ANDROID_TOOLCHAIN_2ND_ARCH=$gccprebuiltdir/$toolchaindir2
    fi

    export ANDROID_DEV_SCRIPTS=$T/development/scripts:$T/prebuilts/devtools/tools:$T/external/selinux/prebuilts/bin

    # add kernel specific binaries
    case $(uname -s) in
        Linux)
            export ANDROID_DEV_SCRIPTS=$ANDROID_DEV_SCRIPTS:$T/prebuilts/misc/linux-x86/dtc:$T/prebuilts/misc/linux-x86/libufdt
            ;;
        *)
            ;;
    esac

    ANDROID_BUILD_PATHS=$(get_build_var ANDROID_BUILD_PATHS):$ANDROID_TOOLCHAIN
    if [ -n "$ANDROID_TOOLCHAIN_2ND_ARCH" ]; then
        ANDROID_BUILD_PATHS=$ANDROID_BUILD_PATHS:$ANDROID_TOOLCHAIN_2ND_ARCH
    fi
    ANDROID_BUILD_PATHS=$ANDROID_BUILD_PATHS:$ANDROID_DEV_SCRIPTS

    # Append llvm binutils prebuilts path to ANDROID_BUILD_PATHS.
    local ANDROID_LLVM_BINUTILS=$(get_abs_build_var ANDROID_CLANG_PREBUILTS)/llvm-binutils-stable
    ANDROID_BUILD_PATHS=$ANDROID_BUILD_PATHS:$ANDROID_LLVM_BINUTILS

    # Set up ASAN_SYMBOLIZER_PATH for SANITIZE_HOST=address builds.
    export ASAN_SYMBOLIZER_PATH=$ANDROID_LLVM_BINUTILS/llvm-symbolizer

    # If prebuilts/android-emulator/<system>/ exists, prepend it to our PATH
    # to ensure that the corresponding 'emulator' binaries are used.
    case $(uname -s) in
        Darwin)
            ANDROID_EMULATOR_PREBUILTS=$T/prebuilts/android-emulator/darwin-x86_64
            ;;
        Linux)
            ANDROID_EMULATOR_PREBUILTS=$T/prebuilts/android-emulator/linux-x86_64
            ;;
        *)
            ANDROID_EMULATOR_PREBUILTS=
            ;;
    esac
    if [ -n "$ANDROID_EMULATOR_PREBUILTS" -a -d "$ANDROID_EMULATOR_PREBUILTS" ]; then
        ANDROID_BUILD_PATHS=$ANDROID_BUILD_PATHS:$ANDROID_EMULATOR_PREBUILTS
        export ANDROID_EMULATOR_PREBUILTS
    fi

    # Append asuite prebuilts path to ANDROID_BUILD_PATHS.
    local os_arch=$(get_build_var HOST_PREBUILT_TAG)
    local ACLOUD_PATH="$T/prebuilts/asuite/acloud/$os_arch"
    local AIDEGEN_PATH="$T/prebuilts/asuite/aidegen/$os_arch"
    local ATEST_PATH="$T/prebuilts/asuite/atest/$os_arch"
    export ANDROID_BUILD_PATHS=$ANDROID_BUILD_PATHS:$ACLOUD_PATH:$AIDEGEN_PATH:$ATEST_PATH:

    export PATH=$ANDROID_BUILD_PATHS$PATH

    # out with the duplicate old
    if [ -n $ANDROID_PYTHONPATH ]; then
        export PYTHONPATH=${PYTHONPATH//$ANDROID_PYTHONPATH/}
    fi
    # and in with the new
    export ANDROID_PYTHONPATH=$T/development/python-packages:
    if [ -n $VENDOR_PYTHONPATH  ]; then
        ANDROID_PYTHONPATH=$ANDROID_PYTHONPATH$VENDOR_PYTHONPATH
    fi
    export PYTHONPATH=$ANDROID_PYTHONPATH$PYTHONPATH

    export ANDROID_JAVA_HOME=$(get_abs_build_var ANDROID_JAVA_HOME)
    export JAVA_HOME=$ANDROID_JAVA_HOME
    export ANDROID_JAVA_TOOLCHAIN=$(get_abs_build_var ANDROID_JAVA_TOOLCHAIN)
    export ANDROID_PRE_BUILD_PATHS=$ANDROID_JAVA_TOOLCHAIN:
    export PATH=$ANDROID_PRE_BUILD_PATHS$PATH

    unset ANDROID_PRODUCT_OUT
    export ANDROID_PRODUCT_OUT=$(get_abs_build_var PRODUCT_OUT)
    export OUT=$ANDROID_PRODUCT_OUT

    unset ANDROID_HOST_OUT
    export ANDROID_HOST_OUT=$(get_abs_build_var HOST_OUT)

    unset ANDROID_HOST_OUT_TESTCASES
    export ANDROID_HOST_OUT_TESTCASES=$(get_abs_build_var HOST_OUT_TESTCASES)

    unset ANDROID_TARGET_OUT_TESTCASES
    export ANDROID_TARGET_OUT_TESTCASES=$(get_abs_build_var TARGET_OUT_TESTCASES)

    # needed for building linux on MacOS
    # TODO: fix the path
    #export HOST_EXTRACFLAGS="-I "$T/system/kernel_headers/host_include
}

2.7 printconfig

function printconfig()
{
    local T=$(gettop)
    if [ ! "$T" ]; then
        echo "Couldn't locate the top of the tree.  Try setting TOP." >&2
        return
    fi
    get_build_var report_config
}

调用get_build_var 函数将lunch 之后的编译配置信息：

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=11
TARGET_PRODUCT=shift
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=cortex-a55
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a55
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-142-generic-x86_64-Ubuntu-16.04.5-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=RD2A.211001.002
OUT_DIR=out
PRODUCT_SOONG_NAMESPACES=vendor/qqq/opensource/commonsys/packages/apps/Bluetooth vendor/qqq/opensource/commonsys/system/bt/conf external/v4l2_codec2
============================================

2.8