Android Optimization Project (III): MultiDex optimize the use of Android apk package name command to view aapt

Before finishing MultiDex optimization, first look Apk compilation process, which helps optimize behind for MultiDex.

A, Apk compilation process

Android Studio What happens after pressing the Compile button?

1. packed resource file, generate R.java file (using tool aapt, this tool in use aapt command to view Android apk package name  mentioned, are interested can find out)

2. aidl processing files, generate java code (not aidl is ignored)

3. Compile java file, to generate the corresponding .class files (java Compiler)

4. class files into a converted file dex (dex)

The packaged into unsigned APK (tools apkbuilder)

6. Use signature tool to sign the apk (using tool Jarsigner)

In step 4, converts class files into dex file, the default will generate a dex file, the number of ways a single dex file can not be more than 65,536, or compiler will complain, but we will certainly integrate a bunch of libraries in the development of App, the number of methods are generally more than 65,536, the solution to this problem is: a dex fit, with more dex to install, gradle add a line configuration: multiDexEnabled true.

Specific configuration program can refer to: Android subcontractors MultiDex Strategy Summary .

Two, MultiDex principle

Although configured the MultiDex subcontracting strategy, but we found only performed on Android 4.4 phones MultiDex.install (context) may consume more than a second of time, so why be so time-consuming it? Here the first analyze the principle of MultiDex.

2.1  MultiDex principle

First we look at the content MultiDex.install () method specific implementation of:

public static void install(Context context) {
        Log.i("MultiDex", "Installing application");
        if (IS_VM_MULTIDEX_CAPABLE) { //5.0 以上VM基本支持多dex，啥事都不用干
            Log.i("MultiDex", "VM has multidex support, MultiDex support library is disabled.");
        } else if (VERSION.SDK_INT < 4) { // 
            throw new RuntimeException("MultiDex installation failed. SDK " + VERSION.SDK_INT + " is unsupported. Min SDK version is " + 4 + ".");
        } else {
            ...
            doInstallation(context, new File(applicationInfo.sourceDir), new File(applicationInfo.dataDir), "secondary-dexes", "", true);
            ...
            Log.i("MultiDex", "install done");
        }
}

We can see from the above code, if the virtual machine itself supports loading multiple files dex, and consequently do not do it; if it does not support loading multiple dex (5.0 The following are not supported), then went doInstallation method.

private static void doInstallation(Context mainContext, File sourceApk, File dataDir, String secondaryFolderName, String prefsKeyPrefix, boolean reinstallOnPatchRecoverableException) throws IOException, IllegalArgumentException, IllegalAccessException, NoSuchFieldException, InvocationTargetException, NoSuchMethodException, SecurityException, ClassNotFoundException, InstantiationException {
    //获取非主dex文件
    File dexDir = getDexDir(mainContext, dataDir, secondaryFolderName);
    MultiDexExtractor extractor = new MultiDexExtractor(sourceApk, dexDir);
    IOException closeException = null;

    the try {
        // 1. This load method, not the first cache, time consuming 
       List Files = extractor.load (mainContext, prefsKeyPrefix, to false );
        the try {
        // 2. Installation DEX 
           installSecondaryDexes (Loader, dexDir, Files) ; 
       } 
    } 
}

 Look 1. MultiDexExtractor # load are performed specifically what:

List<? extends File> load(Context context, String prefsKeyPrefix, boolean forceReload) throws IOException {
    if (!this.cacheLock.isValid()) {
        throw new IllegalStateException("MultiDexExtractor was closed");
    } else {
        List files;
        if (!forceReload && !isModified(context, this.sourceApk, this.sourceCrc, prefsKeyPrefix)) {
            try {
                //读缓存的dex
                files = this.loadExistingExtractions (context, prefsKeyPrefix); 
            } the catch (IOException var6) { 
                Log.w ( "MultiDex", "the Failed to reload existing Extracted Files Secondary dex, Falling Back to Fresh Extraction" , var6);
                 // read cache dex failure, may be damaged, then re-read to extract the apk, like the else block 
                Files = the this .performExtractions ();
                 // save flag to sp, come next if left, do not go else 
                putStoredApkInfo ( context, prefsKeyPrefix, getTimestamp ( the this .sourceApk), the this .sourceCrc, Files); 
            } 
        } the else {
             //No cache, extract the apk read 
            Files = the this .performExtractions ();
             // save dex information to sp, come next if left, do not take the else 
            putStoredApkInfo (context, prefsKeyPrefix, getTimestamp ( the this .sourceApk), the this .sourceCrc , Files); 
        } 

        Log.i ( "MultiDex", "Load found" files.size + () + "Files Secondary DEX" );
         return Files; 
    } 
}

Find dex file, there are two logical, there is a cache is called loadExistingExtractions method, there is no buffer or cache read failure is called performExtractions method, and then cached. Use the cache, then the method must performExtractions should be very time-consuming, analyze the code:

Private List <MultiDexExtractor.ExtractedDex> performExtractions () throws IOException {
     // first determine naming format 
    String = extractedFilePrefix the this .sourceApk.getName () + " .classes " ;
     the this .clearDexDir (); 
    List <MultiDexExtractor.ExtractedDex> Files = new new the ArrayList (); 
    the ZipFile APK = new new the ZipFile ( the this .sourceApk); // APK zip format into 

    the try {
         int secondaryNumber = 2 ;
         // APK format has been changed to a zip, zip file decompression traversal, which is dex file,
        // name regularly, such as classes1.dex, class2.dex 
        for (the ZipEntry for the dexFile = apk.getEntry ( " classes " + secondaryNumber + " .dex " ); dexFile =! Null ; dexFile = apk.getEntry ( " classes " + + secondaryNumber " .dex " )) {
             // filename: xxx.classes1.zip 
            String fileName = extractedFilePrefix + secondaryNumber + " .zip " ;
             // create this classes1.zip file 
            MultiDexExtractor.ExtractedDex extractedFile = new new MultiDexExtractor.ExtractedDex(this.dexDir, fileName);
            //classes1.zip文件添加到list
            files.add(extractedFile);
            Log.i("MultiDex", "Extraction is needed for file " + extractedFile);
            int numAttempts = 0;
            boolean isExtractionSuccessful = false;

            while(numAttempts < 3 && !isExtractionSuccessful) {
                ++numAttempts;
                //This method is to write the file to the compressed file classes1.zip classes1.dex go up to three retries 
                Extract (APK, dexFile, extractedFile, extractedFilePrefix); 

             ... 
            } 
    // Returns the compressed file list dex 
    return Files; 
}

The logic here is to extract the apk, dex traverse the inside of the file, for example class1.dex, class2.dex, and then compressed into class1.zip, class2.zip ..., then returns a list of the zip file.

The first load will only performs decompression and compression process, the second coming in dex sp read the saved information directly back to file list, so the first start time consuming. dex list of found files, back annotation MultiDex # doInstallation above method 2, dex list of found files, and then call installSecondaryDexes method of installation, how to install it? Methods 19 points went to see more achieved SDK:

private static final class V19 {
    private V19() {
    }

    static void install(ClassLoader loader, List<? extends File> additionalClassPathEntries, File optimizedDirectory) throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, InvocationTargetException, NoSuchMethodException, IOException {
        Field pathListField = MultiDex.findField(loader, "pathList");//1 反射ClassLoader 的 pathList 字段
        Object dexPathList = pathListField.get(loader);
        ArrayList<IOException> suppressedExceptions = new ArrayList();
        // 2 扩展数组
        MultiDex.expandFieldArray(dexPathList, "dexElements", makeDexElements(dexPathList, new ArrayList(additionalClassPathEntries), optimizedDirectory, suppressedExceptions));
       ...
    }

    private static Object[] makeDexElements(Object dexPathList, ArrayList<File> files, File optimizedDirectory, ArrayList<IOException> suppressedExceptions) throws IllegalAccessException, InvocationTargetException, NoSuchMethodException {
        Method makeDexElements = MultiDex.findMethod(dexPathList, "makeDexElements", ArrayList.class, File.class, ArrayList.class);
        return (Object[])((Object[])makeDexElements.invoke(dexPathList, files, optimizedDirectory, suppressedExceptions));
    }
}

1. Field reflection of ClassLoader pathList

2. The method of pathList find makeDexElements corresponding field type

3. Expand the array by MultiDex.expandFieldArray dexElements this method, how extended? Look at the code:

   Private  static  void expandFieldArray (Object instance, String the fieldName, Object [] extraElements) throws a NoSuchFieldException, an IllegalArgumentException, IllegalAccessException { 
        Field, jlrField = FindField (instance, the fieldName); 
        Object [] Original = (Object []) ((Object []) jlrField .get (instance)); // remove the original dexElements array 
        Object [] combined = (Object [ ]) ((Object []) Array.newInstance (original.getClass () getComponentType (), original.length + extraElements.. length)); // the new array 
        System.arraycopy (original, 0, Combined, 0, to original.length); // original array contents are copied to the new array
        System.arraycopy (extraElements, 0, Combined, to original.length, extraElements.length); // DEX2, DEX3 ... copied to the new array 
        jlrField.set (instance, Combined); // will be reassigned to the new dexElements array 
    }

Is to create a new array, the original contents of the array (main DEX) to be added and the content (dex2, dex3 ...) into the copy, replacing the original reflection dexElements new array, as shown below:

Tinker principle hot fix is ​​also reflected by adding dex dex repaired to the array to go, except that the hot fix is ​​added to the array of front, rear and MultiDex is added to the array. Say this may not be well understood? Take a look at how to load a class ClassLoader understand ~

2.2  ClassLoader to load the class principle

Whether PathClassLoader or DexClassLoader, are inherited from BaseDexClassLoader, load class code BaseDexClassLoader, the specific file path as follows: /dalvik/src/main/java/dalvik/system/BaseDexClassLoader.java.

FIG Code:

1. dex construction method by passing the path created DexPathList.

2. ClassLoader The final method is to call findClass findClass method of DexPathList

Next, look at the source code /dalvik/src/main/java/dalvik/system/DexPathList.java DexPathList

DexPathList dexElements which defines an array, the method used in the findClass facie

Method findClass logic is very simple traversal dexElements array DexFile objects get inside, loaded by a class of DexFile loadClassBinaryName method.

Class is created by native final method, do not chase down, we are interested can look at native layer is how to create a Class object.

So the question is, which is only 5.0 or less this dexElements main dex (can be considered a bug), no dex2, dex3 ..., MultiDex is how to add dex2 into it?

The answer is reflected DexPathList of dexElements field, and then added to it dex2, of course, dexElements Element object is put inside, only dex2 path, Element format must be converted to the job, so the reflection DexPathList inside makeDexElements method, dex files into Element object can be.

dex2, dex3 ... by makeDexElements method to convert to the new Element Array, the final step is a reflection DexPathList dexElements field, the original and the new Element Element Array array merge, and then reflected assigned to dexElements variable, the last DexPathList dexElements variable contains the newly added dex in it.

makeDexElements method determines the type of file, say above when extracting the extracted apk dex dex obtained, and in turn compressed into dex zip, compressed into a zip, it will reach the second determination go. Think about it, in fact, not compressed into a zip dex, take the first question it also lacks judgment, that Google's MultiDex Why should dex into a zip it?

See in Android development master class in Zhangshao Wen also mentioned this:

In other words, the compression process is redundant, later we will introduce the headlines refer to Google's App MultiDex optimize this extra compression process, the follow-up program will introduce the headlines.

Here we first summarize the principle ClassLoader load of <==> ClassLoader.loadClass -> DexPathList.loadClass -> traverse dexElements array -> DexFile.loadClassBinaryName.

Popular point that: ClassLoader class load time is by traversing the array dex, dex file from which to load a class loaded successfully returns, failed to load is thrown Class Not Found exception.

2.3 MultiDex principle summary

After loading the class ClassLoader understand the principle, we can dexElements reflection array, the new dex added to the back of the array, thus ensuring ClassLoader to load the class time can be loaded from the new dex in the target class, after finishing the final analysis out of the diagram is as follows:

Three, MultiDex optimization

After we understand the MultiDex principle, we should consider how to optimize the MultiDex.

MultiDex focus optimization to solve the install process is time consuming, the main reason is that time-consuming extraction involves extracting dex apk, dex compression, the dex DexFile files into objects by the reflection, the reflective array replacement.

Think of this time-consuming optimization problem, we first think of asynchronous, that is, to open a sub-thread execution install operation, but this really feasible? After practice, we discovered that there is a big problem solutions.

3.1  sub-thread install (not recommended)

The idea of ​​this program are: to open a sub-page thread splash screen to perform MultiDex.install, then finished loading before jumping to the home page. Note that the splash screen pages Activity, including the splash screen page references to other classes must be in the main dex, or load these are not the main dex in the class will get an error Class Not Found before MultiDex.install.

How to ensure the splash screen in the main dex page inside of it? Here we can use to configure Gradle:

    defaultConfig {
        //分包，指定某个类在main dex
        multiDexEnabled true
        multiDexKeepProguard file('multiDexKeep.pro') // 打包到main dex的这些类的混淆规制，没特殊需求就给个空文件
        multiDexKeepFile file('maindexlist.txt') // 指定哪些类要放到main dex
    }

maindexlist.txt 文件指定哪些类要打包到主dex中，内容格式如下

com/lanshifu/launchtest/SplashActivity.class

但是，真正在已有项目中用使用这种方式，会发现编译运行在Android 4.4的机器上，启动闪屏页，加载完准备进入主页直接报错NoClassDefFoundError。NoClassDefFoundError 在这里出现知道就是主dex里面没有该类，一般情况下，这个方案的报错会出现在三方库的中，尤其是ContentProvider相关的逻辑。

应用进程不存在的情况下，从点击桌面应用图标，到应用启动（冷启动），大概会经历以下流程：

1. Launcher startActivity

2. AMS startActivity

3. Zygote fork 进程

4. ActivityThread main()
   4.1.  ActivityThread attach
   4.2. handleBindApplication
   4.3  attachBaseContext
   4.4. installContentProviders
   4.5. Application onCreate

5. ActivityThread 进入loop循环

6. Activity生命周期回调，onCreate、onStart、onResume...

整个启动流程我们能干预的主要是 4.3、4.5 和6，应用启动优化主要从这三个地方入手。理想状况下，这三个地方如果不做任何耗时操作，那么应用启动速度就是最快的，但是现实很骨感，很多开源库接入第一步一般都是在Application onCreate方法初始化，有的甚至直接内置ContentProvider，直接在ContentProvider中初始化框架，不给你优化的机会。

子线程install的方案之所以出现问题也正是因为上述的原理所说，即：ContentProvider初始化太早了，如果不在主dex中，还没启动闪屏页就已经crash了。

总结一下这种方案的缺点：

1. MultiDex加载逻辑放在闪屏页的话，闪屏页中引用到的类都要配置在主dex。

2. ContentProvider必须在主dex，一些第三方库自带ContentProvider，维护比较麻烦，要一个一个配置。

下面我们看一下今日头条是如何优化MultiDex的。

3.2 今日头条优化方案

1.在主进程Application 的 attachBaseContext 方法中判断如果需要使用MultiDex，则创建一个临时文件，然后开一个进程（LoadDexActivity），显示Loading，异步执行MultiDex.install 逻辑，执行完就删除临时文件并finish自己。

2. 主进程Application 的 attachBaseContext 进入while代码块，定时轮循临时文件是否被删除，如果被删除，说明MultiDex已经执行完，则跳出循环，继续正常的应用启动流程。

3.MultiDex执行完之后主进程Application继续走，ContentProvider初始化和Application onCreate方法，也就是执行主进程正常的逻辑。

注意：LoadDexActivity 必须要配置在main dex中。