Android Gradle common configuration

Gradle: DEX method multiDexEnabled exceeds 64K limit and gradle compiled OOM problem solving DEX

Methods exceed 64K limit

UNEXPECTED top- LEVEL EXCEPTION: 


com.android.dex.DexIndexOverflowException: Method ID not in [ 0 , 0xFFFF ] 65536  
   at com.android.dx.merge.DexMerger $ 6 .updateIndex (DexMerger.java: 502 ) 
   at com.android .dx.merge.DexMerger $ IdMerger.mergeSorted (DexMerger.java: 277 ) 
   at com.android.dx.merge.DexMerger.mergeMethodIds (DexMerger.java: 491 ) 
   at com.android.dx.merge.DexMerger.mergeDexes (DexMerger java: 168 ) 
   at com.android.dx.merge.DexMerger.merge (DexMerger.java: 189 )
   at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:454) 
   at com.android.dx.command.dexer.Main.runMonoDex(Main.java:302) 
   at com.android.dx.command.dexer.Main.run(Main.java:245) 
   at com.android.dx.command.dexer.Main.main(Main.java:214) 
   at com.android.dx.command.Main.main(Main.java:106)

 

If you're an Android developer, you've at least heard of Dalvik method of egg pain 64K limit. In a nutshell, in a DEX file, you can call a lot of ways, but you can call them front of 65,536, because that is all the space in the collection of the method call. Tripartite library method if your source code and mad pull cool Diao fried day exceeds this limit

To solve this problem, Android development community was to come up with some solutions, such as dmarcato this, there casidiablo this. They are possible, but need some more stringent conditions.

Finally, Google has decided to provide an official solution, released MultiDex support library at the time of the October 14, in the next few weeks gradle version also supports v0.14.0

Use MultiDex support library

If you're using Android Studio, this very simple to use. If not, it is strongly recommended that you migrated. Because Google will soon know at the Eclipse plug-in and the old Ant-based build system mode.

Step 1. Add depends on your support MultiDex library build.gradle

dependencies {
...
     compile 'com.android.support:multidex:'
... }

Step 2 in the open multiDexEnabled buildType or in productFlavor

defaultConfig { 
   ... 
    multiDexEnabled true 
    ... 
}

Step Application Register

Now, depending on your project situation, you have three choices:

    1, if you do not create your own Application class, in your manifest file

     AndroidManifest.xml configured android.support.multidex.MultiDexApplication it.

android:name="android.support.multidex.MultiDexApplication"

    2, if you have your own class of Application

     He inherited android.support.multidex.MultiDexApplication instead android.app.Application

    3, if your Application inherits the other class, and you do not want to change or not to change the way  rewrite attachBaseContext in accordance with the following method ()

public class MyApplication extends FooApplication { @Override 
   protected void attachBaseContext(Context base) {       
   super.attachBaseContext(base); 
   MultiDex.install(this); 
} 
}

 

No matter what you choose to above, will create more than similar size dex dex file instead of a single large file. When running back load all of these colleagues dex files.

Then compile the app when, Gradle will generate a number of files and a dex apk file so that you can run on the device or emulator.

Out of memory problems

For many dependent projects, probably because of the following compiler error interrupt

Error:Execution failed for task ':app:dexDebug'. ...
 Error Code: 3 Output: UNEXPECTED TOP-LEVEL ERROR: 
 java.lang.OutOfMemoryError: GC overhead limit exceeded at com.android.dx.cf.cst.ConstantPoolParser.parse0(ConstantPoolParser.java:326) 

Add the following code build.gralde android under the label can be solved

dexOptions { 
 incremental true
 javaMaxHeapSize "4g" 
 }

 

Application Starts Slowly 
After our experience, to add this support library, in most cases normal. This is for certain devices, such as Kindle Fire above, the application starts will be much slower than before. All classes loaded in the application startup takes a lot of time. This will lead to a black screen for some time, and even lead to ANR

Although most of the time this can be resolved DEX 64K problems, but should be reserved for use. When you try to use depend on it before, try deleting unnecessary and confusing to use ProGuard, if you have to use the program. Make sure to do a test on your old device

multidex caused performance problems - slow down the app launch speed

background

For the first layman to do some science. Andrews app is to be converted into a java .class file is written. Then the class file (and any dependent jar) classes.dex are compiled into a single file. Then any resource that a dex file and apk file (that is, ultimately downloaded from the app store things) the required combination.

See more  here  .

One drawback of this process is to compile a dex file system only allows a maximum of 65k methods. In the early Andrews, to approach the application of the method to solve this problem is to use Proguard upper limit of 65k to reduce unused code. However, this method has limitations, and only delayed nearly 65k limit of time for the production app.

To solve this problem, Google released a solution to a limit of 65k method in recent compatible library: multidexing. This method is very convenient and allows you to limit 65k method, but (as I said before), there is a very serious impact on performance, may start to slow down the speed of the app.

Set multidex

multidex is a fully mature document solutions. I strongly recommend to follow the  Android developer site  instructions to enable multidex. You can also consult on github  project sample

NoClassDefFoundError?!

Multidexing in the allocation for the project, you may see java.lang.NoClassDefFoundError at run time. This means that the app startup class is not in main dex file. Android SDK Build Tools 21.1 or later versions of Gradle Android plugin support for multidex of. Analyze your app to start a project and generate a list of classes in [buildDir] / intermediates / multi-dex / [buildType] /maindexlist.txt file in the plug-in uses Proguard. But this list is not 100% accurate, it may lose some of the classes needed to start the app.

YesClassDefFound

To solve this problem, you should set out those class in multidex.keep file, so that the compiler knows the main dex file in which to keep the class. .

• Multidex.keep create a file in the project directory.
• The class java.lang.NoClassDefFoundError cited in the report to multidex.keep file. (Note: 
  Do not directly modify the build directory of maindexlist.txt, this file is generated every time at compile time).
• Add the following script to build.gradle. This script will put in the time to compile your project multidex.keep and 
  together generated by Gradle maindexlist.txt.

android.applicationVariants.all { variant ->
    task "fix${variant.name.capitalize()}MainDexClassList" << {
        logger.info "Fixing main dex keep file for $variant.name"
        File keepFile = new File("$buildDir/intermediates/multi-dex/$variant.buildType.name/maindexlist.txt")
        keepFile.withWriterAppend { w ->
            // Get a reader for the input file
            w.append('\n')
            new File("${projectDir}/multidex.keep").withReader { r ->
                // And write data from the input into the output
                w << r << '\n'
            }
            logger.info "Updated main dex keep file for ${keepFile.getAbsolutePath()}\n$keepFile.text"
        }
    }
}
tasks.whenTaskAdded { task ->
    android.applicationVariants.all { variant ->
        if (task.name == "create${variant.name.capitalize()}MainDexClassList") {
            task.finalizedBy "fix${variant.name.capitalize()}MainDexClassList"
        }
    }
}

multidex app startup performance problems

If you use multidex, you need to be aware of its impact on app startup performance. We discovered this problem by starting time tracking app - the app user clicks on an icon to all pictures are downloaded and displayed to the user during this time. Once multidex enabled our app startup time will increase about 15% on all running Kitkat (4.4) and the following equipment. For more information see Carlos Sessa's Lazy Loading Dex files.

This is because Android 5.0 and later used when a run is called ART, it inherently supports loading multiple dex files from the application's apk file.

 

Multidex app solve performance problems start

In the app start to show all the pictures of the gap, there are not many Proguard detected class, so they will not be saved into the main dex file. The question now is, how can we know what calss during app launch is loaded it?

Fortunately, we have findLoadedClass in ClassLoader methods. Our way is when the app starts running when the end of the check-ups once. If the second dex files have any class during app startup to load, then by adding calss name to file multidex.keep way to move them to the main dex file. My project cases have details of the implementation, but you can also do this:

• Run the following util class getLoadedExternalDexClasses in the end you think the place to start app
• Add this to the list above method returns to your multidex.keep file and recompile.

public class MultiDexUtils {
    private static final String EXTRACTED_NAME_EXT = ".classes";
    private static final String EXTRACTED_SUFFIX = ".zip";

    private static final String SECONDARY_FOLDER_NAME = "code_cache" + File.separator +
            "secondary-dexes";

    private static final String PREFS_FILE = "multidex.version";
    private static final String KEY_DEX_NUMBER = "dex.number";

    private SharedPreferences getMultiDexPreferences(Context context) {
        return context.getSharedPreferences(PREFS_FILE,
                Build.VERSION.SDK_INT < Build.VERSION_CODES.HONEYCOMB
                        ? Context.MODE_PRIVATE
                        : Context.MODE_PRIVATE | Context.MODE_MULTI_PROCESS);
    }

    /**
     * get all the dex path
     *
     * @param context the application context
     * @return all the dex path
     * @throws PackageManager.NameNotFoundException
     * @throws IOException
     */
    public List<String> getSourcePaths(Context context) throws PackageManager.NameNotFoundException, IOException {
        final ApplicationInfo applicationInfo = context.getPackageManager().getApplicationInfo(context.getPackageName(), 0);
        final File sourceApk = new File(applicationInfo.sourceDir);
        final File dexDir = new File(applicationInfo.dataDir, SECONDARY_FOLDER_NAME);

        final List<String> sourcePaths = new ArrayList<>();
        sourcePaths.add(applicationInfo.sourceDir); //add the default apk path

        //the prefix of extracted file, ie: test.classes
        final String extractedFilePrefix = sourceApk.getName() + EXTRACTED_NAME_EXT;
        //the total dex numbers
        final int totalDexNumber = getMultiDexPreferences(context).getInt(KEY_DEX_NUMBER, 1);

        for (int secondaryNumber = 2; secondaryNumber <= totalDexNumber; secondaryNumber++) {
            //for each dex file, ie: test.classes2.zip, test.classes3.zip...
            final String fileName = extractedFilePrefix + secondaryNumber + EXTRACTED_SUFFIX;
            final File extractedFile = new File(dexDir, fileName);
            if (extractedFile.isFile()) {
                sourcePaths.add(extractedFile.getAbsolutePath());
                //we ignore the verify zip part
            } else {
                throw new IOException("Missing extracted secondary dex file '" +
                        extractedFile.getPath() + "'");
            }
        }

        return sourcePaths;
    }

    /**
     * get all the external classes name in "classes2.dex", "classes3.dex" ....
     *
     * @param context the application context
     * @return all the classes name in the external dex
     * @throws PackageManager.NameNotFoundException
     * @throws IOException
     */
    public List<String> getExternalDexClasses(Context context) throws PackageManager.NameNotFoundException, IOException {
        final List<String> paths = getSourcePaths(context);
        if(paths.size() <= 1) {
            // no external dex
            return null;
        }
        // the first element is the main dex, remove it.
        paths.remove(0);
        final List<String> classNames = new ArrayList<>();
        for (String path : paths) {
            try {
                DexFile dexfile = null;
                if (path.endsWith(EXTRACTED_SUFFIX)) {
                    //NOT use new DexFile(path), because it will throw "permission error in /data/dalvik-cache"
                    dexfile = DexFile.loadDex(path, path + ".tmp", 0);
                } else {
                    dexfile = new DexFile(path);
                }
                final Enumeration<String> dexEntries = dexfile.entries();
                while (dexEntries.hasMoreElements()) {
                    classNames.add(dexEntries.nextElement());
                }
            } catch (IOException e) {
                throw new IOException("Error at loading dex file '" +
                        path + "'");
            }
        }
        return classNames;
    }
/**
     * Get all loaded external classes name in "classes2.dex", "classes3.dex" ....
     * @param context
     * @return get all loaded external classes
     */
    public List<String> getLoadedExternalDexClasses(Context context) {
        try {
            final List<String> externalDexClasses = getExternalDexClasses(context);
            if (externalDexClasses != null && !externalDexClasses.isEmpty()) {
                final ArrayList<String> classList = new ArrayList<>();
                final java.lang.reflect.Method m = ClassLoader.class.getDeclaredMethod("findLoadedClass", new Class[]{String.class});
                m.setAccessible(true);
                final ClassLoader cl = context.getClassLoader();
                for (String clazz : externalDexClasses) {
                    if (m.invoke(cl, clazz) != null) {
                        classList.add(clazz.replaceAll("\\.", "/").replaceAll("$", ".class"));
                    }
                }
                return classList;
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

in conclusion

Here we observe on multiple devices to enhance the effect of start-up performance. The first column (blue) is not multidexing benchmark app startup time. You can see a significant increase in the second column (red), 
which is enabled multidex but no app startup time of any other additional work. The third column (green) is on the multidex start time and use the app to enhance our approach. You can see in Figure, 
App startup time down to the level before the multidex open, or even lower. Try it yourself, you should be able to observe the performance boost. 

postscript

Just because you can does not mean you should. You should multidex as a last resort because there is a big impact on app startup time but also to solve this problem you need to maintain additional code and solve strange errors 
(for example: java.lang.NoClassDefFoundError). Once the limit is reached 65k number of ways, we should avoid to use multidex to prevent performance issues. We are constantly checking sdk used to identify many 
more than can be removed or reconstructed useless code. Only then is still no way when we consider multidex. At that time our code quality will have a qualitative leap. Do not use direct multidex, first keep the code clean and 
reuse existing form, or refactor the code to avoid the 65k limit method.