android dynamically load external classes

Basic Information

Author：kaedea
GitHub：android-dynamical-loading

We have adopted the dynamic loading technology in the Android project from a very early stage. The main purpose is to achieve the function of allowing users to upgrade the application without reinstalling the APK, which can not only greatly improve the coverage of the new version of the application, but also reduce the The pressure on the server to be compatible with the interface of the old version, and if you can quickly fix some online bugs.

This technology is not a conventional Android development method, and there is no perfect solution in the early days. From "unconsciousness" to stable production, I have always wanted to write some documents about this, which is also the origin of this log. The ideas may not be connected very well, and I will slowly correct them in the future.

technical background

It is common to configure some parameters through the server, and the Android APP obtains these parameters and then makes corresponding logic.

For example, most APPs now have a startup page. If there are some important festivals, the APP server will configure some pictures related to the season. When the APP starts, the original startup picture will be replaced with these new pictures, so that you can Improve user experience.

Furthermore, when an early individual developer publishes an app on the Android market, if the app contains advertisements, it may fail the review. Then, configure a switch on the server and turn off the switch when reviewing the application, so that the application will not display advertisements; after the Android market has passed the review, turn on the advertisement switch of the server, in this way to avoid market review. .

The road is one foot high and the devil is one foot high . The Android market begins to scan the Manifest and even the dex file in the APK to see if there is any ad code in the developer's APK package. If there is, it may fail the review.

I was afraid that the method of configuring switch parameters through the server would not work, and the developers began to think, "In this case, can we not write the advertisement code in the APK first, and then download the advertisement code from the server and run it when the user runs the APP. What about realistic advertising?". The answer is yes, this is dynamic loading:

When the program is running, some executable files that do not exist in the program itself are loaded and the code logic in these files is run.

It looks like the app downloads some code from the server and then executes it!

Dynamic Loading Technology in Traditional PC Software

Dynamic loading technology is widely used in the field of PC software, such as the screenshot function of the input method. The newly installed input method software may not have the screenshot function. When you use it for the first time, the input method will first download and install the screenshot software from the server, and then execute the screenshot function.

In addition, there are a large number of DLL files (Dynamic Link Library) in the installation directory of many PC software, and the PC software executes specific functions by calling the code in these DLLs, which is a dynamic loading technology.

Students who are familiar with Java should know better. The executable file of Java is Jar, which runs on the JVM on the virtual machine. The virtual machine loads the Jar file through the ClassLoader and executes the code inside. Therefore, the Java program can also achieve the purpose of dynamic loading by dynamically calling the Jar file.

Dynamic Loading Technology of Android Application

Android applications are similar to Java programs, except that the virtual machine is replaced by Dalvik/ART, and Jar is replaced by Dex. When the Android APP is running, can we also achieve dynamic loading by downloading a new application or by calling an external Dex file?

However, it is not so easy to implement on Android. If you download a new APK, it will not work without installing the APK. If you let the user manually install the APK and then start it, it is not like dynamic loading, it is purely that the user installs a new application and then starts the new application.

There is absolutely no problem in dynamically calling external Dex files. There are often one or more Dex files in the APK file. Every code we write will be compiled into these files. When the Android application is running, the functions of the application are completed by executing these Dex files. Although once an APK is built, we cannot replace the Dex file inside, but we can achieve dynamic loading by loading an external Dex file, which can be placed in external storage or downloaded from the network.

Dynamically loaded definitions

Before getting to the point, here is a brief definition of dynamic loading technology. The real dynamic loading should be

When the application is running, it implements some specific functions by loading some executable files that do not exist locally
These executables are replaceable
Replacing static resources (such as changing the startup image, changing the theme, or using the server parameter switch to control the hidden reality of advertisements, etc.) does not belong to dynamic loading
The core idea of dynamic loading in Android is to dynamically call external Dex files . In extreme cases, the Dex file included in the Android APK itself is just an entry (or an empty shell) of a program, and all functions are downloaded from the server by downloading the latest Dex file complete

Android dynamically loaded types

In Android projects, dynamic loading can be roughly divided into two types according to the difference in technical implementation:

Dynamically load .so library
Dynamically load .dex/jar/apk (this is what dynamic loading generally refers to now)

First, dynamic loading is actually used in the NDK in Android, which dynamically loads the .so library and calls its encapsulated method through JNI. The latter is generally compiled from C++ and runs on the Native layer, which is much more efficient than the Java code executed in the virtual machine, so Android often dynamically loads the .so library to complete some performance-demanding tasks (such as T9 search, or Bitmap decoding, image Gaussian blurring, etc.). In addition, since the .so library is compiled from C++, it can only be decompiled into assembly code, which is more difficult to crack than Smali, so the .so library can also be used in the security field. It should be noted that in general, we package the .so library together inside the APK, but the .so library can actually be loaded from external storage files.

Second, "Dynamic loading of .dex/jar/apk based on ClassLoader" is what we mentioned above "dynamically load Dex packages compiled from Java code in Android and execute the code logic", which is conventional Android development. A technology that is rarely used. At present, most of the articles on the Internet refer to this kind of dynamic loading.

In an Android project, all Java code will be compiled into a Dex package. When the Android application runs, it works by executing the business code logic in the Dex package. Using dynamic loading technology, the external Dex package can be loaded when the Android application is running, and the purpose of upgrading the application (changing the code logic) without installing the new APK file can be achieved by downloading the new Dex package through the network and replacing the original Dex package. At the same time, the use of dynamic loading technology will generally make Android development more complicated. This development method is not officially recommended, not the current mainstream Android development method, and foreign developers on Github and StackOverflow are also not very concerned about this. Interested, foreign-related tutorials are even more pitiful. At present, only in Datianchao have more in-depth research and applications, especially for some SDK component projects and BAT family projects. The related open source projects on Github are basically Chinese in Maintenance, occasionally a few foreigners request to update the English documentation.

The general process of Android dynamic loading

Regardless of the above dynamic loading, the basic principle is to load some external executable files when the program is running, and then call a method of these files to execute business logic. It should be noted that because the file is executable (so library or dex package, that is, a dynamic link library), due to security issues, Android does not allow direct loading of noexec (non-executable) storage paths such as external storage of mobile phones. executable file on .

For these external executable files, before calling them in the Android application, they must be copied to the data/packagename/internal storage file path to ensure that the library will not be maliciously modified or intercepted by third-party applications, and then load them Go to the current running environment and call the required method to execute the corresponding logic, so as to realize the dynamic call.

The general process of dynamic loading is:

Copy the executable file (.so/dex/jar/apk) to the internal storage of the application APP

load executable

Call specific methods to execute business logic

The following is a more in-depth introduction to the implementation of these two dynamic loading methods.

Dynamically load .so library

The dynamic loading of the .so library should be the earliest dynamic loading of Android, but the .so library can be stored not only in the APK file, but also in external storage. In Android development, there are not many cases of replacing the .so library, but the size of the APK can be reduced by moving the .so library to the outside of the APK. After all, the size of many .so files is very large.

For detailed application methods, please refer to the follow-up log Android dynamic loading supplementary loading SO library of SD card

Dynamically load .dex/jar/apk files

This is the kind of Android dynamic loading technology we often talk about. Later, when we talk about "dynamic loading", if it is not specified, it will be this by default. To facilitate the distinction between concepts, some terminology should be clarified before the discussion.

Main APK and Plugin APK

Main APK: main project APK, host APK (Host APK), that is, the main project we want to use dynamic loading technology;
Plugin APK: Plugin, separate from the main project, we can dynamically load the modules loaded into the main project, one main APK can load multiple plugin APKs at the same time;

Basics: ClassLoader and Dex

The basis of dynamically loading .dex/jar/apk files is the class loader ClassLoader, and its package path is java.lang, which shows its importance. The virtual machine is the Class that needs to be used during the loading period through the class loader. This is Java The basis for program operation. For the working mechanism of the class loader ClassLoader, please refer to the working mechanism of Android dynamically loaded basic ClassLoader

Now there are a variety of open source projects based on Android dynamic loading based on ClassLoader on the Internet. Most of the core ideas have the same goal. According to the complexity and the specific implementation framework, they can be roughly divided into the following three modes.

Simple dynamic loading mode

It is very simple for Android to use ClassLoader to dynamically load external Dex files at runtime. You can change the code logic of the APP without overwriting and installing a new APK. However, it is difficult for Android to use the res resources in the plug-in APK, which means that resources such as new XML layouts cannot be used. At the same time, because the local Manifest manifest file cannot be changed, new Activity and other components cannot be started.

However, you can put all the res resources to be used in the main APK first, and at the same time write all the required activities into the Manifest first, and only update the code by dynamically loading the res resources. If you need to change the UI interface, you can By bypassing XML layouts by creating layouts in pure Java code, Fragments can also be used instead of Activities.

To a certain extent, the simple dynamic loading function can already meet some business needs, especially some early Android projects. At that time, Android technology was not very mature, and early Android devices had a lot of compatibility problems ( Students who have done Android 1.6 compatibility may have a deep understanding), only this simple loading method can run stably. The framework of this mode is more suitable for some projects with relatively few UI changes, such as game SDK, which basically only has the login and registration interface, and basically does not change, and often only code logic is updated.

For detailed application methods, please refer to the follow-up log Android Dynamic Loading Getting Started Simple Loading Mode

Using Proxy Activity Mode

From this stage, it has a little "black technology" flavor. For example, we can let the current Android application start some "new" activities through dynamic loading, and even start a "new" APK without installation (the original APK is called the "main APK", and the new APK is called the "plug-in APK"). The main APK needs to first register an empty shell Activity to proxy the life cycle of the Activity that executes the plug-in APK.

Mainly have the following characteristics

The main APK can launch plugin APKs that are not installed;
Plugin APK can also be installed and started as a normal APK;
The plug-in APK can call some functions in the main APK;
The main APK and the plug-in APK must be connected to a set of specified interfaces to realize the above functions;

For detailed application methods, please refer to the follow-up log Android dynamic loading advanced agent Activity mode

Dynamically create Activity mode

Oh my god, at this stage, it is really the field of "black technology". You can imagine "download a Flappy Bird APK from the Internet and run the game directly without installing it", or "run two or more WeChat at the same time" .

This stage has the following characteristics

The main APK can launch a plugin APK that is not installed;
The plug-in APK can be any third-party APK without accessing the specified interface;

For detailed application methods, please refer to the follow-up log Android dynamic loading black technology to dynamically create Activity mode

Why we use dynamic loading technology

To be honest, I don't know what the product requires! (The policeman is him. He only asked me if I could achieve it, and he didn't ask me if it was difficult to achieve...)

In Android development, the first to use dynamic loading technology should be the SDK project. Now there are a lot of Android SDK projects on the Internet, such as Google's Goole Play Service, which provides developers with functions such as payment and maps, and some SDKs in the Android game market, which are used to provide account and payment functions to game developers. Like ordinary Android applications, these SDK projects also need to be upgraded. For example, our SDK 1.0 version is used in other Android applications, and then released to the Android market. Now we found that SDK 1.0 has some urgent bugs, so we upgraded a version of SDK 1.1. There is no other way but to let people reconnect to version 1.1 and release it to the market. If we have SDK 1.2, 1.3 and other versions, it would be okay to let people re-connect each version, but if the product cares about the experience, he will ask, "Although I don't understand technology, I want to know Is there a way to allow people to access our SDK only once, so that when we release a new SDK version in the future, their projects can also be automatically upgraded?" He replied, "Yes, it can be done using dynamic loading technology, It's just (the development workload will increase dramatically...)", "Then use it, we have to maximize the experience of the product".

Well, I don't mean to hack the product, the current team's product is not bad, but a conversation similar to the above did happen in my project in 2013. It is proposed here only to emphasize the role of dynamic loading technology in Android projects and the resulting costs .

effect and cost

Everything has two sides, especially this kind of unofficially supported unconventional development method. We must weigh its role and cost before adopting it. If you decide to use dynamic loading technology, I personally recommend that you can use this framework in some relatively independent modules of the actual project. After solving some problems encountered, slowly introduce them into the core modules of the project; if you encounter some problems that cannot be The problem to leapfrog is to have alternatives that can be put into production quickly.

effect

Avoid the upgrade process of APK overlay installation, improve user experience, and circumvent some Android market restrictions by the way;
Dynamically repair some emergency bugs of the application , and make the last guarantee;
When the application volume is too large, some modules can be separated in the form of plug-ins through dynamic loading, which can reduce the volume of the main project, improve the compilation speed of the project , and allow the main project and the plug-in project to be developed in parallel;
Plug-in modules can be initialized only when needed in a lazy-loading manner, thereby improving the startup speed of the application ;
From the perspective of project management, the method of dividing plug-in modules achieves code separation from the project level , which greatly reduces the coupling between modules. If there is a bug, it is easy to locate the problem;
When promoting other applications on Android applications , dynamic loading technology can be used to allow users to experience the functions of new applications first, instead of downloading and installing a new APK;
Reduce the number of methods of the main project DEX, the 65535 problem has completely become a history (although it is easy to open MultiDex in Android Studio now, this problem is not difficult to solve);

cost

The development method may become strange and cumbersome, which is different from the conventional development method;
With the deepening of the complexity of the dynamic loading framework, the construction process of the project also becomes complicated. It is possible to build the main project and the plug-in project separately and then integrate them together;
Since the plug-in project is independently developed, when the main project loads the plug-in and runs, the plug-in operating environment is completely different, the code logic is prone to bugs, and debugging the plug-in in the main project is very cumbersome;
Unconventional development methods, some frameworks use reflection to forcibly call some Android system Framework layer codes, some Android ROMs may have changed these codes, so there is a risk of compatibility problems, especially in some old Android devices and some Samsung on the mobile phone;
Dynamically loaded plugins often have some compatibility issues when using system resources (especially themes), especially some Samsung phones;

Other techniques for dynamically modifying code

All of the above are based on the dynamic loading technology of ClassLoader (except for loading SO libraries). One of the characteristics of using ClassLoader is that if the program is not restarted, the classes that have been loaded once cannot be reloaded. Therefore, if you use ClassLoader to dynamically upgrade the APP or dynamically fix the BUG, you need to restart the APP to take effect.

In addition to using ClassLoader, you can also use jni hook to modify the execution code of the program. The former is operated on the virtual machine, while the latter is already working at the Native level, directly modifying the memory address of the application runtime, so when using the jni hook method, it can take effect without re-application.

At present, the most popular projects using the jni hook scheme are Ali's dexposed and AndFix. Interested students can refer to the analysis and comparison of the major hot patch schemes .

Dynamically load open source projects

dynamic-load-apk
android-pluginmgr
Direct-Load-apk
360 DroidPlugin
携程网 DynamicAPK
Nuwa