Mobile Architecture 02 - Tinker Hotfix

Tinker is currently the most compatible hotfix framework among open source frameworks.

Tinker's Github address: https://github.com/Tencent/tinker

1. Why use Tinker

1. Overall Comparison

There are many hot patch solutions currently on the market, among which the more famous ones are Tinker of WeChat, AndFix of Alibaba, Robust of Meituan and the super patch solution of QZone. So why use Tinker? Let's do a simple comparison first.

Tinker	QZone	AndFix	Robust
class replacement	yes	yes	no
So replace	yes	no	no
resource replacement	yes	yes	no
Full platform support	yes	yes	yes
effective immediately	no	no	yes
performance loss	smaller	larger	smaller
Patch pack size	smaller	larger	generally
development transparency	yes	yes	no
the complexity	lower	lower	complex
gradle support	yes	no	no
Rom volume	larger	smaller	smaller
Success rate	higher	higher	generally

In general:

AndFix is a repair solution for the native layer. The advantage is that it takes effect immediately. The disadvantage is that it can only be repaired and the success rate is not high;
Robust is also a repair solution for the native layer. The advantage is that it takes effect immediately and has a high success rate. The disadvantage is that it can only be repaired;
Qzone (non-open source) is a repair solution for the Java layer. The advantage is that it can repair classes and resources. The disadvantage is that the patch package is large, the performance is poor, and it cannot take effect immediately;
Tinker is also a repair solution for the Java layer. The advantage is that it can repair classes, so and resources, and the patch package is small. The disadvantage is that the Rom is large and cannot take effect immediately;

2. Realize the comparison

The hot fix is based on the hook technology, which can dynamically modify the code in the memory, but cannot modify the dex file in the SD card.

AndFix

AndFix uses the native hook method, which directly dalvik_replaceMethodreplaces the implementation of the method in the class. Since it does not replace the class as a whole, and the relative address of the field in the class is determined when the class is loaded, AndFix cannot support the addition or deletion of filed (by replacing initand clinitonly modifying the value of the field).

QZone

The QZone solution is not open source, but HotFix on github takes the same approach. This scheme uses the method of classloader to generate a dex file for the repaired class, and insert the dex file into the front of the dex array of the system. When the system loads this class, it will first search from the signature of the dex array, and load it into the memory after it is found. , so as to achieve class replacement.

However, by using the classloader directly, the repaired class and its reference class are not in the same dex, which will bring about unexpected DEX problem.

Because, when the apk is installed, the virtual machine (dexopt) will optimize the classes.dex in the apk into an odex file. During optimization, if the startup parameter verify is true, dvmVerifyClass will be executed to verify the class. If the direct reference class of a class and calzz are in the same dex, then the class will be marked with CLASS_ISPREVERIFIED. When this class is loaded, since it has the CLASS_ISPREVERIFIED flag, it will first perform dex verification to verify whether its referenced class is in the same dex as it, and if it is not in the same dex unexpected DEX problem.

In order to solve the problem unexpected DEX problem, QZone uses the method of instrumentation, adding the following code to each class to prevent them from being marked with CLASS_ISPREVERIFIED:

if (ClassVerifier.PREVENT_VERIFY) {
    System.out.println(AntilazyLoad.class);//AntilazyLoad处于一个独立的dex中
}

However, it is very meaningful for the Android system to do class verification, and the use of instrumentation in Dalvik and Art will cause some problems.

Dalvik; In the dexopt process, if the class verify is passed, the pre-verify flag will be written, and the odex file will be written after optimization. The optimize here mainly includes inline and quick instruction optimization.
Art; Art adopts a new method, and instrumentation has no effect on the execution efficiency of the code. However, if the class in the patch modifies the class variable or method, it may cause the problem of memory address confusion. In order to solve this problem, we need to add the parent class of the class that modifies variables, methods, and interfaces, and all classes that call this class into the patch package. This can lead to a dramatic increase in patch size.

Tinker

Tinker generates the difference path.dex through the old and new Dex at compile time. At runtime, the difference patch.dex is restored to the new Dex from the old Dex of the original installation package. This process may be time-consuming and memory-consuming, so we put it in a separate background process: patch. In order to keep the patch package as small as possible, WeChat has developed the DexDiff algorithm, which deeply utilizes the Dex format to reduce the size of the difference. Its granularity is each item in the Dex format, which can make full use of the original Dex information, while the granularity of BsDiff is file, and the granularity of AndFix/QZone is class.

2. How to use Tinker

1. Add gradle dependencies

Configure the version and ID of tinker in gradle.properties

TINKER_VERSION = 1.9.1
TINKER_ID = 100

In the project's build.gradle, add tinker-patch-gradle-pluginthe dependencies

buildscript {
    dependencies {
         //添加tinker热修复插件
        classpath "com.tencent.tinker:tinker-patch-gradle-plugin:${TINKER_VERSION}"
    }
}

Add tinker's gradle file: tinker.gradle

/*----------------------------------------配置tinker插件----------------------------------------------*/
apply plugin: 'com.tencent.tinker.patch'
//获取提交git的版本号,作为tinkerid
def gitSha() {
    try {
        String gitRev = 'git rev-parse --short HEAD'.execute(null, project.rootDir).text.trim()
        if (gitRev == null) {
            throw new GradleException("can't get git rev, you should add git to system path or just input test value, such as 'testTinkerId'")
        }
        return gitRev
    } catch (Exception e) {
        throw new GradleException("can't get git rev, you should add git to system path or just input test value, such as 'testTinkerId'")
    }
}

def bakPath = file("${buildDir}/bakApk/")

/**
 * you can use assembleRelease to build you base apk
 * use tinkerPatchRelease -POLD_APK=  -PAPPLY_MAPPING=  -PAPPLY_RESOURCE= to build patch
 * add apk from the build/bakApk
 */
ext {
    //for some reason, you may want to ignore tinkerBuild, such as instant run debug build?
    tinkerEnabled = true

    //for normal build
    //old apk file to build patch apk
    tinkerOldApkPath = "${bakPath}/app-debug-0430-23-10-22.apk"
    //proguard mapping file to build patch apk
    tinkerApplyMappingPath = "${bakPath}/app-debug-1018-17-32-47-mapping.txt"
    //resource R.txt to build patch apk, must input if there is resource changed
    tinkerApplyResourcePath = "${bakPath}/app-debug-0430-23-10-22-R.txt"

    //only use for build all flavor, if not, just ignore this field
    tinkerBuildFlavorDirectory = "${bakPath}/app-1018-17-32-47"
}


def getOldApkPath() {
    return hasProperty("OLD_APK") ? OLD_APK : ext.tinkerOldApkPath
}

def getApplyMappingPath() {
    return hasProperty("APPLY_MAPPING") ? APPLY_MAPPING : ext.tinkerApplyMappingPath
}

def getApplyResourceMappingPath() {
    return hasProperty("APPLY_RESOURCE") ? APPLY_RESOURCE : ext.tinkerApplyResourcePath
}

def getTinkerIdValue() {
    return hasProperty("TINKER_ID") ? TINKER_ID : gitSha()
}


def buildWithTinker() {
    return hasProperty("TINKER_ENABLE") ? TINKER_ENABLE : ext.tinkerEnabled
}

def getTinkerBuildFlavorDirectory() {
    return ext.tinkerBuildFlavorDirectory
}

if (buildWithTinker()) {
    apply plugin: 'com.tencent.tinker.patch'

    tinkerPatch {
        oldApk = getOldApkPath()
        ignoreWarning = false
        useSign = true
        tinkerEnable = buildWithTinker()
        buildConfig {
            applyMapping = getApplyMappingPath()
            applyResourceMapping = getApplyResourceMappingPath()
            tinkerId = getTinkerIdValue()
            keepDexApply = false
            isProtectedApp = false
            supportHotplugComponent = false
        }

        dex {
            dexMode = "jar"
            pattern = ["classes*.dex", "assets/secondary-dex-?.jar"]
            loader = [ "tinker.sample.android.app.BaseBuildInfo"]
        }

        lib {
            /**
             * optional，default '[]'
             * what library in apk are expected to deal with tinkerPatch
             * it support * or ? pattern.
             * for library in assets, we would just recover them in the patch directory
             * you can get them in TinkerLoadResult with Tinker
             */
            pattern = ["lib/*/*.so"]
        }

        res {
            pattern = ["res/*", "assets/*", "resources.arsc", "AndroidManifest.xml"]
            ignoreChange = ["assets/sample_meta.txt"]
            largeModSize = 100
        }

        packageConfig {
            configField("patchMessage", "tinker is sample to use")
            configField("platform", "all")
            configField("patchVersion", "1.0")
        }

        sevenZip {
            zipArtifact = "com.tencent.mm:SevenZip:1.1.10"
        }
    }

    List<String> flavors = new ArrayList<>();
    project.android.productFlavors.each { flavor ->
        flavors.add(flavor.name)
    }
    boolean hasFlavors = flavors.size() > 0
    def date = new Date().format("MMdd-HH-mm-ss")

    android.applicationVariants.all { variant ->

        def taskName = variant.name

        tasks.all {
            if ("assemble${taskName.capitalize()}".equalsIgnoreCase(it.name)) {

                it.doLast {
                    copy {
                        def fileNamePrefix = "${project.name}-${variant.baseName}"
                        def newFileNamePrefix = hasFlavors ? "${fileNamePrefix}" : "${fileNamePrefix}-${date}"

                        def destPath = hasFlavors ? file("${bakPath}/${project.name}-${date}/${variant.flavorName}") : bakPath
                        from variant.outputs.first().outputFile
                        into destPath
                        rename { String fileName ->
                            fileName.replace("${fileNamePrefix}.apk", "${newFileNamePrefix}.apk")
                        }

                        from "${buildDir}/outputs/mapping/${variant.dirName}/mapping.txt"
                        into destPath
                        rename { String fileName ->
                            fileName.replace("mapping.txt", "${newFileNamePrefix}-mapping.txt")
                        }

                        from "${buildDir}/intermediates/symbols/${variant.dirName}/R.txt"
                        into destPath
                        rename { String fileName ->
                            fileName.replace("R.txt", "${newFileNamePrefix}-R.txt")
                        }
                    }
                }
            }
        }
    }
    project.afterEvaluate {
        //sample use for build all flavor for one time
        if (hasFlavors) {
            task(tinkerPatchAllFlavorRelease) {
                group = 'tinker'
                def originOldPath = getTinkerBuildFlavorDirectory()
                for (String flavor : flavors) {
                    def tinkerTask = tasks.getByName("tinkerPatch${flavor.capitalize()}Release")
                    dependsOn tinkerTask
                    def preAssembleTask = tasks.getByName("process${flavor.capitalize()}ReleaseManifest")
                    preAssembleTask.doFirst {
                        String flavorName = preAssembleTask.name.substring(7, 8).toLowerCase() + preAssembleTask.name.substring(8, preAssembleTask.name.length() - 15)
                        project.tinkerPatch.oldApk = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-release.apk"
                        project.tinkerPatch.buildConfig.applyMapping = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-release-mapping.txt"
                        project.tinkerPatch.buildConfig.applyResourceMapping = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-release-R.txt"

                    }

                }
            }

            task(tinkerPatchAllFlavorDebug) {
                group = 'tinker'
                def originOldPath = getTinkerBuildFlavorDirectory()
                for (String flavor : flavors) {
                    def tinkerTask = tasks.getByName("tinkerPatch${flavor.capitalize()}Debug")
                    dependsOn tinkerTask
                    def preAssembleTask = tasks.getByName("process${flavor.capitalize()}DebugManifest")
                    preAssembleTask.doFirst {
                        String flavorName = preAssembleTask.name.substring(7, 8).toLowerCase() + preAssembleTask.name.substring(8, preAssembleTask.name.length() - 13)
                        project.tinkerPatch.oldApk = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-debug.apk"
                        project.tinkerPatch.buildConfig.applyMapping = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-debug-mapping.txt"
                        project.tinkerPatch.buildConfig.applyResourceMapping = "${originOldPath}/${flavorName}/${project.name}-${flavorName}-debug-R.txt"
                    }

                }
            }
        }
    }
}

Then in the app's build.gradle, we need to add tinker's library dependencies and apply tinker's gradle file.

apply from: 'tinker.gradle'
android {
    defaultConfig {
        ...
        multiDexEnabled true
    }

//以下是签名信息，测试环境可以省略
//    signingConfigs {
//        release {
//            try {
//                storeFile file("./keystore/release.keystore")
//                storePassword "testres"
//                keyAlias "testres"
//                keyPassword "testres"
//            } catch (ex) {
//                throw new InvalidUserDataException(ex.toString())
//            }
//        }
//
//        debug {
//            storeFile file("./keystore/debug.keystore")
//        }
//    }
//    buildTypes {
//        release {
//            minifyEnabled true
//            signingConfig signingConfigs.release
//            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
//        }
//        debug {
//            debuggable true
//            minifyEnabled false
//            signingConfig signingConfigs.debug
//        }
//    }

//支持jni的热修复
//    sourceSets {
//        main {
//            jniLibs.srcDirs = ['libs']
//        }
//    }
}

dependencies {
    ...
    //gradle 3.0.0的一定要使用如下的依赖
    implementation("com.tencent.tinker:tinker-android-lib:${TINKER_VERSION}") { changing = true }
    annotationProcessor("com.tencent.tinker:tinker-android-anno:${TINKER_VERSION}") { changing = true }
    compileOnly("com.tencent.tinker:tinker-android-anno:${TINKER_VERSION}") { changing = true }
    //Android5.0以下需要添加multidex库.如果其他的库包含了multidex库，就不需要额外添加了。
    implementation "com.android.support:multidex:1.0.1"
}

2. Generate diff package

Use Gradle-app-Tasks-build-assembleDebug in the upper right corner of AS to compile, it will generate apk file and R file in app-build-bakApk, and install it on the phone;
Change the tinkerOldApkPath and tinkerApplyResourcePath in tinker.gradle to the newly generated apk file and R file path in app-build-bakApk, and then modify the code or resources of the project;
Use Gradle-app-Tasks-tinker-tinkerPatchDebug in the upper right corner of AS to compile, and patch_signed_7zip.apk will be generated under app-build-outputs-apk-tinkerPatch-debug, which is the patch package. Put patch_signed_7zip.apk into the root directory of SD through the adb command
Click LOAD PATCHthe button and restart the App, the hot repair is completed.

3. How to use Release

The usage of Tinker is as follows, taking the release package accessed by gradle as an example:

Backup the installation package and mapping file every time you compile or issue a package;
If there is a need for a patch package, modify your code, library files, etc. according to your own needs;
Input the backup base installation package and mapping file into the configuration of tinkerPatch;
Run tinkerPatchRelease to automatically compile the latest installation package and make a difference with the input benchmark package to get the final patch package.

3. Known issues of Tinker

Due to principle and system limitations, Tinker has the following known issues:

Tinker does not support modifying AndroidManifest.xml, and Tinker does not support adding four major components (1.9.0 supports adding non-export activities);
Due to the limitations of Google Play's developer terms, it is not recommended to dynamically update the code in the GP channel;
On Android N, the patch has a slight impact on app startup time;
Some Samsung android-21 models are not supported, and will be actively thrown when the patch is loaded "TinkerRuntimeException:checkDexInstall failed";
For resource replacement, modifying the remoteView is not supported. For example transition animations, notification icons and desktop icons.

The demo has been uploaded to gitee, and students who like it can download it. By the way, give a like!

Previous: Mobile Architecture 01-Common Design Patterns and AOP Programming

Next: Mobile Architecture 03 - In-depth Analysis of Object-Oriented Database Framework

Analysis and use of mobile architecture 02-Tinker hot repair