Confusion configuration for Android compilation optimization

Confusion configuration for Android compilation optimization

background

In order to use the features of java8 and subsequent new versions of java, Google has added a step to the compilation process - desugaring (desugaring), but this step will lead to longer compilation time, which is why Google launched D8 and R8 compilers to optimize compilation speed .

What is desugar?

Desugaring is to convert some features that are not supported by the underlying bytecode at the grammatical level into the basic bytecode structure at the compilation stage (for example, the generic type on List is actually Object at the bytecode level after desugaring); Android tools The process of desugaring Java8 syntax features can be described as colorful, of course, their ultimate goal is the same: to make the new syntax run on all devices.

D8

The function of D8 is to convert Java bytecode into dex code, and D8 is an alternative to DX. The compilation process is shown in the figure below:

Starting from version 3.1 of Android Studio, D8 is used as the default Dex compiler. If you want to close D8, you can add the following configuration in gradle.properties:

android.enableD8=false
android.enableD8.desugaring=false

Benefits of turning on D8

• Compile faster and in less time
• DEX takes up less content when compiling
• .dex files are smaller
• The .dex files compiled by D8 have the same or better performance

If your project has already used Java 8 to enable D8 compilation as much as possible, otherwise compilation errors may occur.

R8

As a replacement for the original Proguard compression and optimization (minification, shrinking, optimization) parts, R8 still uses the same keep rules as Proguard, and is a new generation of code compression tools. R8 was previously built in the form of D8+Proguard, and R8 integrated obfuscation and D8 tools to speed up the build time and reduce the size of the output apk.

When the Gradle plug-in version reaches 3.4.0 and above, R8 will start code optimization by default. If you don't want to enable R8, you can gradle.propertiesadd the following configuration in it:

android.enableR8=false

Benefits of turning on R8

• Code reduction: circumventing the 64 reference limit
• Resource reduction: remove unused resources
• Obfuscated code: reduce DEX file size
• Optimize code: further reduce DEX file size

reference:

https://developer.android.google.cn/studio/build/shrink-code

Related Evaluation Report

AS multi-module confusion configuration

Under AS multi-module, we can adopt the individual configuration method of each module, that is, the proguard file is configured under each module and introduced in the respective build.gradle file; or the centralized configuration method below can be used to centrally configure the proguard files of each module to A folder, and then centrally import the proguard files of the dependent modules in the app module.
Let's take the app module cmcc_service as an example to illustrate,

buildTypes {
    
    
        release {
    
    
            minifyEnabled true //开启混淆
            shrinkResources true //无用资源去除
            zipAlignEnabled true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'),'proguard-rules.pro',
                    '../proguardDefine/common-rules.pro', '../proguardDefine/live_lib-rules.pro',
                    '../proguardDefine/gs_api_adapter-rules.pro', '../proguardDefine/cmcc_service.pro',
                    '../proguardDefine/lib_voiceassistant-rules.pro','../proguardDefine/cmcc_softprobe-rules.pro'
            if (propertyHaveSigningConfigs)
                signingConfig signingConfigs.release
        }
}

Among them, proguard-android-optimize.txtthere are some general obfuscation rules defaulted by android, proguard-rules.prothe obfuscation rules of the app code, and the place where proguardDefinethe dependent modules under the folder are uniformly placed .proguard

Note: The imported three-party libs are generally equipped with corresponding proguard files, so there is no need to repeat the configuration. When compiling gradle, the related files
proguard files will be merged into one configuration.txtfile.

proguard-android-optimize.txt

# 禁用一些代码简化和优化，以及字段和类合并
-optimizations !code/simplification/arithmetic,!code/simplification/cast,!field/*,!class/merging/*
# 运行优化 passes 的数量为 5，数值越高，混淆效果越好，但耗时也更长
-optimizationpasses 5
# 允许访问和修改保护代码
-allowaccessmodification

# 不允许使用大小写混合的类名
-dontusemixedcaseclassnames
# 不跳过非公共库类
-dontskipnonpubliclibraryclasses
# 详细输出
-verbose

# 保留一些反射所需的属性
-keepattributes *Annotation*,Signature,InnerClasses,EnclosingMethod
# 保留以下三个类及其公共成员
-keep public class com.google.vending.licensing.ILicensingService
-keep public class com.android.vending.licensing.ILicensingService
-keep public class com.google.android.vending.licensing.ILicensingService
# 忽略以下类，不输出 note 信息
-dontnote com.android.vending.licensing.ILicensingService
-dontnote com.google.vending.licensing.ILicensingService
-dontnote com.google.android.vending.licensing.ILicensingService

# 对于本地方法，参见 http://proguard.sourceforge.net/manual/examples.html#native
# 保留包含 native 方法的类和方法
-keepclasseswithmembernames,includedescriptorclasses class * {
    
    
    native <methods>;
}
# 保留公共的 View 子类的 set 和 get 方法
# 以便于使用属性动画
-keepclassmembers public class * extends android.view.View {
    
    
    void set*(***);
    *** get*();
}
# 保留 Activity 中可用于 XML 属性 onClick 中的方法
-keepclassmembers class * extends android.app.Activity {
    
    
    public void *(android.view.View);
}
# 对于枚举类，参见 http://proguard.sourceforge.net/manual/examples.html#enumerations
# 保留枚举类成员
-keepclassmembers enum * {
    
    
    public static **[] values();
    public static ** valueOf(java.lang.String);
}
# 保留实现 Parcelable 接口的类的 CREATOR 静态成员
-keepclassmembers class * implements android.os.Parcelable {
    
    
    public static final ** CREATOR;
}
# 保留 JavaScript 接口方法上的注解
-keepclassmembers class * {
    
    
    @android.webkit.JavascriptInterface <methods>;
}
# 支持库包含对新平台版本的引用
# 在应用链接旧版平台版本时，不要发出警告，因为它们是安全的
-dontnote android.support.**
-dontnote androidx.**
-dontwarn android.support.**
-dontwarn androidx.**
# 该类已弃用，但仍然保留用于向后兼容
-dontwarn android.util.FloatMath
#这段混淆规则用于保护使用了@Keep注解的类和成员不被混淆，以及忽略特定的冗余类。
-keep class android.support.annotation.Keep
-keep class androidx.annotation.Keep
#保留使用了@Keep注解的类和接口的所有成员
-keep @android.support.annotation.Keep class * {
    
    ;}
-keep @androidx.annotation.Keep class * {
    
    ;}
#保留使用了@Keep注解的类的方法
-keepclasseswithmembers class * {
    
    
@android.support.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
    
    
@androidx.annotation.Keep <methods>;
}
#保留使用了@Keep注解的类的字段
-keepclasseswithmembers class * {
    
    
@android.support.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
    
    
@androidx.annotation.Keep <fields>;
}
#保留使用了@Keep注解的类的构造方法
-keepclasseswithmembers class * {
    
    
@android.support.annotation.Keep <init>(...);
}
-keepclasseswithmembers class * {
    
    
@androidx.annotation.Keep <init>(...);
}
#忽略特定的冗余类
#android.jar和org.apache.http.legacy.jar中的类重复
-dontnote org.apache.http.**
-dontnote android.net.http.**
#android.jar和core-lambda-stubs.jar中的类重复
-dontnote java.lang.invoke.**

Some third parties have their own obfuscation rules

These rules are also merged into configuration.txtthe file
retrofit2 obfuscation rules

# The proguard configuration file for the following section is /home/cl/.gradle/caches/transforms-3/29b6aa006718d6829551a18646bf70bb/transformed/rules/lib/META-INF/proguard/retrofit2.pro
# Retrofit does reflection on generic parameters. InnerClasses is required to use Signature and
# EnclosingMethod is required to use InnerClasses.
-keepattributes Signature, InnerClasses, EnclosingMethod

# Retain service method parameters when optimizing.
-keepclassmembers,allowshrinking,allowobfuscation interface * {
    
    
    @retrofit2.http.* <methods>;
}

# Ignore annotation used for build tooling.
-dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement

# Ignore JSR 305 annotations for embedding nullability information.
-dontwarn javax.annotation.**

# Guarded by a NoClassDefFoundError try/catch and only used when on the classpath.
-dontwarn kotlin.Unit

# Top-level functions that can only be used by Kotlin.
-dontwarn retrofit2.-KotlinExtensions

# End of content from /home/cl/.gradle/caches/transforms-3/29b6aa006718d6829551a18646bf70bb/transformed/rules/lib/META-INF/proguard/retrofit2.pro

RxJava2\RxAndroid confusion rules

-dontwarn java.util.concurrent.Flow*

Okhttp3 obfuscation rules

# The proguard configuration file for the following section is /home/cl/.gradle/caches/transforms-3/af3ecb4c3ae4accf6423845d738f047d/transformed/rules/lib/META-INF/proguard/okhttp3.pro
# JSR 305 annotations are for embedding nullability information.
-dontwarn javax.annotation.**

# A resource is loaded with a relative path so the package of this class must be preserved.
-keepnames class okhttp3.internal.publicsuffix.PublicSuffixDatabase

# Animal Sniffer compileOnly dependency to ensure APIs are compatible with older versions of Java.
-dontwarn org.codehaus.mojo.animal_sniffer.*

# OkHttp platform used only on JVM and when Conscrypt dependency is available.
-dontwarn okhttp3.internal.platform.ConscryptPlatform

# End of content from /home/cl/.gradle/caches/transforms-3/af3ecb4c3ae4accf6423845d738f047d/transformed/rules/lib/META-INF/proguard/okhttp3.pro

More possible obfuscation configurations

# 指定不去忽略非公共库的类
-dontskipnonpubliclibraryclasses
# 指定不去忽略非公共库的成员
-dontskipnonpubliclibraryclassmembers
# 混淆时不做预校验
-dontpreverify
# 忽略警告
-ignorewarnings
# 保留代码行号，方便异常信息的追踪
-keepattributes SourceFile,LineNumberTable
# appcompat库不做混淆
-keep class androidx.appcompat.**
#保留 AndroidManifest.xml 文件：防止删除 AndroidManifest.xml 文件中定义的组件
-keep public class * extends android.app.Fragment
-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.preference.Preference
-keep public class * extends android.view.View

#保留序列化，例如 Serializable 接口
-keepclassmembers class * implements java.io.Serializable {
    
    
    static final long serialVersionUID;
    private static final java.io.ObjectStreamField[] serialPersistentFields;
    private void writeObject(java.io.ObjectOutputStream);
    private void readObject(java.io.ObjectInputStream);
    java.lang.Object writeReplace();
    java.lang.Object readResolve();
}

#带有Context、View、AttributeSet类型参数的初始化方法
-keepclasseswithmembers class * {
    
    
    public <init>(android.content.Context);
}
-keepclasseswithmembers class * {
    
    
    public <init>(android.content.Context, android.util.AttributeSet);
}
-keepclasseswithmembers class * {
    
    
    public <init>(android.content.Context, android.util.AttributeSet, int);
}
-keepclassmembers class * extends android.app.Activity {
    
    
   public void *(android.view.View);
}

#保留资源R类
-keep class **.R$* {
    
    *;}

#避免回调函数 onXXEvent 混淆
-keepclassmembers class * {
    
    
    void *(**On*Event);
    void *(**On*Listener);
    void *(**on*Changed);
}

#业务实体不做混淆，避免gson解析错误
-dontwarn com.grandstream.convergentconference.entity.**
-keep class com.grandstream.convergentconference.entity.** {
    
     *;}

#Rxjava、RxAndroid，官方ReadMe文档中说明无需特殊配置
-dontwarn java.util.concurrent.Flow*
#okhttp3、okio、retrofit，jar包中已包含相关proguard规则，无需配置
#其他一些配置

Android.bp confusion configuration

android_app {
    
    
    name: "MyApp",
    package_name: "com.example.myapp",
    srcs: ["java/**/*.java"],
    manifest: "AndroidManifest.xml",
    dex_preopt: {
    
    
        enabled: true,
    },
    apk_cert_permissions: [
        "android.permission.ACCESS_FINE_LOCATION",
        "android.permission.RECORD_AUDIO",
        "android.permission.READ_CONTACTS",
    ],
    certificates: ["platform"],
    resource_files: ["res/**/*"],
    enable_proguard: true,
    proguard_flags: ["proguard-android.txt"],
    dex_preopt_image_dir: "target/product/${TARGET_PRODUCT}/obj/dex_preopt_image",
}

Among them, enable_proguard is set to true to enable code and resource obfuscation, proguard_flags specifies the Proguard obfuscation rule file, and code and resource obfuscation rules can be specified in the file.
resource_files specifies the application's resource files, including the AndroidManifest.xml file.

android.mk confusion configuration

LOCAL_PROGUARD_ENABLED := obfuscation optimization

LOCAL_PROGUARD_FLAG_FILES := proguard.flags
ifeq (eng,$(TARGET_BUILD_VARIANT))
    LOCAL_PROGUARD_FLAG_FILES += proguard-test.flags
else
    LOCAL_PROGUARD_FLAG_FILES += proguard-release.flags
endif

LOCAL_PROGUARD_ENABLEDThere are several configurations:

1. LOCAL_PROGUARD_ENABLED := disabled: Disable obfuscation.
2. LOCAL_PROGUARD_ENABLED := obfuscate: Only enable code obfuscation without optimization.
3. LOCAL_PROGUARD_ENABLED := optimize: Only enable optimization, no code obfuscation.
4. LOCAL_PROGUARD_ENABLED := obfuscate optimize: Enable code obfuscation and optimization at the same time.

By default, the build system should load an android-general obfuscation file LOCAL_PROGUARD_FLAG_FILESfor specifying app-specific proguardfiles.
If we are not very good at configuring these options, you can check the obfuscation configuration of the native application, which will be very inspiring to you.

Obfuscated stack restore

Use android's built-in confusion stack restoration tool proguardgui to solve

Android/Sdk/tools/proguard/bin$ ls
proguard.sh  proguardgui.sh  retrace.sh
/Android/Sdk/tools/proguard/bin$ ./proguardgui.sh

After the script runs, the ProGuard debugging interface will be opened

Fill in the mapping file mapping.txt before and after confusion and the logcat exception log that needs to be restored, and click the Retrace button to restore the exception information before confusion

in conclusion

In short, Android has already written some general obfuscation rules for me. What we care about may be the obfuscation rules for the part of the code we develop ourselves. Good obfuscation work has many advantages: safer code, smaller package size, and faster
operation It is faster, but confusion will also bring unexpected running exceptions. I hope you do a good job of testing and use it as soon as possible.

reference

https://blog.csdn.net/Mr_dsw/article/details/90141647
https://blog.csdn.net/wwj_748/article/details/115874571
Android confusion, what has changed with the newly introduced D8 and R8?
https://toutiao.io/posts/ijfhzkv/preview