Android ------ Meituan's Lint code inspection practice

Overview

Lint is an Android static code inspection tool provided by Google, which can scan and find potential problems in the code, remind developers to fix them early, and improve code quality. In addition to the hundreds of Lint rules natively provided by Android, custom Lint rules can also be developed to meet actual needs.

Why use Lint

During the iteration process of the Meituan Takeaway Android App, online problems frequently occurred. It is easy to write some problematic codes during development, such as the use of Serializable: a class that implements the Serializable interface, if the object referenced by its member variables does not implement the Serializable interface, it will crash during serialization. We analyze and summarize the causes and solutions of some common problems, and share and communicate with the developer group or with the testers to help the relevant personnel take the initiative to avoid these problems.

In order to further reduce the occurrence of problems, we have gradually improved some specifications, including formulating code specifications, strengthening code reviews, and improving testing procedures. However, these measures still have various shortcomings, including the difficulty of implementing code specifications, high communication costs, and repeated communication due to frequent changes in developers. Therefore, their effects are limited, and similar problems still occur from time to time. On the other hand, more and more summaries and normative documents have created a lot of learning pressure for newcomers in the group.

Is there a way to reduce or alleviate the above problems from a technical point of view?

Our research found that static code inspection is a good idea. There are many kinds of static code inspection frameworks, such as FindBugs, PMD, Coverity, which are mainly used to check Java source files or class files; another example is Checkstyle, which mainly focuses on code style; but we finally choose to start with Lint framework because it has many advantages:

Powerful, Lint supports the inspection of Java source files, class files, resource files, Gradle and other files.
Strong extensibility, supports the development of custom Lint rules.
The supporting tools are complete, and the Android Studio and Android Gradle plugins natively support the Lint tool.
Lint is designed for Android and natively provides hundreds of useful Android-related inspection rules.
With the official support of Google, it will be upgraded and improved together with the Android development tools.

After conducting sufficient technical research on Lint, we did some more in-depth thinking based on the actual problems encountered, including which problems should be solved with Lint, how to better promote and implement them, etc., and gradually formed a set of A more comprehensive and effective program.

Introduction to the Lint API

In order to facilitate the understanding of the following, let's take a brief look at the main API provided by Lint.

Main API

Lint rules are implemented by calling Lint APIs, the most important of which are as follows.

Issue: Indicates a Lint rule.
Detector: Used to detect and report Issues in the code. Each Issue must specify a Detector.
Scope: declares the code scope to be scanned by the Detector, for example JAVA_FILE_SCOPE, CLASS_FILE_SCOPE, , RESOURCE_FILE_SCOPE, GRADLE_SCOPEetc. An Issue can contain one or more Scopes.
Scanner: Used to scan and discover Issues in the code, each Detector can implement one or more Scanners.
IssueRegistry: The entry for Lint rule loading, providing a list of Issues to check.

For example, the native ShowToast is an Issue that checks Toast.makeText()for missing calls after calling Toast.show()a method. Its Detector is ToastDetector, and the Scope to be checked is JAVA_FILE_SCOPE. ToastDetector implements JavaPsiScanner. The schematic code is as follows.

    public class ToastDetector extends Detector implements JavaPsiScanner {  
        public static final Issue ISSUE = Issue.create(  
                "ShowToast",  
                "Toast created but not shown",  
                "...",  
                Category.CORRECTNESS,  
                6,  
                Severity.WARNING,  
                new Implementation(  
                        ToastDetector.class,  
                        Scope.JAVA_FILE_SCOPE));  
        // ...  
    }

The schematic code of IssueRegistry is as follows.

    public class MyIssueRegistry extends IssueRegistry {  
      
        @Override  
        public List<Issue> getIssues() {  
            return Arrays.asList(  
                    ToastDetector.ISSUE,  
                    LogDetector.ISSUE,  
                    // ...  
            );  
        }  
    }

Scanner

The main work in the development process of Lint is to implement Scanner. Lint includes several types of Scanner as follows, the most commonly used is Scanner that scans Java source files and XML files.

JavaScanner/JavaPsiScanner/UastScanner: Scan Java source files
XmlScanner: Scan XML files
ClassScanner: scans class files
BinaryResourceScanner: Scan binary resource files
ResourceFolderScanner: Scans resource folders
GradleScanner: Scan Gradle scripts
OtherFileScanner: Scan other types of files

It is worth noting that Scanner, which scans Java source files, has gone through three versions.

JavaScanner was used at the beginning, and Lint parsed Java source code into AST (Abstract Syntax Tree) through the Lombok library, which was then scanned by JavaScanner.
In Android Studio 2.2 and lint-api 25.2.0, the Lint tool replaces Lombok AST with PSI, and deprecates JavaScanner. JavaPsiScanner is recommended.

PSI is an API provided by JetBrains after parsing Java source code in IDEA to generate a syntax tree. Compared to the previous Lombok AST, PSI can support Java 1.8, type resolution, etc. Custom Lint rules implemented using JavaPsiScanner can be loaded into Android Studio 2.2+ and executed in real-time while writing Android code.
In Android Studio 3.0 and lint-api 25.4.0, the Lint tool replaced PSI with UAST, and recommended the new UastScanner.

UAST is the API used by JetBrains to replace PSI in the new version of IDEA. UAST is more language-agnostic, in addition to supporting Java, it can also support Kotlin.

This article is still based on PsiJavaScanner for introduction. According to the comments in the UastScanner source code, it is easy to migrate from PsiJavaScanner to UastScanner.

Lint rules

What problems do we need to check our code with Lint?

During the development process, we pay more attention to indicators such as App Crash and Bug rate. Through long-term sorting and summary, it is found that there are many code problems with high frequency, the principles and solutions are very clear, but they are easy to miss and difficult to find when writing code; and Lint happens to be easy to check out these problems.

Crash prevention

Crash rate is one of the most important indicators of an app. Avoiding Crash has always been a headache in the development process. Lint can check out some potential Crash very well. E.g:

The native NewApi is used to check whether the API provided by the higher version of Android is called in the code. Calling a higher version API on a lower version device will cause a crash.
Custom SerializableCheck. For a class that implements the Serializable interface, if the object referenced by its member variables does not implement the Serializable interface, it will crash during serialization. We have formulated a code specification that requires the class that implements the Serializable interface, and the declared types of its member variables (including those inherited from the parent class) must implement the Serializable interface.
Custom ParseColorCheck. When calling Color.parseColor()the method to parse the color sent from the background, the incorrect format of the color string will cause an IllegalArgumentException. We require that this exception must be handled when calling this method.

Bug Prevention

Some bugs can be prevented by lint checking. E.g:

SpUsage: All SharedPrefrence read and write operations are required to use the basic tool class, and various exception handling will be done in the tool class; at the same time, the SPConstants constant class is defined, and all SP keys must be defined in this class to avoid scattered definitions in the code. conflict.
ImageViewUsage: Check if ImageView has ScaleType set, and whether Placeholder is set when loading.
TodoCheck: Check if there are TODOs in the code that are not completed. For example, some fake data may be written in the code during development, but make sure to delete these codes when it is finally launched. This kind of check item is special, and it is usually checked in the test stage after the development is completed.

Performance/Security Issues

Some performance and security related issues can be analyzed using Lint. E.g:

ThreadConstruction: Direct use of new Thread()creating threads (except thread pools) is prohibited, and a unified tool class is required to perform background operations in the common thread pool.
LogUsage: Direct use is prohibited android.util.Log, and a unified tool class must be used. In the tool class, the Release package can be controlled not to output Log, which improves performance and avoids security issues.

code specification

In addition to code style constraints, code specifications are more used to reduce or prevent bugs, crashes, performance, security and other issues. Many problems are technically difficult to check directly. We indirectly solve them by encapsulating a unified basic library and formulating code specifications. Lint checks are used to reduce the cost of communication within the group, the cost of learning for newcomers, and to ensure the implementation of code specifications. E.g:

The aforementioned SpUsage, ThreadConstruction, LogUsage, etc.
ResourceNaming: Resource file naming convention to prevent resource file name conflicts between different modules.

Implementation of Code Inspection

When a code problem is detected, how to remind the developer to correct it in time?

In the early days, we configured static code inspection on Jenkins, and when we packaged and released AAR/APK, we checked for problems in the code and generated reports. Later, it was found that although static code inspection can find a lot of problems, few people take the initiative to read the report, especially there are too many irrelevant and low-priority problems in the report (such as too strict code style constraints).

Therefore, on the one hand, it is important to determine which problems to check, and on the other hand, when and by what technical means to perform code inspections. Combined with technical implementation, we thought more about this and identified the main goals in the implementation of static code inspection:

Focus on high-priority issues and block low-priority issues. As mentioned earlier, if the code inspection report contains a large number of unimportant problems, it will affect the discovery of key problems.
The solution of high-quality problems must be mandatory. When a high-priority code problem is found in the inspection, a clear and direct error report is given to the developer, and the developer is forced to fix it through technical means.
Certain problems are discovered as soon as possible to reduce risks or losses. Some problems are discovered as early as possible. For example, the API of a higher version of Android is used in the development of business functions, which can be checked out through Lint's native NewApi. If it is found during development, other technical solutions can be considered at that time, and when implementation is difficult, it can be communicated with products and designers in time; but if it is discovered only when the code is mentioned, tested, or even released and launched, it may be too late.

priority definition

Sevirity (priority) can be configured for each Lint rule, including Fatal, Error, Warning, Information, etc. We mainly use Error and Warning, as follows.

Error level: Identify the problems that need to be solved, including crashes, clear bugs, serious performance problems, non-compliance with code specifications, etc., and must be fixed.
Warning level: including code writing suggestions, possible bugs, some performance optimizations, etc., and relax the requirements appropriately.

execution time

Lint checks can be performed in multiple stages, including local manual checks, real-time coding checks, compile-time checks, commit checks, as well as checking when submitting Pull Requests in the CI system, checking when packaging and releasing, etc., which are described below.

do it manually

In Android Studio, custom Lints can be Analyze - Inspect Coderun manually through the Inspections function ( ).

In the Gradle command line environment, you can directly use ./gradlew lintLint to check.

Manual execution is simple and easy to use, but it lacks compulsion and is easily missed by developers.

Real-time checking during encoding phase

Checking while coding means reporting errors in the code window in real time when writing code in Android Studio. Its benefits are obvious, developers can find code problems at the first time. However, due to the imperfect support of Android Studio for custom Lint, the configuration of the developer's IDE is different, and the developer needs to actively pay attention to the error report and fix it. This method cannot fully guarantee the effect.

IDEA provides Inspections and corresponding APIs to implement code inspection. Android native Lint is integrated into Android Studio through Inspections. For custom Lint rules, the official does not seem to give clear instructions, but actual research found that under the conditions of Android Studio 2.2+ and JavaPsiScanner-based development (or Android Studio 3.0+ and JavaPsiScanner/UastScanner), IDE will try to load and real-time Execute custom Lint rules.

technical details:

In Android Studio 2.x version, the menu Preferences - Editor - Inspections - Android - Lint - Correctness - Error from Custom Lint Check（avaliable for Analyze|Inspect Code）states that custom Lint only supports command line or manual running, not real-time inspection.

Error from Custom Rule When custom (third-party) lint rules are integrated in the IDE, they are not available as native IDE inspections, so the explanation text (which must be statically registered by a plugin) is not available. As a workaround, run the lint target in Gradle instead; the HTML report will include full explanations.
In the Android Studio 3.x version, after opening the Android project source code, the IDE will load the custom Lint rules in the project, which can be viewed in the Inspections list of the settings menu, which has the same effect as native Lint (Android Studio will open the source file when the source file is opened. trigger a code inspection for that file).
IssueRegistry.getIssues()Analyzing the method call stack of the custom Lint , you can see that in the Android Studio environment, the custom Lint rules are executed by the org.jetbrains.android.inspections.lint.AndroidLintExternalAnnotatorcall load.LintDriver

Reference code: https://github.com/JetBrains/android/tree/master/android/src/org/jetbrains/android/inspections/lint

The actual effect in Android Studio is shown in the figure:

Automatically check when compiling locally

Configure Gradle scripts to perform Lint checks when compiling Android projects. The advantage is that the problem can be found as early as possible, and it can be mandatory; the disadvantage is that it has a certain impact on the compilation speed.

When compiling an Android project, the assemble task is executed. If assemble depends on the lint task, the Lint check can be performed at compile time. At the same time, LintOptions is configured, and the compilation is interrupted when an Error level problem is found.

The configuration in the Android Application project (APK) is as follows, and the Android Library project (AAR) can be applicationVariantsreplaced libraryVariantswith.

    android.applicationVariants.all { variant ->  
        variant.outputs.each { output ->  
            def lintTask = tasks["lint${variant.name.capitalize()}"]  
            output.assemble.dependsOn lintTask  
        }  
    }

Configuration of LintOptions:

android.lintOptions {
    abortOnError true
}

Check on local commit

Using the git pre-commit hook, you can perform a Lint check before the local commit code. If the check fails, the code cannot be submitted. The advantage of this method is that it does not affect the compilation speed during development, but the problem is relatively lagging behind.

In terms of technical implementation, you can write a Gradle script to automatically copy the hook script from the project to the .git/hooks/folder every time you synchronize the project.

CI check when submitting code

As part of the code submission process specification, it is a common, feasible and effective idea to use the CI system to check Lint problems when sending Pull Requests. Configurable CI checks pass before code can be merged.

Jenkins is commonly used in CI systems. If you use Stash for code management, you can configure the Pull Request Notifier for Stash plug-in on Stash, or configure the Stash Pull Request Builder plug-in on Jenkins to implement a Job that triggers Jenkins to perform Lint checks when a Pull Request is sent.

Both local compilation and code inspection in the CI system can be implemented by executing Gradle's Lint task. You can pass a StartParameter to Gradle in the CI environment. If this parameter is read in the Gradle script, configure LintOptions to check all Lint problems; otherwise, only some high-priority Lint problems are checked in the local compilation environment, reducing the speed of local compilation. influence.

The effect of Lint's report generation is shown in the figure:

Check for package release

Even if the CI system is used to perform a Lint check every time the code is submitted, it is still not guaranteed that everyone's code will be merged without problems; in addition, for some special Lint rules, such as the aforementioned TodoCheck, it is hoped to be checked at a later time.

Therefore, when the CI system packages and releases the APK/AAR for testing or release, it is necessary to do another Lint check on all the codes.

Finalized inspection timing

Considering the advantages and disadvantages of multiple inspection methods and our goals, we finally decided to combine the following methods for code inspection:

During the coding phase, the IDE checks in real time and finds problems at the first time.
When compiling locally, check for high-priority issues in time, and compile after passing the check.
When the code is submitted, CI checks all the problems, and the code can be merged only after the check is passed.
In the packaging phase, the project is completely checked to ensure that nothing goes wrong.

Profile support

In order to facilitate code management, we created an independent project for the custom Lint. The project is packaged to generate an AAR and published to the Maven repository, and the checked Android project depends on this AAR (for the specific development process, please refer to the link at the end of the article).

Although the custom Lint is in an independent project, it is more coupled with the code specifications and basic components in the checked Android project.

For example, we use regular expressions to check the resource file naming convention of the Android project. Every time the business logic changes to add a resource file prefix, it is very cumbersome to modify the Lint project, release a new AAR, and then update it to the Android project. On the other hand, our Lint project is not only used in the takeaway C-side Android project, but also hopes to be directly used in other Android projects on other ends, and there are differences between different projects.

So we tried to use configuration files to solve this problem. Take the LogUsage used to check the Log as an example. Different projects encapsulate different Log tool classes, and the prompt information should be different when an error is reported. Define the configuration file name custom-lint-config.jsonand place it in the module directory of the Android project being checked. The configuration file in Android project A is:

    {  
        "log-usage-message": "请勿使用android.util.Log，建议使用LogUtils工具类"  
    }

And the configuration file of Android project B is:

{
    "log-usage-message": "请勿使用android.util.Log，建议使用Logger工具类"
}

The inspected project directory can be obtained from the Context object of Lint to read the configuration file. The key code is as follows:

    import com.android.tools.lint.detector.api.Context;  
      
    public final class LintConfig {  
      
        private LintConfig(Context context) {  
            File projectDir = context.getProject().getDir();  
            File configFile = new File(projectDir, "custom-lint-config.json");  
            if (configFile.exists() && configFile.isFile()) {  
                // 读取配置文件...  
            }  
        }  
    }

The configuration file can be read in the Detector's beforeCheckProject and beforeCheckLibraryProject callback methods. When an error is detected in LogUsage, an error is reported according to the information defined in the configuration file.

    public class LogUsageDetector extends Detector implements Detector.JavaPsiScanner {  
        // ...  
      
        private LintConfig mLintConfig;  
      
        @Override  
        public void beforeCheckProject(@NonNull Context context) {  
            // 读取配置  
            mLintConfig = new LintConfig(context);  
        }  
      
        @Override  
        public void beforeCheckLibraryProject(@NonNull Context context) {  
            // 读取配置  
            mLintConfig = new LintConfig(context);  
        }  
      
        @Override  
        public List<String> getApplicableMethodNames() {  
            return Arrays.asList("v", "d", "i", "w", "e", "wtf");  
        }  
      
        @Override  
        public void visitMethod(JavaContext context, JavaElementVisitor visitor, PsiMethodCallExpression call, PsiMethod method) {  
            if (context.getEvaluator().isMemberInClass(method, "android.util.Log")) {  
                // 从配置文件获取Message  
                String msg = mLintConfig.getConfig("log-usage-message");  
                context.report(ISSUE, call, context.getLocation(call.getMethodExpression()), msg);  
            }  
        }  
    }

Template Lint Rules

During the development of Lint rules, we found a series of similar requirements: basic tool classes are encapsulated, and we hope everyone can use them; a method can easily throw RuntimeException, and it is necessary to deal with it, but RuntimeException is not mandatory in Java syntax Processing thus often misses...

These similar requirements will also be cumbersome to develop in the Lint project every time. We have tried to implement several templates, which can configure Lint rules directly in the Android project through configuration files.

The following is an example configuration file:

[java] view plain copy

    {  
      "lint-rules": {  
        "deprecated-api": [{  
          "method-regex": "android\\.content\\.Intent\\.get(IntExtra|StringExtra|BooleanExtra|LongExtra|LongArrayExtra|StringArrayListExtra|SerializableExtra|ParcelableArrayListExtra).*",  
          "message": "避免直接调用Intent.getXx()方法，特殊机型可能发生Crash，建议使用IntentUtils",  
          "severity": "error"  
        },  
        {  
          "field": "java.lang.System.out",  
          "message": "请勿直接使用System.out，应该使用LogUtils",  
          "severity": "error"  
        },  
        {  
          "construction": "java.lang.Thread",  
          "message": "避免单独创建Thread执行后台任务，存在性能问题，建议使用AsyncTask",  
          "severity": "warning"  
        },  
        {  
          "super-class": "android.widget.BaseAdapter",  
          "message": "避免直接使用BaseAdapter，应该使用统一封装的BaseListAdapter",  
          "severity": "warning"  
        }],  
        "handle-exception": [{  
          "method": "android.graphics.Color.parseColor",  
          "exception": "java.lang.IllegalArgumentException",  
          "message": "Color.parseColor需要加try-catch处理IllegalArgumentException异常",  
          "severity": "error"  
        }]  
      }  
    }

Two types of template rules are defined in the example configuration:

DeprecatedApi: Direct calls to the specified API are prohibited
HandleException: When calling the specified API, you need to add try-catch to handle the specified type of exception

Matching of problem API, including method call (method), member variable reference (field), constructor (construction), inheritance (super-class) and other types; matching strings support glob syntax or regular expressions (and in lint.xml The configuration syntax of ignore is consistent).

In terms of implementation, it is mainly to traverse a specific type of node in the Java syntax tree and convert it into a complete string (such as a method call android.content.Intent.getIntExtra), and then check whether there is a template rule that matches it. After the matching is successful, the DeprecatedApi rule directly outputs the message and reports an error; the HandleException rule checks whether the matched node handles a specific Exception (or the parent class of Exception), and reports an error if it is not handled.

Check new files by Git version

As new rules for Lint continue to be developed, we have another problem. There are a lot of historical codes in the Android project, which do not meet the requirements of the new Lint rules, but do not cause obvious problems. At this time, accessing the new Lint rules requires modifying all historical codes, which is costly and has certain risks. For example, the new code specification requires the use of a unified thread tool class and does not allow the direct use of Handler to avoid memory leaks, etc.

We tried a compromise solution: only check the files added after the specified git commit. Add a configuration item to the configuration file, and configure an git-baseattribute for the Lint rule. Its value is the commit ID. Only the files added after this commit are checked.

In terms of implementation, execute the git rev-parse --show-toplevelcommand to obtain the path of the root directory of the git project; execute the git ls-tree --full-tree --full-name --name-only -r <commit-id>command to obtain the list of existing files (relative to the path of the git root directory) when the commit is specified. In the Scanner callback method, Context.getLocation(node).getFile()it is determined whether the node needs to be checked by obtaining the file where the node is located and combining with the git file list. It should be noted that when the amount of code is large, the performance consumption of Lint checking on the computer should be considered.

Summarize

After a period of practice, it is found that Lint static code inspection is very effective in solving specific problems, such as finding some relatively clear low-level errors at the language or API level, and helping to constrain code specifications. Before using Lint, many of these problems happened to be easy for developers to miss (such as native NewApi check, custom SerializableCheck); the same problem recurs; code specification execution, especially when new people participate in development, It requires a high cost of learning and communication, and newcomers are often required to modify the code repeatedly because they do not comply with the code specifications. After using Lint, these problems can be solved immediately, saving a lot of manpower, improving code quality and development efficiency, and improving the app experience.

References and Further Reading

References: