Mobile Development: Meituan Takeaway Android Lint Code Inspection Practice

Overview

Lint is an Android static code inspection tool provided by Google, which can scan and find potential problems in the code, remind developers to fix them early, and improve code quality. In addition to the hundreds of Lint rules natively provided by Android, custom Lint rules can also be developed to meet actual needs.

Why use Lint

During the iteration process of the Meituan Takeaway Android App, online problems frequently occurred. It is easy to write some problematic codes during development, such as the use of Serializable: a class that implements the Serializable interface, if the object referenced by its member variables does not implement the Serializable interface, it will crash during serialization. We analyze and summarize the causes and solutions of some common problems, and share and communicate with the developer group or with the testers to help the relevant personnel take the initiative to avoid these problems.

In order to further reduce the occurrence of problems, we have gradually improved some specifications, including formulating code specifications, strengthening code reviews, and improving testing procedures. However, these measures still have various shortcomings, including the difficulty of implementing code specifications, high communication costs, and repeated communication due to frequent changes in developers. Therefore, their effects are limited, and similar problems still occur from time to time. On the other hand, more and more summaries and normative documents have created a lot of learning pressure for newcomers in the group.

Is there a way to reduce or alleviate the above problems from a technical point of view?

Our research found that static code inspection is a good idea. There are many kinds of static code inspection frameworks, such as FindBugs, PMD, Coverity, which are mainly used to check Java source files or class files; another example is Checkstyle, which mainly focuses on code style; but we finally choose to start with Lint framework because it has many advantages:

Powerful, Lint supports the inspection of Java source files, class files, resource files, Gradle and other files.
Strong extensibility, supports the development of custom Lint rules.
The supporting tools are complete, and the Android Studio and Android Gradle plugins natively support the Lint tool.
Lint is designed for Android and natively provides hundreds of useful Android-related inspection rules.
With the official support of Google, it will be upgraded and improved together with the Android development tools.

After conducting sufficient technical research on Lint, we did some more in-depth thinking based on the actual problems encountered, including which problems should be solved with Lint, how to better promote and implement them, etc., and gradually formed a set of A more comprehensive and effective program.

Introduction to the Lint API

In order to facilitate the understanding of the following, let's take a brief look at the main API provided by Lint.

Main API

Lint rules are implemented by calling Lint APIs, the most important of which are as follows.

Issue: Indicates a Lint rule.
Detector: Used to detect and report Issues in the code. Each Issue must specify a Detector.
Scope: declares the code scope to be scanned by the Detector, for example JAVA_FILE_SCOPE, CLASS_FILE_SCOPE, , RESOURCE_FILE_SCOPE, GRADLE_SCOPEetc. An Issue can contain one or more Scopes.
Scanner: Used to scan and discover Issues in the code, each Detector can implement one or more Scanners.
IssueRegistry: The entry for Lint rule loading, providing a list of Issues to check.

For example, the native ShowToast is an Issue that checks Toast.makeText()for missing calls after calling Toast.show()a method. Its Detector is ToastDetector, and the Scope to be checked is JAVA_FILE_SCOPE. ToastDetector implements JavaPsiScanner. The schematic code is as follows.

public class ToastDetector extends Detector implements JavaPsiScanner { public static final Issue ISSUE = Issue.create( "ShowToast", "Toast created but not shown", "...", Category.CORRECTNESS, 6, Severity.WARNING, new Implementation( ToastDetector.class, Scope.JAVA_FILE_SCOPE));

The schematic code of IssueRegistry is as follows.

public class MyIssueRegistry extends IssueRegistry { @Override public List<Issue> getIssues() { return Arrays.asList( ToastDetector.ISSUE, LogDetector.ISSUE, // ... ); } }

Scanner

The main work in the development process of Lint is to implement Scanner. Lint includes several types of Scanner as follows, the most commonly used is Scanner that scans Java source files and XML files.

JavaScanner/JavaPsiScanner/UastScanner: Scan Java source files
XmlScanner: Scan XML files
ClassScanner: scans class files
BinaryResourceScanner: Scan binary resource files
ResourceFolderScanner: Scans resource folders
GradleScanner: Scan Gradle scripts
OtherFileScanner: Scan other types of files

It is worth noting that Scanner, which scans Java source files, has gone through three versions.

JavaScanner was used at the beginning, and Lint parsed Java source code into AST (Abstract Syntax Tree) through the Lombok library, which was then scanned by JavaScanner.
In Android Studio 2.2 and lint-api 25.2.0, the Lint tool replaces Lombok AST with PSI, and deprecates JavaScanner. JavaPsiScanner is recommended.

PSI is an API provided by JetBrains after parsing Java source code in IDEA to generate a syntax tree. Compared to the previous Lombok AST, PSI can support Java 1.8, type resolution, etc. Custom Lint rules implemented using JavaPsiScanner can be loaded into Android Studio 2.2+ and executed in real-time while writing Android code.
In Android Studio 3.0 and lint-api 25.4.0, the Lint tool replaced PSI with UAST, and recommended the new UastScanner.

UAST is the API used by JetBrains to replace PSI in the new version of IDEA. UAST is more language-agnostic, in addition to supporting Java, it can also support Kotlin.

This article is still based on PsiJavaScanner for introduction. According to the comments in the UastScanner source code, it is easy to migrate from PsiJavaScanner to UastScanner.

Lint rules

What problems do we need to check our code with Lint?

During the development process, we pay more attention to indicators such as App Crash and Bug rate. Through long-term sorting and summary, it is found that there are many code problems with high frequency, the principles and solutions are very clear, but they are easy to miss and difficult to find when writing code; and Lint happens to be easy to check out these problems.

Crash prevention

Crash rate is one of the most important indicators of an app. Avoiding Crash has always been a headache in the development process. Lint can check out some potential Crash very well. E.g:

The native NewApi is used to check whether the API provided by the higher version of Android is called in the code. Calling a higher version API on a lower version device will cause a crash.
Custom SerializableCheck. For a class that implements the Serializable interface, if the object referenced by its member variables does not implement the Serializable interface, it will crash during serialization. We have formulated a code specification that requires the class that implements the Serializable interface, and the declared types of its member variables (including those inherited from the parent class) must implement the Serializable interface.
Custom ParseColorCheck. When calling Color.parseColor()the method to parse the color sent from the background, the incorrect format of the color string will cause an IllegalArgumentException. We require that this exception must be handled when calling this method.

Bug Prevention

Some bugs can be prevented by lint checking. E.g:

SpUsage: All SharedPrefrence read and write operations are required to use the basic tool class, and various exception handling will be done in the tool class; at the same time, the SPConstants constant class is defined, and all SP keys must be defined in this class to avoid scattered definitions in the code. conflict.
ImageViewUsage: Check if ImageView has ScaleType set, and whether Placeholder is set when loading.
TodoCheck: Check if there are TODOs in the code that are not completed. For example, some fake data may be written in the code during development, but make sure to delete these codes when it is finally launched. This kind of check item is special, and it is usually checked in the test stage after the development is completed.

Performance/Security Issues

Some performance and security related issues can be analyzed using Lint. E.g:

ThreadConstruction: Direct use of new Thread()creating threads (except thread pools) is prohibited, and a unified tool class is required to perform background operations in the common thread pool.
LogUsage: Direct use is prohibited android.util.Log, and a unified tool class must be used. In the tool class, the Release package can be controlled not to output Log, which improves performance and avoids security issues.

code specification

In addition to code style constraints, code specifications are more used to reduce or prevent bugs, crashes, performance, security and other issues. Many problems are technically difficult to check directly. We indirectly solve them by encapsulating a unified basic library and formulating code specifications. Lint checks are used to reduce the cost of communication within the group, the cost of learning for newcomers, and to ensure the implementation of code specifications. E.g:

The aforementioned SpUsage, ThreadConstruction, LogUsage, etc.
ResourceNaming: Resource file naming convention to prevent resource file name conflicts between different modules.

Implementation of Code Inspection

When a code problem is detected, how to remind the developer to correct it in time?

In the early days, we configured static code inspection on Jenkins, and when we packaged and released AAR/APK, we checked for problems in the code and generated reports. Later, it was found that although static code inspection can find a lot of problems, few people take the initiative to read the report, especially there are too many irrelevant and low-priority problems in the report (such as too strict code style constraints).

Therefore, on the one hand, it is important to determine which problems to check, and on the other hand, when and by what technical means to perform code inspections. Combined with technical implementation, we thought more about this and identified the main goals in the implementation of static code inspection:

Focus on high-priority issues and block low-priority issues. As mentioned earlier, if the code inspection report contains a large number of unimportant problems, it will affect the discovery of key problems.
The solution of high-quality problems must be mandatory. When a high-priority code problem is found in the inspection, a clear and direct error report is given to the developer, and the developer is forced to fix it through technical means.
Certain problems are discovered as soon as possible to reduce risks or losses. Some problems are discovered as early as possible. For example, the API of a higher version of Android is used in the development of business functions, which can be checked out through Lint's native NewApi. If it is found during development, other technical solutions can be considered at that time, and when implementation is difficult, it can be communicated with products and designers in time; but if it is discovered only when the code is mentioned, tested, or even released and launched, it may be too late.

priority definition

Sevirity (priority) can be configured for each Lint rule, including Fatal, Error, Warning, Information, etc. We mainly use Error and Warning, as follows.

Error level: Identify the problems that need to be solved, including crashes, clear bugs, serious performance problems, non-compliance with code specifications, etc., and must be fixed.
Warning level: including code writing suggestions, possible bugs, some performance optimizations, etc., and relax the requirements appropriately.

execution time

Lint checks can be performed in multiple stages, including local manual checks, real-time coding checks, compile-time checks, commit checks, as well as checking when submitting Pull Requests in the CI system, checking when packaging and releasing, etc., which are described below.

do it manually

In Android Studio, custom Lints can be Analyze - Inspect Coderun manually through the Inspections function ( ).

In the Gradle command line environment, you can directly use ./gradlew lintLint to check.

Manual execution is simple and easy to use, but it lacks compulsion and is easily missed by developers.

Real-time checking during encoding phase

Checking while coding means reporting errors in the code window in real time when writing code in Android Studio. Its benefits are obvious, developers can find code problems at the first time. However, due to the imperfect support of Android Studio for custom Lint, the configuration of the developer's IDE is different, and the developer needs to actively pay attention to the error report and fix it. This method cannot fully guarantee the effect.

IDEA provides Inspections and corresponding APIs to implement code inspection. Android native Lint is integrated into Android Studio through Inspections. For custom Lint rules, the official does not seem to give clear instructions, but actual research found that under the conditions of Android Studio 2.2+ and JavaPsiScanner-based development (or Android Studio www.yibaoyule1.com 3.0+ and JavaPsiScanner/UastScanner), IDE Will attempt to load and execute custom Lint rules in real time.

technical details:

In Android Studio 2.x version, the menu Preferences - Editor - Inspections - Android - www.cnzhaotai.com Lint - Correctness - Error from Custom Lint Check（avaliable for Analyze|Inspect Code）states that custom Lint only supports command line or manual running, not real-time inspection.

Error from Custom Rule When custom (third-party) lint rules are integrated in the IDE, they are not available as native IDE inspections, so the explanation text (which must be statically registered by a plugin) is not available. As a workaround, run the lint target in Gradle instead; the HTML report will include full explanations.
In the Android Studio 3.x version, after opening the Android project source code, the IDE will load the custom Lint rules in the project, which can be viewed in the Inspections list of the settings menu, which has the same effect as native Lint (Android Studio will open the source file when the source file is opened. trigger a code inspection for that file).
IssueRegistry.getIssues()Analyzing the method call stack of the custom Lint , you can see that in the Android Studio environment, the custom Lint rules are executed by the org.jetbrains.android.inspections.lint.AndroidLintExternalAnnotatorcall load.LintDriver

Reference code: https://github.com/JetBrains/android/tree/master/android/src/org/jetbrains/android/inspections/lint

The actual effect in Android Studio is shown in the figure:
write picture description here

Automatically check when compiling locally

Configure Gradle scripts to perform Lint checks when compiling Android projects. The advantage is that the problem can be found as early as possible, and it can be mandatory; the disadvantage is that it has a certain impact on the compilation speed.

When compiling an Android project, the assemble task is executed. If assemble depends on the lint task, the Lint check can be performed at compile time. At the same time, LintOptions is configured, and the compilation is interrupted when an Error level problem is found.

The configuration in the Android Application project (APK) is as follows, and the Android Library project (AAR) can be applicationVariantsreplaced libraryVariantswith.

android.applicationVariants.all { variant ->
    variant.outputs.each { output www.thd178.com/  ->
        def lintTask = tasks["lint${variant.name.capitalize()}"] output.assemble.dependsOn lintTask } }

Configuration of LintOptions:

android.lintOptions {
    abortOnError true
}

Check on local commit

Using the git pre-commit hook, you can perform a Lint check before the local commit code. If the check fails, the code cannot be submitted. The advantage of this method is that it does not affect the compilation speed during development, but the problem is relatively lagging behind.

In terms of technical implementation, you can write a Gradle script to automatically copy the hook script from the project to the .git/hooks/folder every time you synchronize the project.

CI check when submitting code

As part of the code submission process specification, it is a common, feasible and effective idea to use the CI system to check Lint problems when sending Pull Requests. Configurable CI checks pass before code can be merged.

Jenkins is commonly used in CI systems. If you use Stash for code management, you can configure the Pull Request Notifier for Stash plug-in on Stash, or configure the Stash Pull Request Builder plug-in on Jenkins to implement a Job that triggers Jenkins to perform Lint checks when a Pull Request is sent.

Both local compilation and code inspection in the CI system can be implemented by executing Gradle's Lint task. You can pass a StartParameter to Gradle in the CI environment. If this parameter is read in the Gradle script, configure LintOptions to check all Lint problems; otherwise, only some high-priority Lint problems are checked in the local compilation environment, reducing the speed of local compilation. influence.

The effect of Lint's report generation is shown in the figure:

write picture description here

Check for package release

Even if the CI system is used to perform a Lint check every time the code is submitted, it is still not guaranteed that everyone's code will be merged without problems; in addition, for some special Lint rules, such as the aforementioned TodoCheck, it is hoped to be checked at a later time.

Therefore, when the CI system packages and releases the APK/AAR for testing or release, it is necessary to do another Lint check on all the codes.

Finalized inspection timing

Considering the advantages and disadvantages of multiple inspection methods and our goals, we finally decided to combine the following methods for code inspection:

During the coding phase, the IDE checks in real time and finds problems at the first time.
When compiling locally, check for high-priority issues in time, and compile after passing the check.
When the code is submitted, CI checks all the problems, and the code can be merged only after the check is passed.
In the packaging phase, the project is completely checked to ensure that nothing goes wrong.