Exploration and best practice in the field of APP pipeline testing | Jingdong logistics technical team

1 background

Due to its particularity (need to connect to the test machine), APP-side UI automation is generally executed locally. The limitations of this execution method have the following disadvantages:

Low timeliness: R&D needs to notify the test every time after packaging, and then go to the packaging platform to pick up the package after the test, there is a time difference
Automated scripts cannot be used for R&D self-test or product acceptance: If you want to use automated scripts during R&D self-test and product acceptance, you need to build a corresponding operating environment and prepare a test machine. The cumbersome steps lead to abandoning the use of automation in R&D/products. In the process of manual verification, testing is often required to assist in placing orders and changing data, which is inefficient
The result of local execution has no persistent storage, which is not conducive to result measurement

2 Scheme analysis

In order to solve the above problems, it is necessary to build an assembly line. The commonly used assembly line platform in the CI/CD field is Jenkins. Jenkins is powerful and can be redeveloped, but the execution of APP automation requires windows execution machines, simulators/real machines. These requirements undoubtedly increase cost of testing. In contrast, the Bamboo platform is a pipeline platform developed based on Jenkins, which not only inherits many functions of Jenkins, but also supports graphical configuration, and is connected to other platforms such as the cyber platform.

As can be seen from the comparison in the figure below, the cost of reusing the existing Bamboo platform is lower, and the following changes need to be made: 1. The automation framework needs to be changed to the airtest framework; 2. The test report needs to be processed to meet expectations.

3 Program implementation

Architecture diagram:

flow chart:

Results of the:

4 Problems encountered during implementation and solutions

1. The debug package is completed by bamboo, but the release package is not successful.

Solution: Upload the sign.properties file in the certificate management, remove the absolute path information in the file, and use the relative path

2. The cyber platform cannot input Chinese characters

Solution: Use poco().set_text() instead of text() method

3. If the script fails to run, it prompts RuntimeError: unable to launch AndroidUiautomationPoco

Solution: The cyber machine has ATX, which will affect the poco initialization. Add code before poco initialization:

# 停止ATX
try:
shell("am force-stop com.github.uiautomator")
shell("/data/local/tmp/atx-agent server --stop")
except Exception as e:
print("兼容非赛博机器")

5 open issues

Access to the coding platform, only supports the airtest framework
. Impact: The original framework needs to be modified, and the cost is high. I have communicated with the cyber platform responsible for the product, and the cyber platform will develop the function of docking with other frameworks later.
There are a lot of data manually configured: modules, use cases, and use case sets.
Impact: In addition to test scripts, modules, use cases, and use case sets need to be configured separately, and the labor cost is high. Afterwards, it is necessary to consider that after the coding platform code is merged, the timing task will be triggered to automatically generate the corresponding modules, use cases, and use case sets.
Unable to specify the model.
Impact: Cannot specify the model for compatibility testing. I have communicated with the product of the cyber platform sub-rules, and there is no plan to modify this item yet.
The impact of Jingguanjia’s lack of access to the online packaging platform
: the apk package needs to be kept locally during the test process, which takes up local memory and is easy to confuse; version management cannot be performed, and the historical version cannot be provided in time when the problem needs to be reproduced; UED walk-through or product acceptance Only Beijing ME can contact R&D to pick up the package, which has poor timeliness; it cannot be connected to the assembly line.

6 Expected benefits

The pipeline not only solves the problem of insufficient environment configuration and test machines, but also has the following advantages:

Low learning cost and simple operation, it is expected to save two-thirds of man-hours
The execution results can be stored persistently, and can be combined with the automated measurement platform for data analysis later
Sunglasses atoms can be added to the pipeline, and Flutter exceptions can be monitored during UI automation testing
The online version can be used for daily monitoring to find problems in time

Author: Fan Wenjun, JD Logistics

Source: JD Cloud developer community Ziqishuo Tech