Key words of this article: Mobile UI automation ideas
Hello everyone, I am Peng Yuyan. In the previous test exchange group, some students asked "Are there any cases of automated testing at work that can be shared?" Yes, there are.
Today I will describe in detail a [Application of UI Automation in Practical Work] . This is a real UI automation case made by our team before the shell house search. Because the APP version has been updated too many times and the code is missing, some The details cannot be fully described.
But I will share the ideas and process of doing this project in detail, hoping to help everyone.
This article is a dry goods warning, the full text is 5300 words, and the estimated reading time is 14 minutes.
Table of contents
1. Main categories of automated testing
2. Scenarios suitable for UI automation
1) [Regression Verification]
2) [Inspection]
3) [Mobile embedded test]
3. Practical cases of UI automation
1) Project Background
2) Project selection
3) Start installation and deployment
4) Execute the test plan
4. Thinking about UI automation
5. Sharing of learning resources for automated testing
1. Main categories of automated testing
The forms of automation can be mainly divided into interface automation and UI automation.
According to the test pyramid theory, the higher the top layer, the higher the ROI (input-output ratio).
Generally, everyone will increase their efforts to invest in the construction of interface automation, and UI automation will basically invest less in the construction of UI automation because the input-output ratio is too low.
But this does not mean that UI automation is completely unnecessary. In the following part, I will list 3 scenarios [suitable for UI automation].
2. Scenarios suitable for UI automation
Generally speaking, there are three scenarios that are more suitable for UI automation: regression verification, inspection, and mobile terminal embedded point testing.
1) Regression verification
When a project is launched or an app is released, traditional manual testing takes a lot of time. When it comes to the release day, especially for releases with major changes, colleagues use manual use case walkthroughs to verify and return, and the pressure is very high!
At that time, it usually took 30 minutes to 2 hours to go online for regression verification, which was a torment for testers and developers.
Sometimes there is not enough manpower, and products and operations are needed to assist in regression verification, which is very inefficient.
At this time, UI automation can be used to cover some functions with less function changes.
2) Inspection
The online environment is complex, and various unexpected problems are prone to occur, and manual inspection and testing are stressful.
for example:
-
operational configuration errors;
-
Some unconventional operations of users;
-
Compatibility issues with different mobile terminals.
It may have been a long time since these problems were exposed to users, to feedback to the development side, and then to development and repair.
In the past, colleagues generally conducted regular manual online inspections to find some online problems.
This does work, but the problem is:
Manual inspection will also consume a lot of testing time, and sometimes it is difficult to stick to it when you are busy.
At this time, you can use UI automation to cover some main processes to ensure that there are no problems with the main process.
3) Buried point test on the mobile terminal
If you keep doing simple and highly repetitive testing work, it will not only be inefficient, but also severely hit the enthusiasm of testers.
The most typical is the buried point test on the mobile terminal. The traditional test method is to operate on the mobile phone, trigger the buried point report, and then capture the packet through the mobile phone to obtain the buried point data, and then perform a test on each field according to the buried point document. One by one manual verification.
This operation is very mindless, but it actually takes a lot of time. If there are too many buried points, the test may not be completed in a day, and it also consumes the energy of the testers: every time the buried point is tested, I feel My eyesight has dropped a lot (useless eyes on data).
At this time, you can try to use UI automation to automatically trigger the buried points, and automatically verify the buried points through some verification logic.
4) In addition, UI automation can also solve:
-
APP Compatibility Test
-
App Competitive Product Comparison Test
-
APP data capture
-
...
If you have more UI automation practice cases, welcome to communicate in the comment area.
3. Practical cases of UI automation
The directory is as follows:
1) Project Background
2) Project selection
3) Start installation and deployment
4) Execute the test plan
1) Project Background
When I was still looking for housing in Keike, with the continuous growth of the company's business, there were more and more APP functions. Every time the APP was released, the testers needed to spend more energy than before on online regression verification.
According to statistics, the number of manual use cases for online regression verification in our department is about 1,000. If all use cases require regression testing, it will take at least 2 hours to complete, which will undoubtedly bring great benefits to APP testers. challenge.
If the use case regression is insufficient, bugs will be left in the production environment, which will bring a bad experience to users.
Traditional manual testing has been difficult to meet the daily testing needs. We found that many use cases can be realized through automation, which will greatly relieve the pressure of APP testing students.
Let me show you some screenshots of the table that converts "manual use cases" into "automation". Although the method is stupid, this is how we counted the coverage of automation scenarios.
Now that the use cases have been sorted out, we are ready to start developing the UI automation framework and scripts. It is definitely impossible to develop from scratch. In actual work, there is not so much time to reinvent the wheel, so the first thing we need to do is to choose A good wheel, and then develop on this basis.
2) Project selection
In the field of UI automation of APP, many excellent UI automation testing frameworks (tools) have emerged in recent years.
For example: Appium, Robotium, Instrumentation, UIAutomator, ATX, etc.
Because the small partners in the team are basically Pythoners, we prefer to consider frameworks that have good support for Python when selecting a testing framework.
After a round of comparison, it is found that Appium and ATX support Python better, and both have mature solutions for Android and iOS.
So later decided to choose between Appium and ATX.
Here are some comparisons between Appium and ATX:
At the beginning of the project, we also built the operating environment of Appium and ATX at the same time.
Later, because the installation and deployment of Appium is relatively complicated, the project startup and operation are slightly slower than ATX (the framework is heavier), and the configuration of Appium is more cumbersome.
So we gradually gave up Appium, and finally used ATX to build UI automation projects.
Here is a brief introduction to the ATX ecosystem:
It can be seen that ATX is still very powerful, and the project is constantly being maintained and updated.
However, the underlying UI automation driver framework alone is not enough, you also need a professional test framework to manage your automated test projects (use cases).
pytest and unittest are the most popular solutions.
We are delighted to find that someone has already made a scaffolding project on GitHub: ATX-Test.
ATX-Test is a UI automation testing framework based on ATX-Server, which can realize parallel testing of multiple devices and generate a unified test report.
The technology stack used includes:
Now that there are relatively useful projects, we can directly carry out secondary development under this framework, so as to avoid reinventing the wheel.
ATX-Test does not currently support iOS, and we can integrate the facebook-wda driver into it during the second transformation.
3) Steps to install and deploy
The prerequisites for using ATX are:
Python runtime environment (Python 3.6+ (community feedback 3.8.0 does not support, but 3.8.2 supports)
Android version 4.4+ (need to install uiautomator2)
iOS (need to install facebook-wda)
Installation and deployment steps
The first step is to install uiautomator2
(suitable for Android)
pip install --upgrade --pre uiautomator2
Initialize the Android phone
# init 所有的已经连接到电脑的设备
python -m uiautomator2 init
# 高阶用法
# init and set atx-agent listen in all address
python -m uiautomator2 init --addr :7912
For detailed usage documents of u2, please refer to:
https://github.com/openatx/uiautomator2
I won't repeat them here.
The second step is to install weditor
pip install -U weditor
start weditor
python -m weditor
weditor can capture Android control elements, automatically generate script code and perform Debug.
The third step is to install facebook-wda (suitable for iOS)
The MacOS runtime environment is required, and Xcode needs to be installed.
Package WebDriverAgentRunner
xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'platform=iOS Simulator,name=iPhone 6' test
WDA needs some configuration to run on the real machine, you can refer to this article:
ATX Documentation - How to install on iOS device
WebDriverAgent：
https://testerhome.com/topics/7220
After the configuration is complete, run the following command (you need to use the password of the Mac and the UDID of the device)
# 解锁keychain，以便可以正常的签名应用
security unlock-keychain -p $your-mac-password-here ~/Library/Keychains/login.keychain
# 获取设备的UDID
UDID=$(idevice_id -l | head -n1)
# 运行测试
xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination "id=$UDID" test
install facebook-wda
pip3 install -U facebook-wda
For the usage documentation of facebook-wda, you can refer to:
https://github.com/openatx/facebook-wda
I won't repeat them here.
The fourth step is the method of iOS grabbing controls. Generally, there are two commonly used methods:
Appium-ios-inspector project address provided by Appium team:
https://github.com/mykola-mokhnach/Appium-iOS-Inspector
Macaca's app-inspector tool project address:
https://macacajs.github.io/app-inspector/cn/
The fifth step is about cloud real machine cluster deployment
What is a cloud real machine cluster? To put it bluntly, it means that the real mobile phone (mobile phone management platform) can be remotely scheduled on the cloud platform.
Existing cloud real-machine clusters can generally be divided into: public cloud real-machine clusters and private cloud real-machine clusters.
The public cloud real machine cluster is a cloud real machine service provided by some companies in the testing industry.
The more famous ones are Testin, WeTest and so on.
-
Advantages: No need to build by yourself, and there are a large number of mobile devices to choose from.
-
Disadvantages: money, expensive.
For private cloud real machine clusters, you need to purchase real machine equipment and dedicated cabinets yourself.
We finally decided to build a private cloud real machine environment for real machine scheduling and management.
The real machine adopts the machine that has been eliminated from the test, and sometimes buys some popular models for UI automation testing.
Are there any advantages and disadvantages of private cloud real machine?
Advantages: The machine is your own, the platform is your own, you can play however you want.
shortcoming:
-
The cost of the real machine is also high.
-
If it is a wired connection, the battery will be charged for a long time, there will be a lot of wear and tear, and the battery will bulge. If it is left unattended during the holiday, the worst case will cause the cabinet to explode.
-
If it is a wireless connection, battery monitoring is another big problem, requiring frequent manual care.
The commonly used private cloud real machine management platform can use:
STF：https://github.com/openstf/stf
or,
atxserver2：https://github.com/openatx/atxserver2
atxserver2 can seamlessly connect to ATX, so we use atxserver2 to manage our cloud real machine.
4) Execute the test plan
Let's take a look at our entire test architecture:
Generate a test report:
If you prefer to use pytest, you can replace uiautomator2 + htmltestrunner with pytest + allure, which is also very fragrant.
Because some company secrets are involved, no specific demonstration will be given in this part.
If you are interested, you can build a UI automation test running environment by yourself.
4. Thinking about UI automation
If you really get in touch with UI automation, you will find that the playability of UI automation is quite strong, and for novice testing, starting from UI automation to learn automated testing is a very good route.
When you see your mobile phone operating the APP according to your preset script, you will have an inexplicable sense of accomplishment, and it will motivate you to continue to learn more.
I have seen many cases of "XXX from getting started to giving up". Most of them are because the learning path is too steep, which leads to hesitation in acquiring knowledge and slowly giving up.
However, UI automation itself is not difficult to operate, and it may be a little more troublesome to build the environment. Generally speaking, UI automation is still very friendly to novices.
Of course, UI automation cannot solve all problems, such as:
-
The APP version iterates too fast, and the UI changes frequently.
-
Test scenarios with complex operating procedures. For example, there are often various pop-up windows (Pinduoduo lets you cut a knife), and the pass rate of use cases will be very low.
-
Base code is meaningless, test scenario design is better than implementation. For example, in order to pursue the ultimate coverage of automated testing, automated testing may be performed for some very marginal functions, which are even rarely triggered by users. It is obviously meaningless to automate this part of the function.
-
Platformization is the trend. Some teams have abstracted the writing of automated scripts into "little bit" operations on the platform. It seems more convenient, but in fact, the writing efficiency is not as good as Coding, so you should weigh the pros and cons and choose the one that suits your team. automation scheme.
Finally, both interface automation and UI automation have their own limitations and scope of application, and the two should be complementary.
Don't leave the work that UI automation is not good at to UI automation.
There are more ways to play UI automation. Interested students can try it according to their company's business:
-
Use AI technology to predict and locate elements and improve the efficiency of script writing.
-
UI automation can leverage image recognition for side-by-side testing.
-
Combine with Maxim, Monkey or traversal testing tools to conduct precise stability testing.
-
Combine with performance testing tools such as Perfdog to conduct mobile performance testing.
If there is something wrong with the writing, please give me your advice. If you have other ideas, welcome to communicate with me in the comment area.
If after reading the above article, you don’t know how to start UI automation testing, I suggest that you study software testing in depth. Don’t worry about not having learning resources. Now I will offer you a 13G super practical dry goods testing learning resource, involving The content is very comprehensive. Including functional testing , linux database, python , interface automation testing , ui automation testing , performance testing...
Including the test software learning roadmap, and this test interview document , more than 50 days of test class videos, 16 surprise actual combat test items, more than 80 software test software, 37 test documents, 70 software test related questions, 40 A test experience level article, thousands of real test questions to share, as well as the 2023 software testing interview book, as well as various selected resumes for software testing job hunting, I hope it will be helpful to everyone... [Follow my VX public account : Programmer Xiaohao gets it for free~]