The value of building a UI automated testing system follows the testing pyramid model, which recommends testers to write and execute tests at different levels. UI testing requires writing and designing test scripts, executing a complete application, and simulating user interaction with the application, so the testing speed of UI testing is relatively slow. However, UI testing has a high business coverage. By simulating user behavior in real scenarios and measuring the functionality and stability of UI components in detail, it can effectively eliminate a large number of repetitive testing problems faced by manual testing.
In April 2022, a UI automation testing community survey conducted by the MeterSphere open source project team showed that among 219 survey forms, 82.19% of the respondents stated that their team was building a Web-side UI automation testing system.
▲Figure 1 Test pyramid model
■ The bottom layer of the testing pyramid is unit testing. Unit testing is the testing of small pieces of logic in the code. These small pieces of logic can be functions, classes, or methods in the class. By writing unit tests, developers can directly call the code under test to evaluate whether its output is as expected, without relying on other components, services, or UI.
■ The middle layer of the testing pyramid is interface testing or integration testing. Interface testing mainly tests the application program interface (API), focusing on whether the interfaces between different modules are valid, whether the information transmitted is correct, whether it meets the interface standards, etc.; the main goal of integration testing is to test whether the entire system can work together.
■ The top level of the testing pyramid is UI testing. UI testing is performed from the most intuitive perspective. In addition to manual execution, it can also rely entirely on automation. UI testing can simulate each user's interaction with the application, such as clicking buttons, entering information, filling out forms, etc., and running the application to see whether it meets the expected results.
It can be seen that the scope of these three test types is different. Unit testing is mainly used to find logic errors at the most basic level. Unit testing is faster and requires fewer resources during runtime; interface testing or integration testing mainly verifies whether the interface between the application's services and the database can work well together, focusing on the interactions between different components. Interaction; UI testing focuses on whether the complete application can be launched. UI testing is the most comprehensive type of testing and therefore tends to consume the most computing resources and time when running.
1. The necessity of UI automated testing
UI automated testing is becoming more and more important in the field of software testing. Its necessity is reflected in improving testing efficiency, reducing costs, and ensuring the accuracy and consistency of testing. It plays an important role in improving software quality and development efficiency.
1. Replace manual repetition
Traditional manual testing requires test engineers to spend a lot of time performing repetitive operations, which is not only inefficient but also error-prone. By using UI automation tools, test engineers can hand over repetitive testing tasks to programs, thereby saving time and energy and spending more time on designing comprehensive test cases and testing new functions;
2. Improve regression efficiency
Regression testing is testing performed after changes have been made to software code to ensure that modifications do not negatively impact existing functionality. Since regression testing needs to be executed frequently, manually executing regression testing wastes a lot of time and human resources. UI automation can efficiently perform regression testing, significantly reducing labor costs while ensuring the accuracy and comprehensiveness of testing;
3. Test during non-working hours
In traditional manual testing, test engineers can only perform testing tasks during working hours. By using UI automation testing tools, test engineers can arrange the execution of test tasks during non-working hours, thereby making better use of time resources, spending more working time analyzing failed use cases and solving problems, and improving test quality;
4. Ensure operational consistency and traceable results.
In manual testing, due to human factors, it is easy to omit or neglect operating steps, leading to inaccurate test results. By using UI automated testing tools, you can ensure that the steps and verifications executed for each test are consistent and repeatable, avoiding the occurrence of human errors.
2. When to intervene in UI automated testing
People often ask, what kind of projects are suitable for UI automation testing? UI automated testing is equivalent to simulating manual testing and operating controls on the page through programs. During the actual testing process, we often encounter problems such as the inability to find controls or the maintenance costs caused by changes in control definitions. We have summarized some time nodes or scenarios suitable for UI automation testing intervention:
1. The product is relatively stable and there will be no frequent demand changes;
2. Core businesses and core interfaces that require frequent regression verification;
3. Choose a stable business process. It is not recommended to choose a business module that is in the exploratory period and needs to be changed frequently;
4. Suitable for projects with long maintenance cycles, not suitable for short-term projects;
5. If you use automated testing tools implemented by writing codes, most of the test team members need to have script development capabilities.
3. Two core issues faced by UI automated testing
In UI automation testing, testing teams generally face two core problems, namely the maintenance cost and learning cost of UI automation.
■ Maintenance costs
UI automated testing relies on the positioning and operation of interface elements and is sensitive to interface changes. Once the interface design changes, the test script may become invalid. Therefore, test scripts need to be updated in a timely manner, thus increasing the maintenance cost of UI automated testing;
■ Learning cost
UI automated testing requires the use of specific testing frameworks and tools. The learning curve of many tools is relatively steep. It is necessary to master the use of corresponding programming languages and tools, as well as test design skills, which increases the learning cost of testers.
4. How to improve the efficiency of UI automation testing?
UI automation testing needs to simulate the manual operation interface, which means it takes a relatively long time to complete the task. Especially when the number of test cases is huge, the execution time of UI automation testing will be longer. This is because the principle of UI automation testing determines that it cannot be executed as quickly as interface automation. The main ways to improve the efficiency of UI automation testing include:
1. Improve use case selection strategy
For use cases in core business and stable business, clear priorities can be defined first to ensure that these use cases are executed first. For uncovered use cases, you can select based on business importance and risk priority, and gradually expand the scope of testing;
2. Simplify use case writing
Abstract repeated test steps and implement them using variables and loop structures to avoid writing independent test processes for each different test data. Use the test process control function to control execution branches through conditional judgments to avoid unnecessary processes and reduce redundant operations during the test process;
3. Reduce assertion settings
The way to use assertions should be as concise and efficient as possible. Consider the necessity of assertions when designing use cases in the early stage, reduce unnecessary assertion settings, and reduce the frequency of communication with the application, thereby increasing the execution speed;
4. Reduce screenshots of test steps
Only take screenshots at necessary steps to avoid taking screenshots of the same content repeatedly. Reduce interception of large-size image files and reduce resource usage;
5. Set a reasonable test waiting time
Analyze the characteristics of the test environment and set waiting times based on stability and response speed. Use the intelligent waiting mechanism to wait for UI elements to be loaded before performing operations to avoid excessive waiting times;
6. Enable multi-threading
Analyze the dependencies of test cases and execute independent test cases in different threads at the same time. Effectively utilize system resources and increase overall speed through parallel execution;
7. Improve server configuration
Increase the processing power and memory capacity of the server to ensure that there are sufficient available resources for UI automation testing. Use load balancing technology to distribute requests to multiple servers to improve overall execution speed and performance.
5. MeterSphere natural language UI automated testing
The MeterSphere open source continuous testing platform covers functional modules such as test tracking, interface testing, performance testing and UI testing. MeterSphere's UI test module is built based on the Selenium browser automation solution and embeds highly reusable test scripts. The test team can carry out low-code automated testing without complex code writing.
1. Written in natural language, low maintenance cost, low learning cost
① Low maintenance cost
Regardless of any testing tool, maintenance costs cannot be zero. Regardless of whether the product is ToB or ToC, it must interact with customers. Therefore, product releases, interface updates, operating activities on specific holidays, etc. will all cause changes to the product. Therefore, a certain maintenance cost is inevitable, but the maintenance cost of UI automated testing can be effectively reduced by selecting appropriate testing tools and cultivating good testing habits.
The UI test module of the MeterSphere open source continuous testing platform supports free combination and free reference of elements, instructions, and scenarios. Elements are the most basic components, instructions represent a collection of common test steps, and scenarios are usually a complete business process. On the MeterSphere platform, elements can be referenced in instructions and scenes, and instructions and scenes can also reference each other. When the product interface changes, the interface elements will also change accordingly. At this time, by updating the element, the instructions and scenes that reference the element will automatically update the element information, which can reduce excessive maintenance costs and workload. Just focus on maintaining the elements. Similarly, when business processes change and existing instructions or scenarios are added, deleted, modified, or edited, all reference relationships will be updated simultaneously. Through flexible reference design, the maintenance cost of subsequent testing is greatly reduced.
Figure 2 is the "rush purchase" UI automation scenario test model written based on the MeterSphere page object mode:
▲Figure 2 MeterSphere-based “rush purchase” UI automation scenario test model
② Low learning cost
There are many automated testing frameworks on the market, including those based on browser interfaces, mobile applications, desktop applications, etc. When we make a choice, we should first choose a popular automated testing framework, which has been tested by a wealth of projects; secondly, we should choose a testing framework written in a mainstream development language, such as Java or Python. The framework is not only highly versatile, but will also be used in the future. It is easier to find relevant solutions when encountering problems during use.
The UI test module of the MeterSphere open source continuous testing platform is fully compatible with the Selenium open source framework. The UI test script is implemented using natural language programming, and the operation instructions are encapsulated in the form of visual controls. Testers can complete the test script through interface-based click, drag and drop operations without writing code.
■ Traditional coding implementation
▲Figure 3 Traditional coding implementation
■MeterSphere natural language orchestration implementation
▲Figure 4 MeterSphere natural language orchestration implementation
2. Advantages of natural language UI automated testing compared to code implementation
Compared with UI automated testing implemented by traditional coding, the advantages of UI automated testing written in natural language are as follows:
■ Higher efficiency
Natural language UI automation can quickly create test cases, reducing the time and error rate of handwritten code, and improving testing efficiency;
■ Lower threshold
For testers who are not familiar with programming languages, writing in natural language is easier to get started, lowering the threshold for learning and use;
■ More intuitive operation
Through interface-based orchestration, testers can more intuitively understand the operation process in test cases, making it easier to find and correct errors;
■ Easier to maintain
Visual interface layout design can help testers easily maintain and update test cases. Update test cases synchronously by modifying common test elements to quickly respond to changes in applications and requirements.
In contrast, the method of using code requires testers to master the knowledge of programming languages and frameworks. Every detail needs to be carefully considered when writing code, and it is relatively easy to make mistakes. At the same time, the way the code is implemented is relatively difficult to maintain and modify, and requires a deeper understanding of the logic and structure of the code. Therefore, in terms of UI automated testing, using natural language orchestration is a more efficient and intuitive way.
Lei Jun: The official version of Xiaomi's new operating system ThePaper OS has been packaged. The pop-up window on the lottery page of Gome App insults its founder. Ubuntu 23.10 is officially released. You might as well take advantage of Friday to upgrade! Ubuntu 23.10 release episode: The ISO image was urgently "recalled" due to containing hate speech. A 23-year-old PhD student fixed the 22-year-old "ghost bug" in Firefox. RustDesk remote desktop 1.2.3 was released, enhanced Wayland to support TiDB 7.4 Release: Official Compatible with MySQL 8.0. After unplugging the Logitech USB receiver, the Linux kernel crashed. The master used Scratch to rub the RISC-V simulator and successfully ran the Linux kernel. JetBrains launched Writerside, a tool for creating technical documents.