How to do automated testing? Real enterprise automated testing process, automated testing classification...

Contents: Guide

foreword

Enterprise automation testing process

The automated testing process includes four stages: test analysis and planning, test design and development, test execution, and test summary

1. Test analysis and plan
Automated feasibility analysis:
Before conducting demand automation testing, it is necessary to confirm whether test automation can be implemented, and if necessary, conduct sampling demo analysis (such as involving third-party systems, etc.).

Automated testing should follow the following prerequisites:

Infrequent changes in requirements:
the project cycle is a long enough period of time;
automated test scripts can be reused in the later stage;
sample demo analysis is to conduct a deeper feasibility analysis of the entity case, the selection of the demo, generally directly select the smoking use case and write it as a test script After execution, check whether the script can run and pass, and whether all the designed test points are covered;

Test requirements analysis:
After confirming that automated testing of requirements is feasible, sort out the requirements and divide the function points that can be automated. The function points are too complicated, and a lot of time will be spent on script writing, which will lose the original intention of automation;

At this stage, to determine the automated test coverage and automated test granularity, the principle is to increase the automated test coverage as much as possible during the project cycle;

2. Develop a test plan.
The test plan mainly includes the following contents:
Access and exit: the time point of automated test intervention, and when the standard is reached, the automated test ends.
Test scope: determine the priority of test requirements and business scope
schedule: at what point in time What is the output?
Risk assessment: Estimate the risks in the project process and provide solutions
. Required resources: personnel, software, hardware, documents and other resources required for testing.
Work arrangement: the division of labor for specific personnel

3. Test design and development
Test case design:
first design the test case, after the test case is written, review the use case, and then develop the test script after the review is passed.

The principles to be followed in the process of designing automated test cases:
a use case is a complete scenario, covering the core business process;
priority is given to implementing positive test cases, supplemented by individual important reverse use cases;
data computing use cases need to be in the data The expected results are provided in the driver for comparison;
use cases for newly added and modified classes need to re-query and compare the query results after the addition and modification are successful;
try to avoid dependence between use cases;
after the use cases are tested, the test scenarios need to be Restore (data clearing, etc.) so as not to affect the execution of other use cases;

4. Script development and version control
According to the written test cases, write the corresponding automated test scripts. The writing of the test scripts follows the script development specifications. The maintainability, reusability, simplicity and robustness of the scripts should be considered during the writing process .

At the same time, pay attention to ensure the structure and consistency of the automation project. After the script is developed, it must be run successfully at least three times before it can be considered that the script has no problems.

Before writing scripts on the same day, pull the master branch code locally to ensure that the local code is the latest version. After the script writing and debugging are completed on the same day, upload it to your own branch, and later unify Merge to the main branch

5. Test execution
When the iterative requirement system test has been completed (such as after the iterative requirement is released online), use the Jenkins tool task trigger method to execute the script unattended

Effective bugs found during the execution process are recorded on Huawei Cloud and submitted to the corresponding developers for repair. The title is prefixed with [Automated Test] for later tracking and processing

After the developers fix the BUG, they need to perform a regression test on the BUG. If the modification plan of the problem deviates from the original requirements, the use cases and scripts need to be modified before the regression test.

6. Test summary
Summarize the results of automated tests, count the found bugs and their causes, and submit test report documents

7. Automated test report
Summary of version operation: automated test coverage, pass rate, defect rate, and quality data compared with the previous version every time a new function is launched

Daily operation summary: post-release automated test quality tracking, problem analysis, and determination of defect causes
Include:
all automated statistical reports, including historical data of all script runs and script success rate trends in the past 7 days;

A single automated statistical report, counting the detailed information of a single run script, divided into failure statistics and all statistics and log review information;

Single automated statistical report, detailed information of a single use case, including checkpoints, execution steps;
finally, maintain, optimize and archive scripts and test cases for test reports;

Automated testing classification

1. Unit-unit test
The unit test needs to test the DAO layer and the Service layer. The unit test of the data-driven layer mainly ensures the correctness and rationality of the SQL script, and the unit test of the Service layer mainly ensures the correctness of the logic of a single function function

The external dependencies of the unit test can be simulated and returned through the Mock framework (Mockito, etc.), so that the automated test can be run anytime and anywhere as much as possible. In a sense, the input-output ratio of this layer of automated testing is the highest

2. Service-interface test (integration test)
in the service layer interface test focuses on whether the input and output functions of an interface are correct, as well as the logic test of the business process. The initial test automation started late, and the code coverage rate was not high, so the initial stage The interface automation strategy is to increase the coverage of business scenarios

Prioritize the coverage of core business scenarios:
the data that the interface depends on chooses the method of using the return value of the interface;
temporarily do not make mocks between systems, and consider more dependencies between systems;
prioritize scene coverage, and then consider code coverage;

This can quickly increase the coverage of business scenarios, and at the same time, the written API interface can be used even if the business process of the product changes in the later stage (the interface logic of the UI layer remains unchanged). Scripts written in this way have lower maintenance costs in the later stage .

The interface test is completed by the tester. During the interface test, the tester does not need to have a very detailed understanding of the implementation of the internal interface code. The script written reflects the sorting out of the relationship between different interfaces and modules, and the coverage of the core business process is higher in the later stage. Coverage can be increased for a single interface in the case of

Decoupling of interface dependencies, such as changing data from calling interfaces to directly inserting data into the database;
gradually refining and splitting business scenarios;
gradually changing from system-level automated testing to gray-box and white-box automated testing;

3.
The UI-page test layer is mainly the page display logic and the integration verification of the interface front-end and the service. Proper UI automation testing is necessary, because UI automation can perform repetitive work for testers when the version is iterated and returned. Liberate the labor force.

Only automate coverage of core functions, and improve script maintainability as much as possible.
UI automation test scripts follow the Page Object design pattern;
reduce dependence on background data through data mocking;

UI testing is done by testers. After covering the core scene, there is no need to invest too much energy and resources in the UI layer, because the UI interface is the layer with the highest frequency of changes, and the later maintenance costs are too high

Continuous integration
With automated testing at all levels, a continuous integration system needs to be established

Process automation to improve automation efficiency
Timing and quantitative repeated execution to maximize the value of automated test scripts

Purpose:
code submission to automatically execute unit tests;
automatically deploy the overall environment after unit tests pass;
automatically execute integrated automation tests (Service/UI);
automatically generate detailed test reports for construction, and automatically notify relevant personnel;

It can promote the development of unit test coverage, formulate interface automation script writing specifications, increase the readability, effectiveness and reusability of the code, and make the script design as far as possible away from the dependence of test, pre-release and online three environmental data , Increase the coverage of core business process UI automation tests under controllable time

When automated testing at all levels is relatively mature, activities such as automatic collection of code coverage, code static inspection, and other left-shifting activities, as well as collection and monitoring of online data and bugs (interface performance, crash rate, and user behavior analysis) can be added later. etc.) to test right-shift activities, build a test platform that conforms to the company's style, etc.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled