8-year test summary, how does the project/team do automated testing? Efficiency value? Hematemesis finishing...

Contents: Guide

foreword

Python automated testing: https://www.bilibili.com/video/BV16G411x76E/

What is the goal of automation?

From a personal point of view, by introducing automated testing tools, you can save time and improve testing efficiency. (Really?)
At the same time, it can reflect one's own code power, enhance one's own value and bargaining power (well, it seems to be like this).
From a team perspective, what were our expectations when we decided to introduce automated testing?

At least the following points:
break through existing bottlenecks, improve testing efficiency, and reduce labor costs (note that reducing labor costs should not be placed at the core, otherwise it will be counterproductive); reduce human
error rates, avoid human fatigue and inertial thinking and Errors caused by opportunism;
improve execution efficiency, and deal with high-intensity continuous rotation tasks, get long-term system stability tests and stress tests for high-concurrency scenarios; increase
software trust, explain this point, personally think that automated testing is used To ensure the quality of the product offline, when we execute the correct automation script, we can trust that the current delivery is basically reliable;

Cost of introducing automation?

From a personal point of view, the investment in automated testing is basically time cost (whether you spend time in the company or go home to study), but from the perspective of the team, things will become more complicated up. We need to comprehensively consider some influencing factors.

There are mainly three aspects:

Time
The duration of the project is short: when there are some projects with a high degree of urgency, and it only takes one month from project establishment to completion, if automated testing is forced to be done at this time, the maintenance cost will be very high, which is not acceptable.

Projects exist for a short time: some outsourcing projects or projects of political missions (such projects are actually not uncommon). It will not exist for a long time, or there is no need for long-term testing, so there is no need to consider introducing automated testing.

Cost Ability
cost of team members: If automated testing is to be introduced, the tester's ability requirements will be relatively high. For example, he must understand the protocol, the transmission process and principle of the protocol, and have certain coding skills. The cost of such personnel will be relatively high.

Cooperation cost of the R&D team: If you want better automation, you must have standardized things. Because testing is not the source of the R&D process. So there are very few things we can change, we can only try and push developers to make some changes.

The cost of time investment: Whether it is developing scripts or maintaining scripts, it requires time investment. Although in the long run, the efficiency of automation will definitely increase. But for a certain iteration, it is necessary to take a part of the original functional testing time to write scripts and maintain scripts. So can these time investments be guaranteed and counted as working hours, instead of relying on team members to use their spare time to invest?

Efficiency issues
When introducing automated testing tools, the tools don't tell you what to automate and what is worth automating.

Blindly chasing automation coverage is not the right thing to do.

In my opinion, we can try from two directions:
Risk-based automated testing: We should test the function points with the highest risk of failure first, and these functions will also bring the greatest negative consequences if the failure occurs.

For example: function points that will affect the use of core processes, relatively concentrated bugs in the testing process, function points that affect service level agreements (SLAs), and function points that can cause economic losses.

Automated testing based on ROI: We know that based on the repair cost of BUG, the lower level of automated testing can generate higher value. We will recommend introducing interface/integration automated testing first to find problems as early as possible.

When we can balance the relationship between these two, the introduction of automated testing can produce real value and be implemented in the long run, otherwise it will easily become a cosmetic project and die halfway.

Mistakes in automated testing?

Doing automated testing for the sake of automated testing
without planning in advance, and managing the goals and expectations of introducing automated testing, is just copying what others say. Blind fantasy, thinking that automated testing can save money, thinking about automating it, how much labor costs can be saved;

Why can't automated testing find BUG?
Discovering more new defects should be the main purpose of manual testing, and automated testing cannot be expected to find more new defects. In fact, automated testing is mainly used to find original defects. It is used for regression testing, and a large number of new business tests still rely more on manual testing.

Automation tools that can record and play back are the best.
In the early automated testing tools, there will be recording and playback functions. How tasteless this function is, students who have actually used it should know that if you say too much, you will cry.

Several automated tests commonly used by enterprises:
interface automated testing direction: Python+requests+pytest+yaml+alluer+Jenkins;
web automated testing direction: Python+selenium4+pytest+POM+allure+Jenkins;
app automated testing direction: Python+ appium+POM+pytest+allure+Jenkins;

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled