7 years ago (2013), an article with the same title was published in InfoQ, but this article is completely new . Before answering "what exactly is agile testing", let me ask a question: do you understand agile development?
If you don't understand it, you need to understand agile first. For example, read an article I wrote before Scrum is no longer Scrum, Scrum is still Scrum , which can help you understand agile. To understand agile, it is more important to go to agilemanifesto.org to carefully read the famous agile manifesto and 12 principles of agile development.
With thinking, listen to the case I will tell you below. In the process of listening, you can examine the case to judge which ones conform to agile values and which ones violate the principles of agile development. Finally, let’s analyze the case together. And answer "what exactly is agile testing".
*** Cases of transformation from traditional testing to agile testing ***
This case comes from a domestic company whose products are mainly smart terminals based on the Android system. The story happened 6 years ago, in 2013, when the company's software R&D department started a series of agile testing-oriented attempts .
Let me introduce the background of the case first . The company's software R&D department has four development departments and a huge testing department. Among them, a department responsible for the development of various R&D tools is also affiliated to the testing department. The ratio of developers and testers is almost 1:1. The responsibilities of the development department are divided according to the responsible functional modules, while the testing department is responsible for all tests at the software system level, including functional testing, performance testing, security testing, and reliability testing. , Compatibility testing, etc. At that time, the traditional waterfall development model was adopted, that is, the V model, and code writing and product testing were clearly divided into two stages.
-
continuous integration attempt
The tools used here include: distributed code version control tool Git + code review tool Gerrit + continuous integration tool Jenkins + self-developed automated testing framework based on Monkey Runner. Based on Git + Gerrit + Jenkins, the development department has realized the automatic construction of the code.
The desired goal is to complete automatic building, automatic deployment, and automated testing after the code is submitted, and automatically generate test reports. In the implementation process, there is no problem with the tool chain, and there is no problem with automatic construction and automatic deployment. The problem lies in the automated testing.
There are about 1,000 manual test cases for a product, but there are only more than 100 test cases that can be converted into automated scripts and executed in an integrated environment for a long time, and only version verification is implemented, which is what we usually call " smoke test". This means that when the developer submits the code, the coverage of the triggered automated test is very limited. Even if the integrated environment can support continuous verification, everyone feels that it is very tasteless.
-
Test-forward and organizational attempts
The definition of test-forward at that time was testing at the software coding stage, rather than waiting until the end of development to start testing. Simply put, it is testing while developing, hoping to shorten the product development cycle through this .
(1) The first stage
The development department is divided according to Scrum Team, and test engineers are recruited according to the ratio of 3:1. At first, I didn't want to understand what kind of testers they needed, so the recruits were basically manual tests, and most of them couldn't understand the code.
Their main tasks in the Scrum Team include : manual testing; reproducing bugs according to development requirements over and over again; doing chores for developers, such as updating a new version of the terminal, finding the hardware model that the development wants to verify, and so on. In the early stages of the project, when the product hardware is not in place or the software integration does not work together, manual testing cannot be carried out.
(2) The second stage
The leaders originally hoped that the agile model could reduce the number of testers, but contrary to expectations, the number of people in the test department responsible for system testing did not decrease, but dozens more testers were added within the development department. The reason is that the testing methods used by the two groups of people are similar. If there are more people, it is simply reflected in the fact that there are more bugs reported. What the testing department values is the coverage rate of the requirements, and the number of people cannot be reduced.
Therefore, in a reorganization, the leader announced that all testers would be transferred to the test department, and asked the test department to reduce the corresponding amount of test outsourcing. Scrum Team can ask the testing department to equip testers in a ratio of 3:1. The testing department can understand the testing scope of the Scrum Team, reducing repeated testing. After the reorganization, the number of people has been reduced, but manual testing is still the main method. Due to changes in the organizational structure, development and testing are often arguing over who has the final say on the testing in the development phase. Development and testing have become more distinct, and the relationship between Even more nervous .
(3) The third stage
After a period of operation, I felt that this was not feasible and did not conform to the characteristics of an agile team, so I decided to transfer the personnel responsible for testing during the development phase to the development department. A good change is: each Scrum Team begins with a senior test engineer as Test Owner, responsible for formulating test strategies and test plans, and coordinating the arrangement between development testing and system testing. The development department has also begun to train the development capabilities of test engineers in the development phase.
-
A unit test attempt
At the beginning, the coverage rate of unit tests was almost 0, and developers only wrote code and fixed defects submitted by testers. Due to the failure of continuous integration and test forwarding, everyone believes that the development department should require developers to do unit testing, and measure the results of unit testing by code coverage. The development also promised to do it, but it has not been effective for a whole year. The reason is: busy, no experience and skills in unit testing .
-
An Attempt to Test Capacity Renewal
The testing department is aware of the importance of automation, but only 5% of the engineers in the department are responsible for automated testing. After layers of applications, the company agreed to replace 10% of the manual test engineers with the last elimination system. Through various methods of internal transfer, external recruitment, and employee training, the proportion of automated test engineers finally reached 25% after one year. The team began to build a unified automated testing framework. The coverage rate of automated testing in API and UI testing has finally seen a significant increase, but the overall requirement coverage rate has not exceeded 30%, and the lack of unit testing is still flawed. Without the participation of developers, testing is always at the UI layer Tossing, of course, is getting twice the result with half the effort.
From this example, it can be seen that in the process of transforming from traditional testing to agile testing, the R&D department of this company did not know what real agile testing is, but crossed the river by feeling the stones and kept trying. It was difficult and took a lot of detours. In the end, it has not yet reached the other side of agile testing. Not to mention, the quality of products and the efficiency of testing have been significantly improved.
*** What is Agile Testing? ***
So what exactly is agile testing? What is certain is that "agile testing" is neither a testing method nor a testing method, but a complete set of software testing solutions specially designed to adapt to agile development . This solution should be able to support continuous delivery, covering the required, correct values, way of thinking, testing process, a series of excellent testing practices and a more suitable testing environment, automated testing framework and tools. Agile testing can adopt various existing testing methods and methods, and the emphasis is different from traditional testing, but the main differences are values, testing thinking, processes and practices, etc.
Agile testing should have the values advocated by the "Agile Manifesto", so we can write the following "Agile Testing Manifesto" according to the format of the "Agile Manifesto":
Agile testing emphasizes "collaboration with development", "automated testing", "customer thinking" and "dynamic testing strategy adjustment" are more valuable.
Then let's go back and look at the above cases, at least 1 and 2, they didn't do it: the testers didn't get enough attention and respect, and the development and testing collaboration wasn't enough, but:
-
There was a separate testing department for a while, "development and testing became more distinct, and the relationship became more tense"
-
"Without the participation of developers, testing is always tossing at the UI layer, of course it is half the effort"
-
"Triggered automated tests achieve very limited coverage"
-
……
In the early stage of transformation, the training of relevant testers was not strengthened, and the requirements of agile mode for testers were not even known, and the testers recruited were unqualified. In the implementation process, there is a lack of testing strategies, and no emphasis is placed on starting from the needs of customers and dynamically adjusting testing strategies.
Agile development still has 12 principles. Did the team in the above case follow them one by one and in a targeted manner? Although the 12 principles of agile development do not seem to talk about testing, testing is a part of the entire software development, so it is natural to abide by these principles and adapt to the basic requirements of agile development, such as:
-
How to support or assist "continuous and early delivery of valuable software"?
-
How to Embrace Change - "Easy to face changing requirements, even late in development"
-
……
Only by adhering to these principles can one obtain the correct posture to adapt to agile development, and only by adopting excellent practices of agile development, such as test-driven development, and closely cooperating with development, testing will not become a "stumbling block" of agile development.
Based on the 12 principles of agile development, I developed the following 8 principles of agile testing:
These principles are explained in detail in the follow-up content of the "Efficient Agile Testing" column.
reference: