The most basic requirement of test case design: coverage of the functions to be tested. This is the most basic requirement, but don’t look at it as a simple sentence. To achieve comprehensive coverage, you need a comprehensive understanding of the functions of the tested product and a clear test scope (especially to clarify what does not need to be tested) , Possess basic testing techniques (such as: equivalence class division, etc.), etc. So is the test case designed to meet the above requirement a good test case?
Answer: In theory yes, but not in actual engineering. The reason why there is such a difference between theory and reality is that in theory, there is something not to be considered, but in actual engineering, it has to be considered - cost. The costs here include: test planning costs, test execution costs, automated test cases, test automation costs, test analysis costs, and additional costs introduced by test implementation technical limitations, bugs in the test environment, human factors, and unpredictable random factors, etc. .
Due to the intervention of cost factors, it is determined that the principle of well-designed test cases in the project is not limited to "covering the functions to be tested in the residence". The following are the other four principles that I have summarized based on my work experience. Make bricks and correct me. These principles are especially for test cases that need to be automated and executed frequently.
1. The principle of minimizing the coverage of a single use case
This principle is the "boss" among all these four principles, and it is also the most easily forgotten and ignored in engineering, and it more or less affects several other principles. Here is an example to introduce, if you want to test a function A, it has three sub-function points A1, A2 and A3, there are two ways to design test cases:
Method 1: Cover three sub-functions with one test case - Test_A1_A2_A3,
Method 2: Use three separate use cases to cover three sub-functions - Test_A1, Test_A2, Test_A3
Method 1 is suitable for small-scale projects, but method 2 is a better choice for projects with a little scale and quality requirements, because it has the following advantages:
The coverage boundaries of test cases are more clearly defined
Test results are more directional to product problems
The least coupling between test cases, the less interference between each other
The direct benefit of these advantages is that the debugging, analysis and maintenance costs of test cases are minimal.
Each test case should be as simple as possible, and only verify what you want to verify. Don't "catch the grass and beat the rabbit" to bring in everything, which will only increase the burden and risk of the test execution phase.
David Astels described in his book "Test Driven Development: A Practical Guide" that it is best to have only one Assert statement for a test case. In addition, simple and clear test cases that cover functional points are also convenient to combine to generate new tests. The concept of Ordered Test was introduced in Visual Studio.
2. Principles of test cases replacing product documentation functions
Usually in the early stage of development (the first two days of each Scrum Sprint), we will use Word documents or OneNote to record product requirements, function descriptions, and any details that can be determined at present, and outline the appearance of the functions to be realized. It is convenient for the team to communicate and refine, and reach a consensus on product functions within the team. Assume that after we reach a consensus at this time, the described function is A. As the product development deepens, the team will have an updated understanding of the product's functions, and the product functions will be more specific and refined. At the end of an iteration or Sprint At that time, the final realized function is likely to be A+. In this way, after constantly listening and absorbing user feedback, modifying product functions, and after multiple iterations, the function that was originally described as A is likely to eventually become Z.
This is the time to look at the previous Word documents and OneNote pages, but the A is still recorded. The reason for this is that very few people are going to (and able to) keep updating those documents to accurately reflect the current accurate state of product functionality. It’s not that I don’t want to do it, but it’s really difficult! Note here: Early Word or OneNote documents are still necessary, which can at least ensure that the team has a consistent and accurate understanding of the functions to be implemented in the early iterations.
Is there nothing that consistently and accurately describes what a product does?
Answer: Of course there is, that is the production code and test cases. The product code realizes the product function, it must accurately describe the current function of the product, but due to various programming techniques, such as: object-oriented, abstraction, design patterns, resource files, etc., the product code is difficult to read easily Understand, it is often to read the code on the premise of knowing the product function, instead of looking at the code to understand the function.
Good code will have detailed comments, but the comments here are explanations and notes for the implementation code, not descriptions of product functions.
Then there are only test cases, and the test should faithfully reflect the product function, otherwise the test case will fail to execute. In the past, everyone just regarded test cases as test cases. In fact, the understanding of test cases should be raised to another level. It should be able to play the function of product description documents.
This requires that the test cases we write are sufficiently detailed, and the organization of the test cases should be adjusted and prioritized. It is far from being able to complete it with general-purpose tools such as Word, Excel or OneNote, and more dedicated test case management tools are needed. To assist, such as the introduction of Microsoft Test Manager in Visual Studio 2010.
In addition, for automated test cases (whether at the API or UI level), the code should also be written in a different style from the product code, and readability and descriptiveness should be key considerations. In the test code, of course, excellent design ideas such as object-oriented and design patterns can be introduced, but they must be used in moderation. Often, the process-oriented coding method is more conducive to organization, reading and description.
3. Principles of single-input cost and multiple-input cost
Rather than saying that this is a principle for judging test cases, it is better to say that it is a thinking angle and principle for thinking about problems. Cost is always the primary factor to be considered when making decisions for any project, and so is the test in the project. The consideration of cost should also be objectively and comprehensively reflected in the entire stage of test design, execution and maintenance. When you encounter some dilemmas in the test and need to make decisions, try to analyze it from the perspective of cost, which may be helpful to your decision-making.
The cost in the test can be divided into: single input cost and multiple input cost according to its time span.
For example: writing test cases can be regarded as a single input cost, because writing test cases is generally carried out in the planning stage of the test (the beginning stage of each Sprint of Scrum), although there will be small changes later, most of them are It is basically formed in the initial design stage; the same is true for automated test cases, which are also one-time investment; the execution of test cases (including: manual and automated test cases) is a multiple input cost, because each one When the new version is built, all test cases must be executed (or only high-priority test cases are executed for BVT testing), test results are analyzed, test case failures are debugged, and failure causes of test cases are determined (product defects, test case defects, test cases Framework flaws or random issues that lead to test case failures) to verify that the overall quality of the version meets the specified standards.
The reason why we want to introduce single and multiple costs is to distinguish the impact of different activities on test costs in the test, so as to help us rationally arrange the investment in different stages and make correct decisions, so as to ensure that in the limited Under the premise of affording the test cost, the test work can be carried out effectively to the maximum extent. For example, when we realize that the design and automation of test cases is a one-time investment, while the execution of test cases is a repeated investment, we should actively think about how to improve the efficiency of test execution that requires repeated investment. When one-time investment needs to be balanced with multiple activities, priority should be given to the efficiency of multiple-input activities. In fact, there is a lot of work that can be done here.
For example: the first principle - the principle of minimizing the coverage of a single use case - is a good example, test the three function points A1, A2 and A3 of the A function, on the surface, use the use case Test_A1_A2_A3 when designing and implementing automation The simplest, but it will cause many problems in the repeated execution phase:
First of all, the failure analysis of such a use case is relatively complicated, and you need to confirm which function point caused the test failure;
Secondly, the debugging of automation use cases is more complicated. If it is a problem with the A3 function point, you still need to go through A1 and A2 continuously before reaching A3, which increases the debugging time and complexity;
Third, manual test cases with many steps increase the uncertainty of manual execution, and automated test cases with many steps increase the possibility of failure of automatic execution, especially those use cases based on UI automation technology;
Fourth, (Last but not least) coupling irrelevant function points together reduces the possibility of early detection of product regression defects, which is a taboo in testing work. For example: If Test_A1_A2_A3 is an automatic test case and is executed in the order of A1->A2->A3, when there is a bug in A1, the entire test case fails, and A2 and A3 are not executed by the test. If the bug of A1 takes a long time to fix due to some reasons at this time, Test_A1_A2_A3 is always considered to have failed because of the bug of A1, while A2 and A3 are always not covered, and there are potential dangers and loopholes here .
When you finally fix the A1 bug before the product is about to be released, and take it for granted that Test_A1_A2_A3 should pass, the problems of A2 and A3 will break out at this time, and you have to continue to work overtime to fix the problems of A2 and A3. Not to be alarmist, when A2/A3's code is related to A1's bug fixes, the problem can be worse when you have a lot of test cases designed like this..., really! :(
To sum up, the design of Test_A1_A2_A3 only reduces the investment of one-time design and automation, but increases the burden and risk of test execution that requires multiple investments. Therefore, when decisions are required (in fact, such decisions are often Happened, especially when designing test cases) Choose Test_A1_A2_A3 or Test_A1, Test_A2 and Test_A3, be sure to consider the cost of the investment.
4. The principle of simplifying test result analysis and debugging
This principle is actually an extension and continuation of the previous one - the principle of single input cost and multiple input cost - for automated test cases. When writing automated test code, it is important to consider how to make test result analysis and test debugging easier, including: use case logs, debugging auxiliary information output, etc.
Because the execution of test cases belongs to multiple investments, testers have to analyze test results and debug test cases frequently, and the investment in this part of activities is considerable. Sometimes, some auxiliary APIs provided by the test framework can help to realize this principle well. For example: Coded UI Test provides a similar API, see - VS 2010 Test Function Learning (18) - Coded UI Test three must-know functions to assist in better debugging of automated test cases based on the Coded UI framework experience.
The testing theory points out the general direction for the testing work. In actual engineering, we need to constantly "activate" these theories, so that the theory and practice can better fit together. In my opinion, software engineering projects, whether successful or not, good or bad, are invaluable to each of us involved. As caring people, we have experienced many things that have never been mentioned in books. As long as you continue to observe, experience and summarize, you will have more knowledge, understanding and discovery of your own. Many people write books to praise the beauty of code and testing. In fact, engineering projects are also beautiful, just to see if you can look at it more objectively.
Finally: The following are the supporting learning materials. For friends who do [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse also accompanied me through the most difficult journey. I hope it can help you too!
Software testing interview applet
The software test question bank maxed out by millions of people! ! ! Who is who knows! ! ! The most comprehensive quiz mini program on the whole network, you can use your mobile phone to do the quizzes, on the subway or on the bus, roll it up!
The following interview question sections are covered:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. web, app, interface automation, 7. performance testing, 8. programming basics, 9. hr interview questions, 10. open test questions, 11. security testing, 12. computer basics
