The art of software testing is a must-read book for testers. Reading this book two years ago gave me a lot of theory and guidance. Now I review it and organize some study notes according to my personal understanding.
Chapter 1 The Psychology and Economics of Software Testing
The most important sentence: The attitude of the tester is more important than the actual test process itself
1.1 The psychology of testing
Misunderstandings of testing
- Software testing is to prove that there are no errors in the software
- Software testing is to prove that the software can correctly complete its predetermined functions
- Software testing establishes a process of confidence that the software'does what it should do'
The above point of view is a misunderstanding of testing. Testing is not done to verify that the software has no problems, but to verify that the software has problems through testing, modify the original program errors, and increase the value of the software through testing, and improve the reliability and stability of the program. It is the value of testers.
Don't test the program just to prove that the program can run correctly. On the contrary, you should assume that the program has bugs at the beginning, and the test is to find as many bugs as possible.
Definition of test : Test is the process of executing a program in order to find errors
Psychoanalysis
If the test psychology begins to think that there is no error in the program, the subconscious will tend to achieve this goal, and it may choose fewer test data that cause the program to fail, and it is easy to miss the test; on the other hand, if you start to think that the program is wrong, you will be psychologically Tend to find program errors and prove one's own judgment. When designing test data, it is possible to choose a variety of data and methods, and it is possible to find more problems.
Successful test cases: can induce program errors and promote software quality improvement in a certain direction
Summary: There is no program without BUG, only undiscovered BUG
1.2 The economics of testing
It is impossible for software testing to find all errors, because it is impossible to exhaust all possible inputs, and software testing cannot be completely tested.
1.2.1 Black box testing
Black box testing
Data-driven testing, input/output-driven testing, the test target has nothing to do with the internal mechanism and structure of the program, focusing on judging whether the program is output according to the specification;
Test criteria
Exhaustive input test, judge all output results. In fact, it is impossible to exhaust all the inputs. Taking into account economics, one can only select some representative input data to determine the result; therefore, the goal of test input is to pass a limited number of test cases and discover more as much as possible. Questions in order to obtain the best test results.
Common methods : equivalence class, boundary value, scenario analysis, causality diagram, decision table, error inference, orthogonal experimental design, function diagram
Disadvantages: I don't know anything about the internals of the program, only 80% of the problems can be found, and 20% of the hidden problems require luck.
1.2.2 White box testing
White box testing
Logic-driven testing allows to check the internal structure of the program, internal logic, and test requirements to meet the design.
Test judgment standard: exhaustive path test, test case coverage rate of each execution path and control flow in the program. In fact, multiple if judgments in the program will produce a lot of possible execution logic, so it is impossible to test completely. Only select part of the known process path to execute the test process
Requirements for the white box test method:
- Statement coverage: Each statement in the program is executed at least once
- Branch coverage or judgment coverage: A branch in the program is checked at least once, that is, the true value judgment of each branch statement is executed once, and the false value judgment is also executed once
- Condition coverage: When the judgment formula contains multiple conditions, the value of each condition is required to be verified
- Path coverage: make the program execute along all possible paths
Therefore, white box testing cannot achieve complete testing. It can only find as many problems as possible through limited execution paths and condition judgments;
The final conclusion: the combination of white box testing and black box testing is an optimal testing strategy.
1.3 Principles of software testing
Principle 1: A required part of the test case is the definition of the expected output or result
The expected result is not defined in advance, there will be "what you see is what you want", and the subconscious will want to see the correct result. Pre-defining can overcome this psychological tendency. The two parts that the test case must contain:
- Data description
- Accurate description of the correct output result under the above input data
What is the problem? : I think there is a problem with something, it must be that I have a specific understanding about something, and the current situation is not in line with my cognition. This kind of cognitive difference or cognitive dissonance is the problem.
If there is no expectation for the output of the program, there will be no so-called accidents, and there will be no problems.
Principle 2: Programmers should avoid testing their own programs
1. Psychological factors, subconsciously do not want to find errors, think that there is no problem with the program written by yourself
2. Unable to change the way of thinking to expose errors in your program
3. The programmer may have a wrong understanding of the requirements, leading to errors in the program
Principle 3: Organizations that write software should not test their own software
The reason is the same as principle 2
Principle 4: The results of each test should be thoroughly checked
The results of the execution may be incompletely checked, and it is possible that the error itself has always existed, but it has been ignored
Principle 5: The writing of test cases should consider not only valid and expected input, but also invalid input and unexpected input
Many problems exposed by the software are discovered when the program is running in a new or unexpected way, so unexpected and invalid input can better find the problem
Principle 6: Check whether the program "does not do what should be done" and only tests the general, the other half of the test is to check whether the program "does what should not be done"
Principle 7: Avoid test cases that are discarded after use, unless the test software itself is a one-time software
Principle 8: When planning testing, you should not tacitly assume that you will not find errors
Testing is not a process of proving the correct operation of the software, but a process of executing the program in order to find errors. If no errors are found, it can only be said that the software has no errors in the current cognitive range, and there may also be errors in the unknown cognitive range;
Principle 9: The possibility of more errors in a certain part of the program is proportional to the number of errors found in that part
80% of errors in the program are caused by 20% of the code
Principle 10: Software testing is a very creative, very intellectually challenging work
Software testing is to design a reasonable test set, find more errors, and require creative thinking;