program instrumentation
- Brief introduction of the method
The program instrumentation method is simply a method to achieve the purpose of testing by inserting operations into the program under test.
Questions to consider
a. What information to detect
b. Where to set the detection point in the program
c. How many detection points need to be set - Assertion statements
Insert certain statements (types, values) to determine the characteristics of variables in specific parts of the program
program mutation
Unlike the aforementioned structural and functional tests, it is a bug-driven test. (For a specific type of program error)
It is almost impossible to find all the errors in the program. The solution is to narrow the scope of the error search as much as possible, so as to focus on the possible errors that are most harmful to the software.
- Strong program variation
When the program is developed and simply tested, the remaining errors in the program are some small errors that are difficult to find.
The goal of program mutation is to detect these simple errors and their combinations.
Program variation means that for each element in the test data set, the program P and its variation factor must be tested, so it is required that the test data set D and the variation factor set m(P) need to be carefully selected. This is the key to the success of the strong mutation method .
m(P) is obtained by making small changes to P, and it is also a program, called the variation factor of P. Suppose P has a test data set D, if P is correct on D, you can find a certain set of variation factors of P M = M ( P ) ∣ M ( P ) is the variation factor of P M={M(P)|M(P) is the variation factor of P}M=M ( P ) ∣ M ( P ) is the variation factor of P. If every element in M has an error in D, the correctness of the program is considered to be high.
In layman's terms: that is, the program P is correct on the test set D, but the variation factor set M is all wrong on the test set D, and the correctness of the program is considered to be high.
If there is no error in D for some elements in M, there may be three situations:
a. These variation factors are functionally equivalent to P. (Strongly to avoid)
b. Existing test data are insufficient to find the difference between P and its variation factor. (Add test data)
c. P may contain errors while some of its variators are correct.
Budd once listed some mutation operations, because there are too many, they are omitted here.
Weaknesses of program mutation:
a. To run all mutation factors, thus multiplying the cost of testing.
b. Deciding whether a program is equivalent to its mutator is a recursively unsolvable problem. - Weak program mutation
Mutation methods that only test the program under test are called weak mutations and do not produce a variation factor.
The five most basic program components:
a. Variable reference: the use of variable values.
b. Variable definition: assign a new value to the variable.
c. Arithmetic expressions: the difference between the expression and the correct expression is a constant; the expression is a constant multiple of the correct expression; the coefficient of the expression is wrong.
d. Relational expression: The difference between a relational operator error and a constant error.
e. Boolean expressions: Arithmetic relations are complex relations connected by the three logical operators NOT, AND, and OR. B = L ( E 1 , E 2 , … , E n ) B = L(E_1, E_2, …, E_n)B=L ( E1,E2,…,En) , using different test sets such that( E 1 , E 2 , … , E n ) (E_1, E_2, …, E_n)(E1,E2,…,En) are various combinations of truth values.