Black box testing is also called functional testing, data-driven testing or functional testing based on requirements specifications. This type of testing focuses on testing the functional requirements of the software. Treat the test object as a black box, completely disregarding the internal logical structure and internal characteristics of the program, and only check whether the function of the program conforms to its function description according to the "Requirements Specification" of the program. The tester does not need to understand the internal structure of the program code, completely simulates the end user of the software product to use the software, and checks whether the software product meets the user's needs. The black-box testing method can better and more realistically examine the realization of the functional requirements of the system under test from the user's point of view. In all stages of software testing, such as unit testing, integration testing, system testing and acceptance testing, black box testing plays an important role, especially in system testing and confirmation testing, its role is irreplaceable by other testing methods .
"Black box" test model: From the perspective of using software or programs, test from the corresponding relationship between input data and output data.
effect
If there is a problem with the design of the external feature itself or the specification is wrong, the black box test cannot find it. Therefore, black box testing cannot replace white box testing. Black box testing focuses on testing the functional requirements of the software. It is a test on the program interface, mainly to find the following errors:
- Functional errors or missing functions;
- interface errors or interface errors;
- performance errors;
- Whether it can correctly receive input data and produce correct output results; (input, output)
- Whether there is a data structure error or an external information access error;
- Are there errors in program initialization and termination.
main content
(1) Acceptance test.
Black box testing is to accept the test output results from the interface of the software, which has the characteristics of acceptance testing.
(2) α/β test.
The test is a test performed by members of the project team on the software under test, and the alpha/beta test is a test in which people outside the project team participate. Alpha/beta testing is also suitable for black box testing. That is to say, when the test finds an error and the developer revises it, the project manager will also make corresponding adjustments to the product plan, and the product features are constantly being revised.
(3) Menu/Help Test.
In the process of software testing, the developers will fix the errors found by the testers and modify some functions of the software. At the same time, the project manager will also adjust the characteristics of the software according to the situation. Therefore, in the process of software development and testing, all functions can be adjusted. Therefore, in the final stage of software product development, the most problems are often found in the documentation.
(4) Release test.
Before the official release, the product is tested very carefully. In addition to dedicated testers, thousands or even hundreds of thousands of other users and collaborators are required to test the product through use. Then the error information is fed back to the technical department when it comes to the release test. If there is an error that must be corrected, the release of the software must be postponed. During the postponement time, the software product needs to be re-tested comprehensively, which will consume a lot of time, manpower and physical resources.
(5) Regression testing.
At this stage, it is first checked whether previously found errors have been corrected. Regression testing keeps corrected bugs from recurring and doesn't create new bugs.
(6) RTM test.
RTM testing refers to the testing conducted during the product release phase. During this testing phase, every error needs to be corrected with high-level approval. Because it is very easy to modify the software at this time to produce other errors, only those errors that cannot be repaired will be allowed to be modified. If the software still has many serious errors during the release phase, it cannot be released on time.
standard
The set of test cases designed in black box testing needs to meet the following two criteria:
- The designed test cases can reduce the total number of test cases that need to be designed to achieve reasonable testing.
- The test cases are designed to tell us whether there are certain types of errors, rather than just pointing out whether there are errors related to a specific test.
Test Methods
1. Equivalence class division
The method of equivalence class division is to divide the input domain of the program into several parts (subsets), and then select a small number of representative data from each part as test cases. The representative data of each class is equivalent to other values in this class in the test, that is: if one example in a certain class finds an error, other examples in this equivalence class will also find the same error. mistake.
An equivalence class refers to a subset of an input domain. In this subset, the individual input data are equivalent for uncovering errors in the program, and it is reasonable to assume that testing a representative value of some equivalence class is equivalent to testing other values of this class. Therefore, all the input data can be reasonably divided into several equivalence classes, and one data in each equivalence class is taken as the input condition of the test, so that a small amount of representative test data can be used to obtain better test results.
There are two different cases of equivalence class division: effective equivalence class and invalid equivalence class.
Effective equivalence class: It refers to the set of input data that is reasonable and meaningful for the specification of the program. Valid equivalence classes can be used to check whether the program implements the functions and performance specified in the specification.
Invalid equivalence class: The exact opposite of the definition of a valid equivalence class.
To design a test plan using the equivalence class division method, it is first necessary to study the function description of the program (software), then divide the equivalence classes of the input data, and design test cases according to the equivalence classes divided. Test case design meets the following principles:
1. Specify a unique number for each equivalence class;
2. Design a new test case to make it cover as much as possible the valid equivalence classes that have not been covered until all valid equivalence classes are covered; 3
. Design a new test case to cover the invalid equivalence classes that have not been covered as much as possible until all invalid equivalence classes are covered.
example:
A city's phone number consists of two parts. The names and contents of these two parts are:
1) Area code: three or four digits starting with 0 (including 0);
2) Telephone number: seven or eight digits starting with non-zero or non-1.
Assuming that the program being debugged can accept all phone numbers that meet the above requirements and reject all numbers that do not meet the requirements, please use the equivalence classification method to design its test cases.
Requirements: divide equivalence classes first, and then cover all equivalence classes according to coverage rules.
2. Boundary value analysis
Boundary value analysis is by selecting test cases on the boundary of equivalence classes. Boundary value analysis not only pays attention to the boundary of the input condition, but also must consider the boundary of the output domain. It is a supplement to the equivalence class partitioning method.
Boundary value analysis test case design principles:
- 1. If the input condition stipulates the range of values, the value just reaching the boundary of this range and the value just beyond the boundary of this range should be taken as the test input data.
Example: Enter the value range "1-9", then the input test case can be: 1, 9, 0.9, 9.1- 2. If the input condition specifies the number of values, use the largest number, the smallest number, one less than the smallest number, and one more than the largest number as the test data.
Example: An input file can have 1-64 records, then the input test cases can be: 1, 64, 0, 65 (records)- 3. According to each output condition stated in the specification, use the previous principle 1.
- 4. If the input domain or output domain given by the specification of the program is an ordered set, the first element and the last element of the set should be selected as test cases.
- 5. If an internal data structure is used in the program, the values on the boundary of the internal data structure should be selected as test cases.
- 6. Analyze the specification to find other possible boundary conditions.
3. Causality diagram
Both equivalence class division and boundary value analysis focus on the consideration of input conditions, but do not consider the connection and combination of input conditions. The cause-and-effect graph method fully considers the mutual combination and mutual restriction relationship between the input conditions.
Basic notation for causal diagrams:
- Identity: Indicates a one-to-one correspondence between cause and effect. If the cause occurs, the effect occurs; if the cause does not occur, the effect does not occur.
- Not: Indicates a negative relationship between cause and effect. If the cause occurs, the effect does not occur; if the cause occurs, the effect occurs.
- Or: It means that if one of the several reasons occurs, the result will appear, and only when none of these reasons appear, the result will not appear.
- And: It means that the result will only appear if several causes appear. If one of the causes does not occur, the effect does not occur.
Notation for constraints in a causal diagram:
- E (mutually exclusive): Indicates that the two reasons a and b cannot be established at the same time, and at most one of the two may be established.
- I (inclusive): Indicates that at least one of the three reasons a, b, and c must be true.
- O (unique): Indicates that there must be one of a and b, and only one is established.
- R (required): Indicates that when a appears, b must appear. It is impossible for a to occur and b not to occur.
- M (masked): Indicates that when a is 1, b must be 0. And when a is 0, the value of b is uncertain.
Basic steps to generate test cases with causal diagrams:
- In the description of the analysis software specification, which are the causes (that is, the input conditions or the equivalent classes of the input conditions), which are the results (that is, the output conditions), and assign an identifier to each cause and effect.
- Analyze the semantics in software specification descriptions. Find out the corresponding relationship between cause and effect, and between cause and cause. Based on these relationships, draw a cause-and-effect diagram.
- Some combinations of causes and causes and between causes and effects are impossible due to grammatical or environmental constraints. To indicate these special cases, constraints or restrictions are marked on the causal diagram with symbols.
- Convert the cause-and-effect diagram into a decision table.
- Take out each column of the decision table as a basis to design test cases.
The test cases generated from the causal diagram (local, under the combination relationship) include the cases of taking true and false of all input data, and the number of test cases formed reaches the minimum, and the number of test cases increases linearly with the increase of the number of input data Increase.
4. Composition method of decision table
The decision table has been used in the previous causal diagram method. Decision tables are tools for analyzing and expressing situations where multiple logical conditions perform different operations.
- Condition Stub: Lists all the conditions of the problem. The order in which the conditions are listed is generally considered to be immaterial.
- Action Stub: Lists possible actions specified in the question. The order in which these operations are listed is not constrained.
- Condition Item (Condition Entry): Lists the value of the condition on its left column. True and false values in all possible cases.
- Action Item (Action Entry): lists the actions that should be taken under the various values of the condition item.
Rules: The specific value of any combination of conditions and the corresponding operations to be performed. A column running through condition items and action items in a decision table is a rule. Obviously, as many sets of condition values are listed in the decision table, there are also as many rules, that is, as many columns as there are condition items and action items.
Steps for establishing the decision table:
① Determine the number of rules. If there are n conditions, each condition has two values (0, 1), so there are 2n kinds of rules.
② List all condition piles and action piles.
③ Fill in the condition item.
④ Fill in the action item. Wait until the initial decision table.
⑤ Simplify. Merge similar rules (same action). (optional)
example:
The calculation plan of a certain school’s teaching remuneration is as follows:
(l) The basic teaching remuneration is 10 yuan per class;
(2) If the number of students in the class exceeds 40, the teaching remuneration will increase: basic teaching remuneration × 0.1;
(3) If the number of students in the class exceeds 60 For each person, the remuneration increases: basic remuneration × 0.2;
(4) If the teacher is an associate professor, the remuneration increases: basic remuneration × 0.1;
(5) If the teacher is a professor, the remuneration increases: basic remuneration × 0.2
(6) For lecturers, the remuneration will not increase;
(7) For teaching assistants, the remuneration will be reduced: basic remuneration × 0.1.
5. Random test
Random test means that the test input data is randomly selected from all possible input values, which is a basic "black box" test method. Random selection uses random simulation methods, including pseudo-random number generators and hardware random simulators to generate input data. This method can obtain a large amount of test data, and testers only need to specify the value range of the input variable, and provide the necessary transformation mechanism when needed, so that the generated random numbers obey the expected probability distribution.
6. Guessing wrong method
Guessing wrong method is based on experience and intuition to speculate all possible errors in the program, so as to design test cases in a targeted manner.
The basic idea of guessing wrong method is: to enumerate all the possible errors in the program and the special cases where errors are prone to occur, and select test cases according to them. Error-prone situations in the program include: ①The input data is zero or the output data is zero; ②If the number of input or output is allowed to change, the number of input or output is 1 or 0, etc. In addition, it is necessary to carefully analyze the specification of the program, pay attention to find out the missing or omitted parts, so as to design a corresponding test plan, and check whether the programmer handles these parts correctly.
7. Scenario
method Almost all software uses event triggers to control the process. Event-triggered scenarios form a scenario, and different trigger sequences and processing results of the same event form an event flow. This kind of thinking in software design can also be introduced into software testing, which can vividly depict the situation when an event is triggered, which is helpful for test designers to design test cases, and at the same time makes test cases easier to understand and execute.
Basic flow and alternative flow: As shown in Figure 1, each path through the use case in Figure 1 is represented by the basic flow and alternative flow, and the straight black line represents the basic flow, which is the simplest path through the use case. Alternative streams are represented by different colors. An alternative stream may start from the basic stream, execute under certain conditions, and then rejoin the basic stream (such as alternative streams 1 and 3); it may also originate from another alternative stream. flow (such as alternative flow 2), or terminate the use case without rejoining a flow (such as alternative flows 2 and 4).
Advantages of black-box testing: It is suitable for functional testing, usability testing and acceptability testing; testing program functions against instructions; it can test the working logic of long and complex programs, and is easy to understand.
Disadvantages of black-box testing: It is impossible to conduct a complete and complete input test, and some software bugs or artificial faults cannot be detected through black-box testing. Precisely because the test data for black-box testing comes from the specification, the main disadvantage of this approach is that it relies on the correctness of the specification. In fact, one cannot guarantee that the specification sheet is completely correct. If redundant functions are specified in the specification, or some functions are omitted, this is completely powerless for black box testing.