3.1 Overview of Software Testing Process
Software testing process and the software development process is similar, as well as several parts of the test plan, test design, test development, test execution and test evaluation.
(1) testing program. According to user needs report on the functional requirements specifications and performance indicators, the definition of the corresponding test report demand that all subsequent test work will be carried out around the test requirements. At the same time, select the appropriate test content, reasonable arrangements testers, test time and test resources.
(2) test design. Test design refers to the decomposition test planning stage to develop test requirements, subdivided into a number of executable test process, and select the appropriate test cases for each test process, to ensure the validity of test results.
(3) test development. Establish automated test process can be repeated use.
(4) test execution. Automatic test During the test development phase of the establishment, and found defects to track management. It is generally performed by a test unit testing, integration testing, validation testing and regression testing steps.
(5) testing and evaluation. Binding domain quantify test coverage and defect tracking reports on the progress of work and the quality and efficiency of application software development team of a comprehensive evaluation.
In the test procedure, the test performed by performing the following steps: unit testing, integration testing, validation and acceptance tests, shown in Figure 3.1.
Figure 3.1 Software test execution process
(1) unit test: By testing each minimum software modules, the implementation of a test program for each unit of the source code, the program checks whether each module is implemented correctly predetermined function to ensure that it can work.
(2) Integration Testing: The module has been tested for assembly integration, aimed at testing program structure issues associated with software design.
(3) validation testing: testing the software meets the functional and performance requirements in the requirements specification, software configuration is determined completely and correctly. At the same time testing software product can coordinate with the actual operating environment in other parts of the system (such as hardware, database and operating personnel).
(4) acceptance testing: software testing process as the final product quality, the main software allows users to test and re-execute a subset of tests have been done to ensure no introduction of new errors.
3.2 Unit Testing
1. Unit testing content
Means for testing the test program module, there are the following five main tasks: the test module interface, local data structure of the test, the boundary conditions of the test, the test execution path and error handling test, shown in Figure 3.2.
Figure 3.2 unit test task to be solved
1) Test Interface Module
Data stream by the module under test to test, checks the data out of the module is correct. Thus, a module must interface comprises a parameter table, the parameters of sub-module call, global data, file input / output operations for testing and the like. In particular, to the following.
(1) The number of modules coincides parameter accepts the actual number of module input parameters.
Types of formal parameters and actual parameters of the module (2) match the input.
Whether in the form of unit parameters and actual parameters used by the module (3) inputs are aligned.
(4) When the call other modules, the number of the actual number of transmitted parameters is called with the formal parameters of the module are the same.
(5) When the call other modules, actual parameter transmitted with the formal parameters of the calling module match.
(6) calls another module, the unit and the actual parameters of the transmitted form the calling module parameters are consistent.
(7) internal function call number, and order attribute parameter is correct.
(8) In the case of the module with a plurality of inlets, whether the current reference parameters independent of the inlet.
(9) whether the read-modify parameters.
(10) whether a global variable in all of their modules have the same definitions reference.
If includes an external I / O module within, you should also consider the following factors:
(1) file attributes are correct.
(2) OPEN and CLOSE statement is correct.
(3) the length of the buffer capacity of the recording match.
Whether to open the file before (4) during read / write operations.
(5) At the end of the file handle is closed the file.
(6) the text writing / input for errors.
(7) I / O error checking and whether to do a deal.
2) local data structure of the test
The test checks the integrity issues local data structure, such as a data type specification, initialize, defaults, etc., and to test the influence on the global data module.
In the course of the module must be tested inside the module can maintain the integrity of the data, including the contents of internal data, in the form of mutual relations and errors do not occur.
It is noted that the local data structure types of errors: incorrect or inconsistent type description; incorrect initialization or default values; variable name error, such as misspellings or writing error; underflow, overflow or an address error.
3) execution path testing
Important test case execution path module test, in which the basic execution paths and cycle path test can often find a lot of errors. Since the test must be able to discover the error calculation error, incorrect determination or normal control flow generated.
Common mistakes: incorrect or misleading arithmetic priority; mixed mode operation error; initialization error; the accuracy is not accurate enough; incorrect symbolic representation of the expression.
And a cover for the determination conditions, the test cases can be found the following errors: Error comparison of different data types; determining incorrect or incorrect; incorrect logic operation or priority; place due to an error should be equal accuracy can not be equal to variable; incorrect or nonexistent loop terminates; when it comes to recycling branch can not quit; inappropriately modify the loop variable.
4) Error Handling Measurement
Error handling inspection module contains errors or defects. For example, whether to reject unreasonable input, whether the error description is difficult to understand, whether fault location is wrong, whether the cause of error report is wrong, whether the processing of error conditions is incorrect, the error condition prior to the error handling is already causing system the intervention.
Key test error handling module when an error occurs on the job, error handling facilities which are valid.
5) boundary condition test
Testing the boundary conditions is the final step of the testing unit, boundary value analysis must be used to design test cases. The test is working properly at the boundary, the test module set to limit the data processing.
Some boundary-related data types, such as first, last, maximum, minimum, maximum, minimum, maximum, minimum and so on.
In the boundary conditions of the test, the test should be designed to check the following:
(1) if there is an error at the 0th, ...... 1 n times n cycles.
Takes the maximum value (2) calculation or determination, whether there is an error minimum value.
(3) data flow, control flow is exactly equal to, greater than, if the error is less than the determined comparison value.
Step 2. Test Unit
Unit testing is usually carried out at the coding stage. Complete source code through the preparation and review and verification, after confirming no syntax errors, began to design test cases for unit testing.
Module is not an independent program, in considering the test module, taking into account its links with the outside world, so use some auxiliary module to simulate other modules associated with the module under test. Auxiliary module is divided into the following two:
(1) driving module (Drive): is used on an analog module of the module under test, corresponding to the main program module under test for receiving test data, and sends the data to the module under test, the module under test start Finally, the output of the measured results.
(2) the stub (Stub): used to simulate the course of the module under test module invoked. Stubs were only a few general data processing, you do not need all the features of the sub-modules are brought in, but not do not do anything.
The module under test, it is associated with the drive module and the stub together form a "test environment", shown in Figure 3.3.
Figure 3.3 unit test test environment
3.3 Integration Testing
Process according to the software design requirements, the modules are connected together through unit testing, the software system as defined composition called "Integrated." Integration testing is to test for this process, according to the dependent relationship between the interface of the module. Figure 3.4 shows an example of the software hierarchy of FIG.
Because integration testing is not carried out in a real environment, but rather in a separate development environment or test environment, integration testing required for personnel selected from the general development group, under the supervision of the head of the development. Development team leader responsible for ensuring the implementation of full integration testing using appropriate test technology at a reasonable quality control and supervision. Integration testing, testing by an independent observer to monitor the testing.
3.4 schematic diagram of the software hierarchy
1. The main task of integration testing
Is one integrated test system software testing techniques assembly, according to design requirements of the individual modules together after assembly of the unit test, the software requires realistic system software structure, found in various errors related to the interface. Integration testing is mainly suitable for the following several software systems:
(1) require a higher software quality software systems, such as aerospace software, telecommunications software, the underlying system software.
(2) the use of a broader scope, a larger number of users of the software.
(3) using the software development language program with a pointer similar to C / C ++.
(4) libraries, middleware and other products.
The main task of integration testing is to address the following five issues:
(1) The modules are connected together, each module calls the inspection data via the interface is lost.
(2) combining the respective functions, checking functions can achieve the expected requirements.
Function is (3) a function module to another module would be adversely affected.
If (4) global data structure in question, would not be unusual modifications.
Error (5) whether the accumulated single module is enlarged, so as to achieve an unacceptable degree.
2. Integration Testing Method
The main test software integration testing of structural problems, so the test based on module interface, mostly black-box testing, white box properly supplemented. Integration testing should follow the following steps.
Step 1: Check the relationship between the composition of a complete system modules.
Step 2: interaction and communication needs between the evaluation module, confirm the interfaces between modules.
Step 3: generating a set of test cases.
Step 4: an incremental test modules sequentially added to the system test process and logical / functional sequence is repeated.
Integration testing process with particular attention to the test key module, the key module typically has one or more of the following features:
(1) corresponds to several demands simultaneously function;
(2) having a high-level control functions;
(3) complex, error-prone;
(4) special performance requirements.
The main object of the test is to verify the integration and interaction of each of the interface modules of the software system composition, and therefore the integration of data from the test requirements and the difficulty of the content is not very high. Integration testing generally do not use real data, you can use a portion of representative test data. When creating test data, boundary conditions should ensure that data fully tested software system.
Including non-incremental integration testing integration testing and incremental integration testing.
1) Non-incremental Integration Testing
Integration Testing Non-incremental-step method for testing, after an individual unit tests for all modules, the program structure according to FIG connecting the modules together, the connection procedure after the test as a whole.
2) Incremental Integration Testing
Incremental integration testing method comprises incremental top-down test, and a bottom-up test incremental integration testing sandwiches from.
(1) Incremental top-down test. Top-down test stepwise incremental and gradual integration structural diagram according to a top-down test. The integrated module is the first order integrated master module (main program), and then follow the hierarchy of control software integration downwardly. Top-down manner may be integrated and depth-first strategies breadth-first strategy, shown in Figure 3.5.
FIG down 3.5 schematic incremental self test
(2) bottom-up incremental test. Incremental bottom-up test from the "atomic" module (software module structure of the bottom), beginning with the bottom-up structure of FIG gradual integration and testing. Figure 3.6 represents a bottom-up process of incremental test.
3.6 a bottom-up schematic diagram of an incremental test
(3) sandwich integration testing. Integration also called sandwich hybrid integration, it top-down and bottom-up rolled into the advantages and disadvantages. Sandwich system is divided into three layers are integrated with an intermediate layer as a target layer. The top layer of the target layer using top-down test integrated test mode, the use of bottom-up strategy integration layer below the target layer, the target layer is finally tested.
As illustrated using the procedure pile S2, S3 and S4, the user interface M1 test 3.7, using drivers D5 and D6 function of the bottom module M7, M8 and M9 were tested. When the entire integrated system, the program pile S2, S3 and S4 into the intermediate layer module M2, M3 and M4, drivers D5 and D6 of the replaced module M5 and M6 of the intermediate layer, the intermediate layer such that the functional module is to be tested.
FIG integration testing sandwiches schematic 3.7
3.4 confirmation test
For confirmation tests to verify the validity of software, i.e., to verify the functionality and performance of software and other characteristics match the user's requirements.
In the testing phase needs to be done to confirm the work shown in Figure 3.8. You must first test the effectiveness of the review as well as software configuration, installation and acceptance testing and testing after by the experts, to become a software deliverable.
FIG test confirmation step 3.8
1. validity test
Validity of the test under simulated environment, the use of black box testing methods to verify the software under test meets the requirements listed in the requirements specification. For this purpose, test plans, test provisions of the kind to do, to develop a set of test steps, describe the test cases. To determine whether the software features consistent with the requirements by implementing predefined test plans and test procedures to ensure that all the software functional requirements are met, all of the software to achieve performance requirements, all documents are correct and easy to use.
2. Software Configuration Review
The purpose of the review is to ensure that software configuration software configuration of all components, including the actual operating environment of the system are complete support environment, the quality of all aspects of compliance with all the requirements. In validation testing process, should strictly abide by the provisions of steps to use the user's manual and operating manual, in order to check the completeness and correctness of the documentation, recording omissions and errors found, and appropriately supplement and correct.
3.5 Acceptance Test
User acceptance testing is mainly to test, software developers and quality assurance personnel should also participate. By the user to participate in the design of test cases, test data input through the user interface, the output of the analysis results of the test. The actual data is generally used in the production test. During the test, in addition to considering the function and performance of the software, but also to deal with portability, compatibility, maintainability, error recovery features and other software to confirm.
3.5.1 α test and β test
After delivery of the software, users will be how to actually use the program for developers is unpredictable. Because the user issues such a misunderstanding of the software operation method will often occur during use; abnormality data combination; generating for some users seem to be clear, but for others it hard to understand the user output, and so on.
α and β test test is used to find errors may be the end-user can find. α test to test a user may be performed in the development environment, the test may be performed within the company user in simulation of the actual operating environment. α test is a test performed under a controlled environment. The purpose of the test is FURPS α evaluation of software products that function, usability, reliability, performance and support, with particular emphasis on the interface and features of the product. Α commence after the end of the test can be started from the encoded software product, or starting after the module (partition) the test is completed, the test may be confirmed that the product during certain degree of stability and reliability.
β test is a test conducted in one or more of the user's actual environment by a plurality of users of the software. And α test different is that developers often do not test site. Therefore, β test is carried out in the field application software developers under uncontrollable environment. In the β test, the user record all the problems encountered, including real and subjective judgments, and regularly report to the developer; the developer after a comprehensive user reports, make changes, and finally the software products delivered to all users use. β test focuses on supporting the product, including documentation, customer training and support for production capacity. Only when α test reaches a certain level of reliability, β can begin testing.
3.5.2 Regression Testing
Any stage of the software life cycle, as long as the software has changed, you can give the software defect caused the problem. The change may be due to software errors found and made changes, but also may be due to integration or maintenance phase added a new module and other circumstances. Regression testing is a test to verify the integrity and accuracy of the technology has changed the system, refers to the re-implementation of a subset of tests have been done to ensure that the modification does not introduce new errors or before due to the change caused by discovery error is not discovered, that is, to ensure the change did not bring unintended side effects. Therefore, all phases of the software development will be carried out multiple regression tests.
1. Prerequisite for the implementation of regression testing
(1) When the software contained errors are found, if the error tracking and management system is not perfect, these errors may be missed modifications;
(2) the developer of misunderstanding or inadequately, it could lead to a revision made only fixes a bug in the external appearance, but did not fix the error itself, resulting in modification failed;
(3) modification is also possible to produce side effects leading to parts of the software have not been modified to produce new problems, so that would work normal function generates an error.
2. Regression testing of two strategies
Regression testing is testing throughout all stages of the testing activities, the purpose of defect inspection has been found that there is no correct modification and revision process has caused no new defects. The following strategies can be used for regression testing.
1) complete repeat test
Repeat testing is complete all the test cases all over again completely executed, to confirm the modified test methods surrounding the issue of the validity and whether the changes affected. The disadvantage is due to take full implementation use cases, thus increasing project costs, will also affect the project schedule, it is difficult to fully implement.
2) selective repeat test
Refers to selective repeat tests can be performed to select a part, in order to confirm the correctness of the issue modified perimeter and is subject to modifications of the test method of impact. Here are several methods of selective repeat tests:
(1) covers the modifications method.
(2) Method outside influence.
(3) index reached a method.
(4) cross-section based on an operation test.
(5) Risk-based selection test.
3. Regression testing process
Regression testing process typically has the following steps.
Step 1: policy formulation stage in testing, regression testing to develop strategies.
Step 2: Determine the regression test version.
Step 3: Regression testing release, regression testing regression testing in accordance with policy.
Step 4: By regression testing, a single defect tracking off.
Step 5: Regression testing is not passed, a single defect return; developers to re-edit, do regression testing again.
4. Comparison of regression testing and general test
Regression testing and general test compared to a number of different points, the following were allocated from the availability of a test plan, test scope, time, development of information, complete terms of time and efficiency, etc. are introduced.
Availability (1) test plan: general test systems according to specifications and test plans, test cases are new; and regression testing may be a change of specification, the modified program and the need to update the test plan.
(2) Test Scope: The general objective of the test is to detect the correctness of the entire program; the goal is to detect regression testing the correctness of the relevant portion of the modified system and its integration with the original function.
(3) allocation of time: the time required to test the general budget before the software is usually developed; and the time required for regression testing (especially corrective regression test) are often not included in the schedule of the product.
(4) the development of information: general knowledge test development and information on available at any time; and regression testing may be carried out at different places and times, need to retain information to ensure proper development of regression testing.
(5) completion time: Since only a portion of the regression tests of the test program, thereby completing the time required for testing is generally less time than normally required tests.
(6) Execution Frequency: regression testing often many times during the life cycle of a system, once the system has been modified on the need for regression testing.
