Author| Liu Yanqing, Test Manager, Security Evaluation Department, Shanghai Control Security
Forum| Jianyuan Forum Guantong
Introduction: The first part introduces the rail transit signaling system from the aspects of railway system classification and composition, urban rail transit system classification and composition, urban rail transit system functions, and urban rail transit system development. The second part describes the signal basis The principles and meanings of signal machines, switch machines, track circuits, etc. are explained. The third part analyzes the necessity of safety design, control design, demand analysis and realization of the rail transit system. The fourth part focuses on the interlocking The principle of the system is explained. This article will start from the rail transit software life cycle, focusing on different stages and different types of software to explain the key points of testing at each stage.
01
What is software testing
1.1 Definition of Test
"Testing is the process of proving that bugs don't exist"
"The purpose of testing is to show that a program correctly implements the intended function"
"Testing is the process of building confidence that a program does what it is supposed to do"
"Tests can only prove the presence of defects, not the absence of tests"
Software testing defined by IEEE: "The process of using manual or automatic means to run or test a system, the purpose of which is whether it meets the specified requirements or to clarify the difference between the expected results and the actual results."
“Testing is the process of executing a program with the intent of finding errors.”--- Glen Myers
1.2 Purpose of software testing
Verify that all components of the software are properly integrated; verify that all requirements have been correctly implemented; find and ensure that defects are resolved before the software is deployed.
1.3 Significance of software testing
Testing accounts for 30-50% of the total development cost; ensures the correctness, completeness and consistency of the program; significantly reduces the expense of completing and maintaining the software; greatly reduces the liability or risk associated with deploying poor quality software.
02
software life cycle
The commonly used software life cycle models are: traditional waterfall model, waterfall V model, incremental model, and iterative model.
Figure 1 Traditional waterfall model
Figure 2 Waterfall V model
Incremental model: The requirements are basically clear, the first requirements analysis includes all requirements, and the requirements changes are mentioned in the subsequent stages.
Iterative model: the requirements are not clear, and the requirements analysis of each stage only includes the requirements of this stage.
03
Rail Transit Software Testing Process and Strategy
3.1 Basic process of rail transit software testing
Test plan→test case→prepare data→execute test→output and analyze test results.
Figure 3 Basic flow of rail transit software testing
(The purple background is the operation performed, and the blue background is the output document or process data)
3.2 Testing strategy for rail transit software testing
It is necessary to adopt appropriate testing strategies in different testing stages according to different inputs. For example, software module design needs to be tested during the unit testing phase. The software structure design needs integration testing to test and verify the documentation at this stage. For software requirements, software validation testing activities are required for validation testing. System structure design, as well as multiple subsystem integration, integration verification in the system integration stage. System requirements require system-level confirmation tests to verify and confirm them.
Fig. 4 Test strategies for each test phase of rail transit software
3.3 Test files output by each test stage
Figure 5 Test files output by each test stage
3.4 Test defect management and tracking
Defects found in testing may be code defects or documentation defects. For document defects, testers still submit CR according to the expected differences, and analysts analyze and explicitly modify the documents. If it is a code defect, you need to modify the code and re-release the code according to the process. For regression testing, if there is an impact analysis report, the range of use cases for this regression should not be less than the range required by the impact analysis.
During the test, it should be judged whether the use case is passed strictly according to the expected result of the use case. If it is found that the use case is wrong or the steps need to be refined or additional steps are required during the execution of the test, the use case can be modified, but in the test report, the version of the test case is the new version to ensure that the test execution is consistent with the actual use case version.
3.5 The difference between white box testing and black box testing
White box testing is also called logic-driven testing, which has the following characteristics: the internal working logic of the program is known; design and construction of test cases and test data based on the program logic structure; Whether the path works correctly according to the predetermined requirements; check whether the internal action or operation of the program meets the design specifications.
The characteristics of black-box testing: do not consider the internal working logic of the program; start from the user's point of view; design test cases and test data based on the functions that the program should implement and the defined requirements specifications; verify whether the functions are implemented and consistent with the requirements.
Black-box test design methods: Equivalence Partitioning (EP), Boundary Value Analysis (BVA), Error Guessing (EG), Cause and Effect Diagram, etc.
04
Common stages of software testing
4.1 Unit testing
Unit testing includes dynamic testing, code review and code walk-through.
Dynamic testing is to dynamically run the smallest component unit (usually a function) of the program through instrumentation and driving, and verify whether the function is consistent with the function defined in the module design document. The dynamic test is carried out according to the software detailed design specification.
Code review is to use tools or humans to check whether the code meets the defined coding rules.
Code reading is to manually read the code to check the logic errors in the code or to improve the structure optimization. Code walk-outs can take the form of group reviews or purely individual code readings.
Figure 6 unit test dynamic test
Module interface: verify whether the data can be input and output correctly;
Boundary conditions: errors on the boundaries are very common;
Coverage conditions: or path analysis, that is, to test the basic path and cycle of execution;
Error handling: A good design should estimate the errors that may occur after it is put into use, and provide corresponding processing measures to ensure logical correctness.
Test case design follows: Design sufficient tests based on the principle of structured testing (white box) to achieve the required coverage (statement coverage; decision coverage; condition coverage; decision/condition coverage; condition combination coverage; path coverage)
Recommendation: First design black-box test cases, then analyze the code, analyze from the perspective of white-box, and increase or decrease test cases according to coverage requirements.
Note: There is a bad tendency in practice. Unit testers do not understand the functions implemented by the module under test purely to achieve the coverage goal. They only build test cases from the perspective of white boxes, and often cannot find functional errors in the module.
4.2 Software integration testing
On the basis of the unit test, according to the selected integration method, assemble modules or tasks or threads, and verify whether the message transmission between the integrated objects is correct (if necessary, it needs to be verified by instrumentation).
This test is carried out according to the software structure design specification. The objects of the test are interfaces between modules, function call interfaces, message interfaces, shared queues, files, etc. The object of integration can be determined after analyzing the actual situation, it can be integration between functions, or integration between tasks or threads.
Ways of software integration testing: Big-bang integration; Top-down integration; Bottom-up integration.
Figure 7 Big Bang Integration
Big-bang integration: also called non-incremental integration. Its advantages: no need for stakes and drivers; disadvantages: it is not easy to locate the problem.
Figure 8 Top-down integration
Depth first: M1-M2-M5-M8; M1-M2-M5-M8-M6; M1-M2-M5-M8-M6-M3-S7; M1-M2-M5-M8-M6-M3-S7-S4 .
Width priority: M1-M2-M3-S4; M1-M2-M3-S4-M5-M6-S7; M1-M2-M3-S4-M5-M6-S7-M8.
Advantages: You can see the framework of the system from the beginning;
Disadvantages: stubs need to be inserted, stubs cannot reflect the real situation, so the test may not be sufficient, and it is inconvenient to view the output in the stubs.
Figure 9 Bottom-up integration
Bottom-up integration: It belongs to incremental integration. Its advantages: easy to view the output. This method is more advantageous when the key module is at the bottom. Disadvantage: You can only see the framework of the system at the end.
Suggestions for performing integration testing: core tasks should implement module-level integration; non-core tasks can only implement integration between tasks or processes; integration testing should be based on interface testing, supplemented by functional testing; the integration strategy should use the big bang method or The incremental integration method depends on the situation; the integrated module of the integration test can be in the unit of thread, or in the unit of function module, or in the unit of interface or in the unit of function.
Tool support is required: module integration can use corresponding white box testing tools; task/process integration should use protocol analysis software or hardware test and analysis equipment such as logic analyzers and oscilloscopes.
4.3 Software Validation Tests
Combining the integrated software as an entire element with other software elements such as computer hardware, peripherals, some supporting software, data and personnel, and performing a series of assembly and validation tests on the software in the actual operating environment .
This test is carried out according to the software requirements specification.
Figure 10 Software Confirmation Test
Safety Test: Verify whether the software will produce dangerous output, for example, if the board is placed in the wrong position in the interlock software, the software should shut down.
Recovery Test (Recovery Test): artificially make software errors, check whether the software can be automatically restored, and whether the initialization and parameters are correct during automatic recovery.
Backup Test: Verify that the software is capable of automatically backing up data.
GUI test: interface display, whether it is friendly
Robust Test (Robust Test), also known as fault tolerance test. Ignore the fault and continue running. For example, the failure of the interlocking standby machine does not affect the software output; when the interlocking main machine fails, the standby machine is upgraded to the main machine, and the entire software can still output correctly.
Installation Test: Verify whether the installation can be completed according to the installation manual, whether there are reasonable prompts during the installation process, and whether there are restrictions on the installation environment. Verify that it can be uninstalled.
4.4 System Integration Test
Verify the correctness of the independent functions of each subsystem and its interface and data transmission, and meet the characteristics specified by the entire system design. The test is carried out according to the system structure design.
Figure 11 System integration test
4.5 System Confirmation Test
Functional Test (Functional Test): Verify whether the function implementation meets the software requirements.
Performance Test: CPU, network, memory, system response time, instruction execution time, etc. under specific load.
Stress Test (Stress Test): also called strength test. Resource overload (such as users), verify the maximum pressure the system can withstand, find out where and how the system fails.
Volume Test: Excess data capacity, verifying the maximum capacity of the system, generally related to the database.
Security Test (Security Test): prevent illegal intrusion.
Safety Test: Verify whether the system will generate dangerous output, for example, if the board is placed in the wrong position in the interlock system, the system should shut down.
Recovery Test (Recovery Test): artificially make software errors, check whether the system can be automatically recovered, and whether the initialization and system parameters are correct during automatic recovery.
Backup Test: Verify that the system can automatically back up data.
GUI test: interface display, whether it is friendly.
Robust Test (Robust Test), also known as fault tolerance test. Ignore the fault and continue running. For example, the failure of the standby machine of the interlocking system does not affect the system output; when the main machine of the interlocking system fails, the standby machine is upgraded to the main machine, and the entire system can still output correctly.
4.6 Acceptance testing
Finally, before the system under test is officially put into the actual operating environment, the environment and operation process are completely real, without using any simulation equipment or simulation programs. This test is performed on site.
05
Summarize
Most of the rail transit software systems adopt the life cycle model of the waterfall V model. Because it involves requirement analysis, outline design, detailed design, etc., there will be unit testing, integration testing, and software system testing to verify or confirm the input of each stage.
The focus of testing at each stage is different, and various testing methods need to be adapted. For example, EG's method is to implant errors in the program to verify whether the existing test cases can be found and whether the test cases are sufficient. The test case must contain the expected output. Programmers should avoid testing their own code, and program development organizations should avoid testing their own code. Thoroughly check the results of each test. It should be illegal, unexpected conditions and normal, Anticipated conditions Designing test cases, checking whether the program executes the specified behavior is only part of the test, and checking whether the program executes unspecified behavior, these are the attention points in the process of designing use cases and executing tests, respectively from independence , Sufficiency, etc.
references:
[1] Basic concepts and methods of software testing
https://wenku.so.com/d/fd880b6a210bf899575fa128313b820b.
[2] Analysis of the application of software testing technology in C program development https://wenku.so.com/d/f78781bb1f8160cbc1cc8cd26c361db4.
[3] System Test Type
https://blog.csdn.net/weixin_44819949/article/details/125059604.