1 The concept of system testing
Definition of System Testing : The integrated software system, as an element of the entire computer-based system, is combined with other system elements such as computer hardware, peripherals, certain supporting software, data and personnel, and is tested in the Under the actual operation (use) environment, a series of assembly tests and confirmation tests are carried out on the computer system.
The object of system testing: the system where software and hardware are integrated, the integrated product should not be an independent software and hardware environment. It includes not only the software to be tested, but also the hardware and peripherals that the software depends on, and even some data, some systems that support the software, etc.
Purpose: By comparing with the requirements definition of the system, it is found that the software does not conform to or contradicts with the system definition, so as to verify that the functions and performance of the software meet the requirements specified in its specification.
Common system classifications:
Pure software: QQ...................
Software and hardware: mobile phone, PSP, air conditioner, elevator
Software, hardware and maintainers: large systems
2 Environment for system testing
Real environment: directly link the entire system with its cross-linked physical equipment for testing
Advantages: can find some problems that can only occur in real environments
Cons: Building such an environment is expensive, and it's also expensive to test run
Simulation environment: It can realistically simulate the input and output of the real physical environment required for the operation of the software under test, and can organize the input of the software under test to drive the operation of the software under test, and at the same time receive the output results of the software under test.
Advantages: The software dependencies of the simulation environment and the real environment are the same, and it can ensure the repeatability, integrity and scalability of the test
Example 1 : Requirements for building a certain system environment
Hardware environment:
CPU: main frequency above 2GHZ, 4 cores and above
Memory: 4GB and above
Hard disk: more than 10G free space
Software Environment:
Platform requirements: Windows XP/2003 Standard Edition + Apache2.2.X + Mysql5.0 and above
Directory permissions: directory /Attachment, /Cache and file permissions that need to be written
Recommended platform: XAMPP is recommended as the installation medium
Factors to be considered when selecting test tools:
1. The degree of matching between the test tool and the software system under test
2. The main functions and auxiliary mechanisms provided by the test tool
3. Service and technical support for testing tools
4. Price of testing tool
Test Data:
Features:
1. Data can exist in the form of messages, things, records, files, etc.
2. There are many sources of data
3. Real data is the best, but in many cases it is difficult or impossible to obtain
source:
1. Product data
2. Construct data manually
3. Generate data (generally constructed by tools and equipment)
4. Capture data (rarely used, related to capturing data sources, allowing manual modification)
5. Random data (rarely used in system testing, it is easy to obtain, not realistic enough, commonly used in performance testing)
product data
1. The most authentic
2. Cannot cover all the scenarios needed
3. The data is sensitive and it is difficult to guarantee the correctness
4. Changes over time
5. The amount of data may be too large (thus reducing the speed of test execution)
Constructing data manually
1. Time-consuming and boring
2. If there is a lack of understanding of the function of the system, the data is not true
3. Sometimes the only means of obtaining the unique data needed for a specific test case
generate data
1. Generally constructed from tools or equipment
2. The data depends on the sophistication of the tool and the tester's specifications on how to structure the data
capture data
1. Agnostic to capture data source
2. Allow manual modification
random data
1. Easy to get, not real enough
2. Very useful for strength or load testing
3 types of system testing
Common test types:
Functional testing (configuration testing, recovery testing, backup testing)
Performance testing (stress testing, stability testing, capacity testing)
GUI testing (usability testing)
compatibility test
Security Test (Network Test)
Installability test
document test
function test
Concept: According to the SRS and requirements list, verify whether the function realization of the product meets the requirements specification of the product
Target:
1. Are there any incorrect or missing functions (doing wrong or doing less)
2. Whether the function realization meets the hidden requirements of user needs and system design
3. Can it accept the input correctly and output the result correctly?
Functional test steps:
1. Label each specific functional requirement
2. Label each implicit functional requirement
3. Classify, analyze and label possible functional abnormalities
4. Divide functions into critical functions and non-critical functions
5. Test and analyze each function, analyze whether it is measurable, how to test it, possible input and output
6. Scripting and automation
Performance Testing
Concept: Used to test the performance of software in an integrated system
Goal: Measure various indicators of the system to confirm whether there are various performance bottlenecks in the system
Features: Mixed white box testing and black box testing approach
Two aspects of performance testing consideration:
1. Verify that the performance achieved by the system is completely consistent with the performance requirements
2. What is the specific performance of the detection system?
Common Approaches: Probes, Test Drive
Common tools: Loadrunner, WebLoad, SilkPerformer
GUI testing
Concept: Testing of Software System Interfaces
Goals: 1. Test the consistency between interface implementation and interface design 2. Confirm the correctness of interface processing
Focus: On the interface layer and the functional interface layer (the GUI system is divided into three levels: the interface layer, the interface layer between the interface and the function, and the function layer)
Common tools: QTP, QARun
compatibility test
Concept: Consider how the software under test will behave under other software (such as operating systems) or hardware devices.
Objectives: 1. Combination with auxiliary software (operating system, other applications, test software, monitoring software, browser, etc.); 2. Combination with hardware design (because compatibility testing requires more hardware, generally no Special requirements will only be tested against software requirements).
security test
Concept: To verify whether the protection mechanism integrated in the system can actually protect the system from illegal intrusion, and is used to protect the integrity and confidentiality of the data itself. In a broad sense, it also includes physical security and business security.
Scope: mainly consider several aspects: system login, user management, firewall, system data, WEB security, database security, internal communication, system anti-virus testing, etc.
Protection testing is a common test in security testing, mainly used to test the information protection mechanism of the system
Installability test
Concept: mainly based on the list of test features of the software, software installation, configuration documents, and test cases in the design of the installation process, to find errors in the software installation process.
Goal: Find errors in software installation, errors in installation manuals.
Inspection work to be done before installation test:
1. Whether the installation documents are complete
2. Whether the program files for installing the software are complete
3. Whether the tested installation files are complete
4. Whether the file format of the software matches the file format required in the installation guide
Commonly used automated installation testing software: Total Uninstall: It can monitor all software installation processes and record any changes it makes to the system FileRiver: Can automatically monitor and accurately record subtle changes in files and sub-files in multiple folders .
document test
Concept: Mainly for software requirements specification, installation manual, configuration guide and other documents, the test content is mainly to write specifications, correctness of content, unambiguity, and completeness.
Goal: Verify that the user documentation is correct and that the procedures in the runbook work correctly.
4 Process of system testing
Division of testing phases
System Test Planning Phase: Complete System Test Plan (Planning and Procedures)
System test design stage: complete the system test plan
System test implementation stage: complete system test cases, system test procedures, system test pre-test (smoke test) items
System test execution phase: Execute system test pre-test, system test cases, carry out regression test, verify repaired problems, submit system pre-test report, system test report, defect report
Process: Software Requirements Analysis, Design (Summary), System Testing (Execution)
| stage |
entry criteria |
enter |
export guidelines |
output |
| plan |
SRACompletion Requirements Specification Baseline |
software development plan Software Test Plan, SRS |
System test plan review and pass |
System Test Plan |
| design |
System test plan review and pass |
System Test Plan, SRS |
System test plan review and pass |
System Test Solution |
| accomplish |
System test plan review and pass |
System Test Plan System test plan, SRS |
System Test Cases System Test Procedure System test pre-test item review and pass |
System Test Cases System Test Procedure System Test Pre-Test Items |
| implement |
System test cases, system test procedures, and system test pre-test items are reviewed and passed, and integration test execution is completed |
System Test Plan System Test Solution System Test Cases System Test Procedure System Test Pre-Test Items |
System test report review and pass |
System Pretest Report System Test Report bug report |
Roles and Responsibilities in System Testing
Development representative: resolve resource requirements and supervise system test results
QA: Quality assurance of the system testing process, participation in relevant reviews, and auditing of the process
Configuration management group: configure and manage system test documents, test codes and other related configurations
Software Development Group:
1. During the system test planning phase, provide the software development plan SDP and participate in the review of the system test plan
2. In the stage of system test design and implementation, provide SRS, requirement analysis, test suggestions, respond to system test requirements, and participate in the review of software system test plans
3. During the system test execution stage, track and solve the defect problem report of the software test project team, and participate in the review of the system test report
Software testing group:
1. In the system test planning stage, formulate the system test plan and organize the review
2. In the stage of system test design and implementation, formulate software test plan and organize review, implement test case, test code and test tool design according to software system test plan, and write test procedures
3. System test execution stage, execute system test, feedback and track defect report, complete system test report and organize review, output test case, summary and other experience documents
System Analysis Group:
1. Propose system testing requirements
2. Carry out test requirement tracking
3. Carry out software system testability analysis, determine the object, scope and method of system testing
Essentials of a System Test Plan
1. Clarify the test object of the system test
2. Complete the requirements tracking of system testing
3. Clarify system pass or fail criteria
4. Suspension criteria for system testing and necessary conditions for recovery
5. Clarify the assignment of system testing tasks
6. Work products to be delivered after system testing
Key points of system test design
1. Preparation of test environment and test data
2. Test tool development/scenario design
3. System test case design
4. Selection of testing strategy (eg: several rounds of testing)
The difference between system test plan and system test plan
System test plan: specify and constrain the organization, resources, principles, etc. of the whole process of system testing, formulate tasks and time schedules for each stage of the whole process of system testing, and propose evaluation, risk analysis and management requirements for each task .
System test plan: describe the characteristics of the system to be tested, the test method, the planning of the test environment, the design and selection of test tools, the design method of test cases, and the design plan of test codes. (detailed documentation of the design test method)
Note: The project needs to be carried out under the guidance of the system test plan. The system test plan proposes what to do, while the system test plan clearly states "how to do it".
System Test Execution Overview
1. According to a certain system test plan, according to the system test cases, complete the various operational tasks of the test
2. According to the system test plan, setting up the system test environment is an important step in the execution of the system test. The time of the test environment will seriously affect the authenticity and correctness of the test results
3. The system test execution phase should be completed: environment preparation, test operation, test record, test report
4. Timing of execution: execute the system test after the integration test execution is completed
System Test Pre-Test
Purpose: To verify the basic functions of the software system or predict the main system functions to ensure that the subsequent system test execution can go smoothly.
Schedule: System pre-test should be carried out after the development project team submits the application for software version transfer to system test, mainly the "Software System and Test Report" that needs to be input to complete the transfer to system test review.
Personnel arrangement: The main body that performs the activities of verifying the basic functions of the software can be the software development project team or the software testing project team or a joint organization.
Transfer system test review
1. The main body of the review responsibility is the software project testing team, which needs to complete the software transfer system test review form
2. After the software version transfer system test passes, the system test process can be started
3. After the system test process is started, the software version, test code, document, environment, etc. related to the system pre-test should be baselined in the configuration management
System test report writing and review
1. According to the test pass of the system test plan, after the system test is completed, write a system test report
2. The system test report needs to pass the review, and the responsible person is the software test project team, which is participated by the software development project team, configuration management team and QA
3. If the review fails, the system test report will be returned. Submit an application for review after reviewing non-conformities and resolution of issues, or restart the system testing process
4. After passing the review, system test related documents, codes, tools, etc. must follow the software code and development documents to complete the configuration baseline, and the system test process is over
Purpose of Writing a System Test Daily
1. Testers summarize the daily test work, so that they can understand their test progress and test situation, and adjust the work plan for the next day
2. Testers give evaluation results to the tested objects every day to adjust the testing strategy for follow-up work
3. The testers report the difficulties in the test to the test manager to ensure the smooth progress of the test
4. The test manager understands the test progress of each staff member through the test daily report, grasps the overall progress of the test, and adjusts the plan in a timely manner when discovering risks in the progress
5. The test manager understands the defect development trend of each module through the test daily report, and judges whether the test can be withdrawn
6. The development manager can understand the quality of the current software through the software test daily, and can adjust the human resources for defect modification,
7. If the software has multiple test groups testing in parallel, the test daily can provide a means of testing and communicating with each other
System Test Steps
Requirements Analysis -> Plan, Scheme -> Use Case Design -> Environment Construction -> Execution -> Test Report
Classification of requirements: 1. Project requirements (users raise their own requirements) 2. Product requirements (product personnel conduct research and writing)
Baseline: The first statement: the software version is relatively stable. The second statement: the document is sealed up after multiple reviews and revisions, and anyone who needs to modify must submit an application.
Here are the videos and copywriting I have organized. If you need it, you can click the small card below
