Software Critic - Software Testing

Software Testing and Software Quality

Classic definition of software testing (see more tutorials)

为了发现软件中存在的错误而进行的操作。目的：为了发现错误

software testing purpose

• find bugs in software
  软件测试只能证明软件中存在错误，不能证明软件没有错误。

object of software testing

• software code
• document
• data

database test object

• database connection
• Database security testing
• Database Interface Test
• Testing of defined stored procedures and triggers

software quality

Software 特性的总和
also satisfies specified or potential user needs, 能力
and is also a manifestation of the "capability" of software features.

composition

• internal quality
• external quality
• Quality of use - from the perspective of users

Quality Assurance (QA) VS Software Testing

【quality assurance】

• Throughout the whole process of software completion, focusing on the process of software development 过程, 步骤and 产物.
  【software test】
• Activities that do not care about the process, only those that care about the process 产物.
• Important work: problem analysis, tracking and regression testing
• It is an important part of software quality assurance

Software Product Quality Evaluation Criteria

• By measuring internal properties
• By measuring external properties -- 可靠性etc.
• By measuring the properties of the quality of use - 生产率, 安全性, 满意度,有效性
  

Principles of Software Testing (6)

• All tests should be traced back to user needs (conform to requirements)
• should be tested early and continually
• The testing work should be avoided by the person or team who originally developed the software (except for unit testing or module testing)
• Exhaustive testing is not possible, the test needs to terminate
• Pay due attention to the cluster phenomenon in the test
• Strictly follow the test plan to avoid the randomness of the test

Software Testing Classification

Development stage division (5)

• unit test
• Integration Testing
• System test
• confirmation test
• Acceptance Test

Unit tests (module tests)

• Need to design test cases from the internal structure of the program
• Multiple modules can be unit tested in parallel and independently

Test content

• Stub module - simulate the module under test 所调用的模块, used to test the upper module
• Driver module - simulates the module under test 上一级模块, used to test the lower module

Integration testing (assembly testing, joint testing)

Documents involved

• Outline Design Document
• Detailed Design Document

Considerations when assembling

• When connecting each module, will the data passing through the module interface be lost?
• Whether the functionality of one module adversely affects the functionality of another module
• Whether the combination of various sub-functions can achieve the expected parent function
• Is there a problem with the global data structure
• Whether the accumulated errors of each module will be enlarged to an unacceptable level

Module assembly method

• One-time assembly (big bang) - test after all one-time assembly (suitable for small software)
  Advantages: Simple operation and low cost
  Disadvantages: Difficult to locate problems found during testing
• Value-added assembly
  [1] Top-down value-added method
  Advantages: easy to find branch errors
  Disadvantages: Error-prone bottom-level modules will not be found until the end of the test
  [2] Bottom-up value-added method
  [3] Mixed value-added method

completion sign

• 测试计划All integration tests specified in
• Fixed bugs found
• The test results have passed the review of the special group or reached the industry standard

System test

• Find out where the software does not conform to or contradict the system definition
• Integrate the entire system elements (including hardware, peripherals, network, personnel and system software, supporting platforms, etc.)

confirmation test

• Verify that the functionality and performance of the software is consistent with the requirements
• Carry out validity testing - usually black box testing
• Software Configuration Review

一般由独立的第三方测试机构进行，应当严格遵守操作手册和用户手册规定的使用步骤，以便检查文档资料的完整性和正确性

Acceptance Test

• 用户为主
• Generally use actual data in production for testing
• decide whether to accept or reject the system

Division of Test Technology

• white box testing
• black box testing
• gray box testing

Implement organizational testing

• Developer’s test (α test)
  ——Tester: developer-based
  ——Test environment: under the development environment
• User-side testing (β testing)
  - tester: user
  - test environment: in the user's application environment
• Third-party testing (independent testing)
  - testers: third-party independent testing agencies

Software Testing Process Model

V model

• Low-level testing: source code correctness testing (unit testing, integration testing)
• High-level testing: testing of user requirements (system testing, acceptance testing)

V-model limitations

• Testing is performed after software development is complete, which conflicts with "testing early and continuously"
• Early problems (requirements or design problems) are not discovered until the end, which greatly increases the cost of problem solving

W model

W model features

• It is conducive to early detection of problems and reflects the principle of "early and continuous software testing"
• Emphasize that testing accompanies the entire software development cycle
• Add tests that should be performed simultaneously in the software and development phases in the V-model

W model limitations

Software development and testing maintain a linear front-to-back relationship, and strict instructions are required to indicate that the previous stage is completely completed before the next stage can be formally started. This fails to support iteration, spontaneity, and adjustment to change.

H model

• A software testing model is a独立的流程
• When a test point in time is ready, software testing moves from the test preparation phase to the test execution phase

X model

• positioned探索性测试
  

pretest model

• Business requirements are best defined before design and development
• Advocates test design based on business needs, and thinks 设计阶段it is the best time for test execution and test design
• Combining test execution with development and “编码-测试-编码-测试-编码-测试”reflecting it in the development phase, each delivered development result must be tested in a certain way
• 验收测试独立于技术测试, to ensure that the design and program coding can meet the needs of end users
  Comparison of Several Test Models
  Detailed comparison of common test models

test flow

input stream

• Software configuration (requirements specification, outline design specification, detailed design specification, etc.)
• test configuration
• test tools

Software Failure Classification and Management——A Survey of Major Issues

Software Failure Terminology

• 1. Software 人为errors——
• 2. Software defects——compared with the product manual偏差
• 3. Software failure - 内部status
• 4. Software failure - 外部behavioral consequences

Cause of software failure

Major: There is a problem with the product manual
Secondary: There is a problem with the software design manual

Status of Software Errors(6)

• new information new
• open open
• fix
• declined
• Deferred
• close closed

Automated testing related tools

use quality model

System/Software Product Model

Contents of the test record

• 测试计划or with test cases测试规格说明
• 人员identities involved in the test
• Everything related to the test case 结果, including any所有失败

The main causes of software testing risk

• 测试计划insufficient
• 测试过程deviation
• 测试方法mistaken