How to determine the endpoint of a software testing project

How to determine the endpoint of a software testing project

1. In principle, we prefer a standardized development system to normalize this proposition, without having to worry about it. But being able to end testing at milestones or planned deadlines is an expectation for most software projects, not a given reality. Ideally, we can strictly implement the plan, and then submit the deliverable at the deadline or milestone point required by the plan to confirm whether the milestone is met and whether the next stage of work can be carried out - but as the premise, this Just the ideal situation.
2. Now let's be realistic. Why do we have such a problem (how does a software determine the test end point)? Often it's because we don't know when we can end a software test. No matter how the textbook states that a software cannot be tested as long as it is still in its life cycle, the reality requires us to end the testing of a certain stage of the software at a certain point in time. The question, then, has actually turned into a way to determine the end point of that phase of testing. The method might be a specification, a set of processes, some deliverables, some reviews, some statistical convergence curve or just some validation. Personally, I believe that no matter what form this method takes, the basic requirement is to reach an agreement to confirm that the agreement is effective-then the test at this stage is over. As for when this point is, I think it is completed. All requested time for these confirmations only.

Before software dies, if there is no end point for testing, then software testing is endless and can never end. The end point of software testing should be formulated according to the specific situation of your company, and cannot be generalized! Personally, I think the end point of the test is determined by the following conditions:

1 Based on the principle of "testing phase"

Every software is generally tested In the stages of unit testing, integration testing, and system testing, we can formulate detailed test end points for unit testing, integration testing, and system testing , respectively . After each test phase meets the end criteria, the next phase of testing is performed. For example, for unit testing, we require that the test end point must meet the requirements of "100% of the core code has passed Code Review", "100% functional coverage", " code line coverage is not less than 80%", "does not exist" A and B defects", "at least 60% of all found defects are included in the defect tracking system and the defect repair rate at all levels reaches the standard" and so on. End points for integration tests and system tests both have associated end criteria, of course.

2 Based on the principle of "test case"

test Designers design test cases and invite project team members to participate in the review. Once the test cases are approved, they can be used as a reference standard for the end of the test when testing later. For example, during the testing process, if you find that the pass rate of the test case is too low, you can refuse to continue the test and continue after the developer fixes it. When the pass rate of functional test cases reaches 100%, and the non-functional test cases reach more than 95%, it is allowed to end the test normally. However, when using this principle as the end point of the test, it is very important to grasp the quality of the test cases.

3 Based on the principle of " defect convergence trend "

In the life cycle of software testing, with the passage of testing time, the defect graph found by the test first gradually increases, and then the test reaches a certain stage, and the defect becomes a downward trend until the defects found are almost zero or it is difficult to find defects. until. We can determine whether the test can be ended through the trend of the defect trend chart, which is also a judgment criterion.

4 Based on the principle of " defect repair rate "

We divide software defects into several severity levels in the test life cycle, which are: critical errors, major errors, minor errors, general errors, minor errors and test recommendations. Then when we determine the end point of the test, the defect repair rate of serious errors and major errors must reach 100%, and functional errors are not allowed; the defect repair rate of minor errors and general errors must reach more than 85%, and a small amount is allowed. Functional defect, later version Solved ; the defect repair rate for minor errors is preferably above 60% to 70%. For the problems suggested by the test, it is not necessary to modify them for the time being.

5 Based on the principle of " acceptance testing "

Many companies are doing project software. If it is necessary to determine the end point of the test, it is best to test to a certain stage. After reaching or approaching the standard specified by the testing department, submit it to the user for acceptance testing. If it passes the user's test and acceptance, the test of the testing department can be terminated immediately; if some defects are found during the customer's acceptance test, the defects can be modified in a targeted manner, and then submitted to the customer after the verification is passed, and the corresponding test can also end.

6 Based on the principle of " coverage "

Regarding the principle of test "coverage rate", I personally feel that as long as the "coverage rate" of test cases covers all the software requirements proposed by customers, including industry implicit requirements, functional requirements and performance requirements, etc., as long as the test case execution coverage rate When it reaches 100%, the test is basically over. For example, "the minimum statement coverage rate in unit testing cannot be less than 80%", "the test case execution coverage rate should reach 100%" and "the test requirement coverage rate should reach 100%" can be used as end determination points. If you are not at ease, you have to check the execution effect of the test case and check whether the use case is omitted. You can perform "sampling tests" and "random tests" on commonly used functions. For coverage in unit testing, integration testing and system testing, each stage cannot be ignored.

7 Principles based on " Project Planning "

In most cases, each project needs to write a schedule for development and testing from the beginning, and correspondingly each milestone in the test plan will also set a limit on the test progress and the test end point, generally speaking with the project Group members (development, management, testing, marketing, sales) reach a consensus, and the team collectively agrees to develop a standard end point. If any part of the project is delayed, the testing time is shortened accordingly. In most cases, all the specified test content and regression tests have been run and can be used as an end point. Many non-standard software companies use the project plan as a test end point, but if it is used as an end point, the test risk is high, and the software quality is difficult to guarantee.

8 Based on the principle of " defect measurement "

This principle may not be used by many people, and the understanding is relatively small. We can use common defect analysis techniques and defect analysis tools for the defects that have been found, and use charts to count them for easy access, and measure the defects by time period. I remember that zhuzx raised the issue of defect analysis technology on this forum before, and I will not repeat it. We can also use "Test-time defect density" and "Run-time defect density" as an end point. Of course, the most appropriate criterion for the end of the test should be "the number of defects is controlled within an acceptable range". For example, how many errors of any severity level are allowed in 10,000 lines of code, which is easier to quantify and implement, and becomes the mainstream of test defect measurement.

9 Based on the principle of " cost of quality "

一个软件往往要从“质量/成本/进度”三方面取得平衡后就停止。至于这三方面哪一项占主要地位,就要看是什么软件了。比如说是：人命关天的航天航空软件,那还是质量重要些,就算多花点钱、推迟一下进度，也要测试能保证较高质量以后才能终止测试，发布版本。如果是一般的常用软件，由于利益和市场的原因，哪怕有bug，也必须得先推出产品，没办法呀。一般来说，最主要的参考依据是：“把找到缺陷耗费的代价和这个缺陷可能导致的损失做一个均衡”。具体操作的时候，可以根据公司实际情况来定义什么样的情况下算是“测试花费的代价最划算、最合理”，同时保证公司利益最大化。如果找到bug的成本比，用户发现bug的成本还高，也可以终止测试。

10基于“测试行业经验”的原则

很多情况下，测试行业的一些经验，也可以为我们的测试提供借鉴。比如说测试人员对行业业务的熟悉程度，测试人员的工作能力，测试的工作效率等等都会影响到整个测试计划的执行。如果一个测试团队中，每个人都没有项目行业经验数据积累，拿到一个新的项目，自然是一头雾水，不知道从何处开始，测试质量自然不会很高。因此通过测试者的经验，对确认测试执行和结束点也会起到关键性的作用。

◆    ◆    ◆   ◆    ◆  

测试人都懂，不可能找遍所有的bug，可是你的老板不懂，做测试计划和方案时，你该怎么汇报并让他信服呢？