Bug reports, software testing is one of the most important was the job output. Even the software testing industry, you can compare this narrow description to define him with: 'test is to find defects'. Testers reported flaws, problems can be a good reaction product, fix these problems, it can effectively reduce product risk.
In fact, not only defect reports can help research team found the problem, he can also play an important role in the process of feedback.
Bug report is one of two core elements of our test report, he formed the main content of our test reports with the implementation of the test. The defect report, we should report something, is not just the number of defects it? We are concerned that today, how to use 'quantitative analysis' of the form, to make our bug report.
We project a real defect report to illustrate this subject, the project situation is this:
- The project is a COTS product customization secondary development project
- Project cycle plan for four months, the actual completion time of six months
- The overall project is a person who is less than 10 small projects
- The use of continuous integration, high-speed iterative development mode
1. We want to see the first report called 'defect arrival rate report', see below:
Refers to a defect arrival rate per unit time, the number of reported defects. The figure was the number of defects per month according to statistics quoted and classified according to severity.
Resolution:
① defect arrival rate decreases significantly the previous four months
② amount of defects May recovery is mainly reflected in the number of low-severity flaws
③ number of defects of the severity level to normal
④ June defects rise significantly
Combined with the actual project we analyze this report: bug after two months the number of rises mainly because during this time our tests were introduced centralized regression testing and acceptance testing (UAT we will test, customers reported the bug is introduced into the interior of our defect management system). Customer quoted deficiencies, severe high level, this may be because the customer to understand the severity of the defect, it is not consistent with our understanding of the development team caused. We may need to communicate better and communicate with our customers on this aspect.
2. defect removal rate analysis:
Defect removal rate refers to the proportion we introduced in various stages of development and clear addressed by the present stage of the defect.
Based on the theory of software testing which we emphasize, software testing should be involved in the project as soon as possible, the general requirements to be involved in the requirements analysis phase, and we use the method of static testing to test the various stages of output. At this stage we should go to pursue the problem as clearly as possible defects arising at this stage. The defect removal rate performance is our clear ability to find defects and problems that stage when introduced in stages, in turn, he can reflect how many questions are left over from one stage to the next stage.
For example, in the requirements phase, we produce: requirements document inside on the introduction of 10 defects, we work through the needs assessment phase of testing in when found defective and clearly two of them. The defect removal rate of the phase is 2/10 = 20%. And the defect rate is left (10-2) / 10 = 80%, 80% of the defects are left into the next stage.
It is more intuitive we can make the following tables and charts:
Analysis: For us to analyze more reports, we may be able to draw the following conclusions:
① requirements phase defect removal rate is low, indicating lack of needs assessment work
② acceptance phase quote needs of a significant number, indicating poor communication with the user's requirements team
③ Test unit overall low number of defects found
④ test persons outside the lower number of bug reports
In addition, through the investigation we found that the actual project, among others, the actual team to test group has a tendency to not love the bug report. In fact, the quality of a project is to be with the team all personnel are closely related, not just the test team tasks. Should we encourage more people to take the initiative to other aspects of the project submitted by the defect, it is worth thinking about a problem.
3. defect distribution rate analysis
It refers to a defect distribution of the number of defects for different functional modules, as quoted. The figure is a small e-commerce platform defect distribution rate case.
Resolution:
① large amount of defective merchandise overall browsing module quoted
② payment module reported more serious defects
This chart lacking intuitively degrees, but not the most accurate representation of the quality characteristics of each module. Here we can do a weighted: for example, different severity of defects, give him different scores, secondary computing, again recount.
Example: 5 weight serious shortcomings, critical defects weights 3, other weights 1 yields:
You can see the proportion of the payment module up, he displays both modules and project quality indicators most noteworthy of two parts, which can guide us into test resources.
4. defect repair rate analysis
Refers to the defect repair rate in a certain unit time, it reported, and the remaining number of defects to be repaired comparison. This is a more intuitive reporting.
Resolution:
① the number of defects reported after a steady decline in the latter part of the project ushered in a rebound
② development team bug fixes ability landslide occurred in April, according to the survey because of the development of core staff has been deployed
bug number left is not optimistic when ③ project closure, mainly due to last two months caused by the surge in the number of reported bug
By reporting display and analysis, we can draw some useful conclusions: for example, we should consider whether the action forward regression testing; another example we can see that the stability of the core team members is an important element of the success of a project.
There are many other forms of statistical reports we can consider, such as:
5. bug fixes round statistics
Count the number of bug fixes round is activated by statistical flaws, defects have to go through to observe how many rounds of repair can be closed such data.
Resolution:
Most defects ① project has undergone several restoration
② reflects a degree of development and testing team communication problems
③ Part of the reason is that the test team test description is not sufficient
6. defect efficient statistics
Defects efficiency refers to the reported defects among us, the percentage of valid defects.
Analysis: we can clearly see that the test environment has a great influence on the degree of effectiveness to the test
7. phase defect distribution statistics
Phase defect distribution refers to the distribution of the level of a serious bug and how within a certain period of time, we are reporting.
Analysis: May a significant increase in the number of times a bug severity June bug number of the most serious level rise
8. type of defect distribution statistics
Analysis: Most types of defects is focused on functionality, it may reveal our lack of attention to other indicators of quality
9. testing activities defect rate statistics
Resolution:
① test the efficiency of the system and not our imagination so high (more than 200 defects, only less than 40% from the system testing phase)
② number of defects of worrying external test (in fact referring to the results of UAT test)
③ regression testing and exploratory testing system testing found the problem is not found, indicating that they are very effective and necessary testing activities
Above we discussed the measure on the test defect data, the type of thinking that we can report to consider the statistics, but also combined with the actual project status for each report for a certain amount of resolution. Practical work, we can also make many other types of reports, as long as we believe the statistics of this aspect may bring useful information and thinking to us, we can go to him to report.
There are a few issues to talk about:
First of all: Our this example, the number of sampling sample is too small, the problem is caused by the randomness of the data will be too large, there may be cases distortion occurs. If the project size is large enough, sampling more samples, we measure the defect data analysis will be able to more accurately reflect the state of the project has been testing our product quality characteristics.
Second: If we always do before the defect data metrics at the end of the project phase, he projects the role of feedback is very limited - after all projects have been winding down. In fact, when we talk about test reports, test reports should not just only know that the test is complete report, as well as test reports. If we can introduce defects in the testing process measurement data report, then we can better test and development process of feedback, so as to achieve process improvement results.
Finally: If you want the test report, able to collect more useful data defects, we are required to conduct a more detailed definition of the information contained defects. For example, it requires developers to address deficiencies at the same time, a clear fill the root causes of defects arising from the stage, so that we can count us to make a defect leakage rate in the second node / removal rate report.