The improvement of the software testing process is a continuous process, and the last process improvement effect was remarkable. For details, please refer to "Notes on the Improvement of Software Testing Process" . Therefore, when the new version of the QA report came out, the test members quickly held an internal discussion meeting. The discussion method was mainly brainstorming. All questions were raised around the reported data, and then the most urgent need to be solved was determined. question.
So we finally determined that we should focus on solving the problem of high residual bug rate. First, explain the definition of the residual bug rate. The denominator is the total number of valid bugs found in this version, and the numerator is the number of bugs found but not resolved in this version. The data of the two clients of iOS & Android is about 20% or more, and the data of the background is more than 40% (the previous version was as high as more than 60%). We analyzed the cause of the problem from the software development process: the standard for the client to enter the full test is to solve all high orders and 80% of medium and low orders. After the full test, there will be some new bugs. At this stage, as long as it is judged as not affecting None of the bugs posted will be resolved. When the full test is carried out, it is often just 80% of the standard. Coupled with some bugs left by the full test, the final residual bug rate is easily more than 20%. As for the background part, since there are no strict full testing standards, and some background functions can be temporarily blocked by setting switches, and some can be repaired after the client has reviewed them, the residual bug rate is higher.
Based on the above analysis, we have improved the test access requirements for the full testing phase and the release phase on the original basis. For example, the resolution rate of 80% of low and medium orders has been increased to 90%. We believe that more bugs should be fixed before entering the full test. , which will help improve the quality of the version. However, in the actual discussion meeting with the development, we encountered a lot of resistance, because the development leader thought that they could not meet our requirements, and questioned the significance of the data of the bug rate, and repeatedly emphasized that the new requirements could be completed on time. The development tasks of the company are already very good. Modifying bugs can only be based on the priority. The high-level orders that affect the release must be changed. Others can be negotiated.
After the meeting, I reflected on why the meeting did not achieve the expected results. First, the analysis of the problem is still at a relatively shallow level. The high bug retention rate is a problem, but it is not only the problem seen on the surface. This problem can be further decomposed, such as type analysis of legacy bugs, and analysis of pending bugs. Second, the content of the meeting and the expected goals were not communicated with the development leader before the meeting, resulting in a long meeting that did not achieve the expected goals. There are differences and collisions in the process improvement this time, which is a challenge and an opportunity for growth. The key to solving the problem is to solve the differences of consciousness. As long as everyone has a consensus to solve this problem, the problem will definitely be improved.
(to be continued)