Students who do testing should often encounter that the system has no problems in the UAT environment product acceptance after passing the test environment test, but bugs occur as soon as it is put into production, and the more serious situation is that everyone works overtime overnight to test and fix bugs; Testing, and even operation and maintenance, the product blames each other!
In order to improve the current situation that the meeting is a large-scale dumping site, the leader announced one day: Everyone who writes more than ten bugs in the code will be fired!
The development immediately pulled all testers into a small WeChat group and began to discuss countermeasures.
Test: We can't help it. Write the code better?
Development: No matter how good the writing is, it is impossible to write no more than 10, and think about it, if we don't write bugs, what do you test? If you can't detect bugs, will you be fired? Think for yourselves!
Test: What you say makes sense, so what should we do?
Development: Easy to do, we only need to "plan bugs", just like "planned economy". The test was launched today, and it will be launched in three days. There are only 9 bugs in total. You only need to make a decision to mention only three bugs every day.
Development: OK, OK, let's do it.
After a period of time, the leader found that the number of bugs in the system was significantly reduced, so he felt that his leadership ability was excellent, and he was complacent all day long.
See, nothing is lost (nor improved) by leadership, development, and testing, the company is once again a happy family (communication from top to bottom is harder), and the company's bug tracking system no longer effectively reflects the status of bugs. The real situation (a system wasted in vain).
There are policies and measures to counter. The leaders may have follow-up policies, but follow-up countermeasures will definitely be worked out below.
If a company does not improve production efficiency and communication methods, but just beats its head and makes policy, then nothing will change.
Then, after the test goes live
There is a bug in the production environment. Whose fault is it?
Let's first understand what are the main sources of build environment bugs?
1. User Feedback
Users feedback their own opinions and problems encountered through the feedback background, Weibo, APP Store and other channels during use;
2. Internal feedback:
Problems caused by the online version of DingTalk/WeChat/QQ group feedback from various business lines, as well as online problems reported by internal personnel during use;
3. Monitoring background collection:
Problems in user use collected by various operating platforms/monitoring backgrounds.
Quality Analysis of Pot
Now that the team has got the online bug, in order to better divide the responsibility of the pot, and then deal with the online bug, we need to carefully analyze the quality and level of the bug.
For the convenience of analysis, we can simply divide online bugs into three levels:
1. Simple and easy questions:
If it is a relatively simple problem that can be reproduced, then as a tester, this pot cannot escape. Because this is a matter of basic working attitude.
How to avoid such problems? There are a few small ways:
1) Pay attention to the design of the test strategy, and sort out the test focus and test methods of each iteration (version).
2) Test case design: Skillfully use various use case design methods to design and write test cases, so as to ensure the coverage of use cases.
3) Test case review, to avoid falling into the blind spot of testing, and let the product and R&D work together to confirm whether the scenario coverage is sufficient.
4) Seriously execute the test cases, this is very simple, but very important. Because people will be depressed, in some cases, if you are lucky, you may pass the use case directly.
2. Problems that only occur in specific scenarios or data
This kind of situation may not really be considered in the test environment, so you don’t have to worry too much about this kind of problem in the pot test, because it is excusable.
We mainly do that every time we encounter it in the online environment, we must add it to the test case library; if the company has an automated test framework, then we must also add it to the automated test case library;
Of course, at the same time, think about why only this situation occurs, and do a good job of testing the environment and testing is the comparison of data. This kind of problem needs to pay attention to the usual accumulation and form your own experience. This kind of "pot" can be carried by the team together, but an improvement plan must be given, and you cannot fall in the same place many times.
3. Deep-seated occasional problems:
The possibility of this kind of bug being found in the test environment is also relatively low; on the one hand, it happens occasionally, and on the other hand, the depth of testing is more sophisticated, so there is no need to be too burdened with this kind of bug.
And we need to change our mentality to deal with this kind of problem, because this is actually a good testing ground for improving the team's technical capabilities, and we just need to concentrate on solving it.
This kind of problem, under normal circumstances, do a good job of online monitoring, timely warning, and discover it earlier than customers or not too far behind, so as not to cause serious business or economic losses and impacts to users, it is enough; you can prepare relevant issues in advance It is good to use words to appease customers and at the same time allow the team to have a buffer to solve the problem.
By categorizing online bugs in this way, the psychological burden of testing is actually much less. Isn’t it a bad thing to have some “bugs”, and they may not always be severely held accountable by the leaders.
Establish a correct concept of quality
Through the above analysis of online bugs, we found that it is necessary to establish a correct quality concept for each tester and the people around the tester.
Relevant practitioners know that the mission of testing is to find product bugs, ensure the quality of the software, and strive for it for life; but in fact, how far is the quality assurance? Is it possible to find all bugs? Is it the responsibility of the test as long as something goes wrong? This needs to be corrected.
Realize that testing cannot find all bugs in software
We need to recognize this point, just as no developer dares to say that the code he developed has no bugs, and no tester dares to say that the software he has tested has no remaining bugs.
Our responsibility as a tester is to find software bugs as much as possible, but there are always some bugs that cannot be fully discovered, and these will be left in the offline environment to be encountered by users. This is a natural and normal phenomenon.
So, to sum up, an excellent test should do the following:
1) The first thing to do in testing is to ensure that there are no major accidents in the system, the lowest quality assurance, and no low-level errors;
2) If problems arise, reflect on and find solutions in a timely manner, and sum up experience and lessons to find ways to solve or avoid the recurrence of such problems;
3) Do your best to overcome difficult-to-measure problems; constantly remind yourself of your skills in the process of overcoming, and strive to grow on the road of improving test efficiency + deepening test depth!
4) Prevent problems in advance through better practice. For example, unit testing, code walk-through testing, interface testing, buried point testing, chaos testing, etc. will help us find problems early.
Summary of online bug handling
Although it is necessary to divide responsibility for online problems, it can be divided according to the quality type of the bug analyzed above, because the responsibility can be better avoided next time when the same problem recurs.
At the same time, it is more important to do a good job in handling online problems:
1. First check whether the test environment also has such problems. If so, it is obvious that the test environment has not been tested;
At this time, it is necessary to analyze the reasons that are not found in the test environment, optimize the test process and use case library, such as strengthening the review of use cases, cross-testing, and review of test results, etc.;
2. If the test environment does not exist but there is a problem in the online environment, it is likely that the configuration file of some online components caused the problem, such as redis, multi-instance service and other configurations that caused the problem;
At this time, it is necessary to optimize the test environment and make it as close as possible to the online environment, so that more bugs can be found in the test environment.
3. If the problem is solved, do a good job of summarizing and improving the current work content and process.
If it is not resolved, it is necessary to evaluate the risks brought about by this problem, determine whether it needs to be rolled back this time, and how to appease users.
4. The review of the problem, the clue process for the problem, the process and solution of the problem, the specific cause analysis, and the implementation of various tasks to be optimized.