Plan Test Series

Reprinted from
http://www.uml.org.cn/Test/200911128.asp


(1) - The beginning of everything is difficult The

test plan should be the first test document in the entire testing process, but in general, it is not something that testers learn The first stop. Perhaps because of the difficulty of starting everything, the test plan is really confusing.

When many experienced testers teach newcomers, the first step is not to start with the test plan according to the test process, but to start with the execution of the test case - although this is a helpless move, but for the novice testing In other words, there is still a lot to learn. Gossip has gone a bit far, back to the main topic I want to introduce, planning the test.

Yes, it's a plan test, not a test plan. Although we just talked a little bit about the test plan. But we really need to care about planning tests, not test planning. Always remember that we are doing tests, not documentation, and although we often need various documentation such as test plans, test cases, test reports, those are not the essence of testing.

Since it is a planned test, we must first understand what the test is going to do. The author abstracts it as: a specific person does a specific thing at a specific place at a specific time to achieve a specific goal. In fact, any job can be abstracted into the previous sentence, so we also need to link this sentence with the testing work we are engaged in.

The so-called person, of course, refers to the tester, and the "specific person" adheres to the principle of "division of labor according to ability" to perform their duties. Test case designers do test design, test case executors do execution cases, and so on.

The so-called "specific time" means that our testing process is divided into various stages, and the test points that each stage focuses on are different.

The so-called "specific place" refers to the test environment, which means that we must take into account some special types of tests that require a special environment when planning our test work. This environment includes hardware facilities (such as Mobile phone test, you have to try it with a mobile phone, you can't always talk about it on paper) environment, computer hardware environment and software environment.

The so-called "specific things" refers to our testing technology itself, that is, such as test case design, test case execution, writing test code, deploying test environment and so on.

The so-called "specific goals" refers to the purpose of our test. Testing is costly, and both manpower and material resources are needed. Since we have invested in testing, we expect to get something. The most common slogan in testing is to improve the quality level, and some are still calling for quality assurance, which is what we call "targets". However, unfortunately, these slogans are not very useful, because in actual software projects, we pay more attention to measurable testing work, that is to say, we need to have a measurable "target" - that is, " specific goals" - maybe how many bugs were found, maybe how much test coverage was achieved, etc.

When planning the test, we need to consider more than the test itself. From the above analysis, we can see that we should pay attention to "people, times, places, events, and responsibilities", which is the ancient Chinese emphasis on "time and place benefit people". and" and the like. It should be pointed out that in the process of planning our tests, the question that is most often overlooked is what should we test to achieve. When planning a test, remember to agree on a test goal, which is reflected in the test plan document as "test end criteria".

There is a lot of content about the planned test, in the next article, the author will expand and share with you one by one.

(2) - Test Plan

In the previous article, we mentioned that planning tests should take into account many issues such as people, events, and time, and also mentioned that planning tests should focus on planning the process rather than the test plan document.

This article is devoted to some topics related to test planning. There are now a lot of templates about test plans on the Internet - the use of flood is just to mean a lot, not derogatory, I just missed a good word for a while - these templates are very useful for making a test plan document, but Copying these documents does not help us plan our test work well, but the topics in these test plans can help us plan our test work well and avoid omissions effectively.

I won't give a test plan template that I use frequently, because there are too many templates and enough. In the testing work, the author once wrote two test plans, one is similar to the version currently circulating on the Internet, and the other is the so-called "pragmatic test plan" mentioned in one of the author's blog articles—— In fact, it is a document that is closer to the test design book, but some companies do call it a test plan, and in this series of articles, the author will not discuss such a test plan, nor will I consider "how to design a test plan" in detail. A functional test case" degree planning work.

As mentioned earlier in this article, there are many test plans circulating on the Internet, but there are many similarities. Often some testers randomly go to the Internet to find a few test plan templates, and then they can make a decent test plan. Test plan out. Based on my own experience and some relevant materials (of course, including many test plan templates circulating on the Internet), the author lists the relevant topics related to the plan in the test plan:

test end criteria,
some related conventions, and "terms" are added to some templates. Column Documents and definitions generated in the
test work (test case documents, defect report documents, etc.)
The coordination work before the test work team, mainly including the relevant help the development team needs to provide to the
test Scope of
the test Test schedule (time schedule) Table) Testing
strategy Resource requirements during testing Task assignment of testers Risks that may be encountered during testing



Metrics and statistics of test work
Test tool related planning
and more.

The above topics are common and help us to do a good job in planning. As for the planning of testing costs, the author believes that it is appropriate to estimate but not to pursue too much, because in the actual operation process, the delay of testing work, the purchase of testing tools, Training costs caused by personnel turnover will disrupt this plan, and the costs listed in the test plan will not be directly linked to finance, and the specific costs must follow the company's special process, so topics such as "testing costs" are discussed in the author's There is not much thought in the process of planning a test.

PS: 2009-3-9 update: The

test plan has three levels:

the first level: everything works

Second level: nothing works

Third level: only partially useful

Fourth level: everything

works Fifth level: everything Useless

(3) - People

In the first article of this series, the author mentioned that the essence of planning is "a specific person at a specific time and a specific place to do a specific thing to achieve a specific goal", In the reply to the previous article, Tudou replied to his views on the test plan, which is the definition of 5W1H:
> WHY: Why write a test plan;
> WHAT: What to test;
> WHEN: The start and end time of different stages of the test;
> WHERE: Where to put the document;
> WHO: Who should do it;
> HOW: How to test;

This definition is more detailed for the test plan than mine. However, just as the author declares in the blog signature: pragmatism from the grassroots. Therefore, the definition of 5w1h is not suitable for software workshops with three or five people grabbing more than ten shots. For many companies that have just started their testing activities (only have "specialized testers" in the past two years, note that they are "specialized" and not necessarily "professional") - and this kind of company, as far as some colleagues I contacted, said As mentioned above, there are not a few in China - perhaps some simplified versions of things will be more suitable for them now, and we will gradually get on the right track when we gradually grow up. In this article, I continue my grassroots pragmatism and share some of my humble opinions about the people involved in planning a testing event.

For a while now, more or less people have mentioned the relationship between tools and people on software-related forums. In my opinion, this is a very nonsense question. The role of people cannot be replaced by tools. The reason why people are people and not with other people Animals are in a primitive state of existence because people "use" tools. But that little thing about people and tools is another story.

There is an old saying in China, "Raise an army for a thousand days and spend it in one moment". This sentence is often said by the general (the person in charge of the test) to the soldiers (ordinary testers) at the time of battle. There is also a method in ancient China called the strategy of "wartime when the soldier is idle and the farmer", that is, our vast working people plant our land with peace of mind when there is no war. Once the war breaks out or the country needs it, we put on our armor to fight. . These two sentences give us a hint: we should train our testers or our test team.

Let's take "a thousand days of raising soldiers for a time". As I mentioned above, people often think of this sentence when they are about to fight, but we might as well think about it in reverse. It takes a thousand days of accumulation to use them for a while. of. This is also a reminder to us that everyone in a good test team should be excellent and we need to "raise a thousand days" before "using it for a while". This kind of accumulation cannot be formed in one or two days, as the so-called freezing three feet is not cold in one day. Why talk about this when talking about planning tests? The reason is that "skilled women are hard to cook without rice". If we find that there is no talent available when we are making plans, then our plan may not be able to go on, or we have to prepare to recruit new people to the middle of the ranks, or we can only Outsourcing testing to professional teams, which undoubtedly increases the risk of the project, because newcomers or other teams make us do not understand, only God knows what they will do. When we hand our destiny to God, it is quite Yu is playing with fire. We need to pull "raising a thousand Japanese soldiers" into our plan, and plan our testing work, testing direction, etc. from a longer-term perspective. For the cultivation of talents, the division of labor system is generally used, that is, one or some people are proficient in certain test skills and have an understanding of other skills. Ideally, we are in the direction of the test (or There are "experts" in each test technology related to the company's main development direction, so that a test team can be guaranteed to cope with unpredictable test tasks.

For the grassroots, the company is likely to be just you as a tester at the beginning. There are several situations:

> When the company entrusts you with the arduous task of "building a professional software (testing) team", don't be complacent. Has been valued by boss;

> The company just uses you to advertise that it has tests, and uses you to write test plans, test reports and other professional testing documents submitted to customers for viewing - documents - personnel

The above two situations are relatively common. In my opinion, these two situations are very good for you to create opportunities for learning. In the first situation, you can use the company's "build a professional software (testing) team." "Signature learning; in the second case, if you just write documents, the remaining time can be put to good use, and the purpose is that you want to improve your skills. As for our learning direction, the author roughly summarizes:

> Test theory (including basic concepts, processes, management, etc. of testing. For testing, this is the basics)

> Test documentation (although the content of the documents on the Internet is very important for At present, it may not be completely useful to you, but it is necessary to know how a professional or complete document is written)

> Testing tools (for testers who are just starting out, if you are not a developer, it is recommended that you first Use tools that others have already written)

> Development knowledge (add it if you have it, add it if you don't, always learn, because this is for the future, and this knowledge will help us better test)

The author at the beginning of the article Mentioned the issue of people and tools. There are many kinds of testing tools, including performance testing tools, functional automation testing tools, and so on. However, I saw a blog post yesterday. The author of the blog post deeply felt that the question that almost everyone is discussing is how to use the test tool, but there are very few posts about the development of the test tool. The author also thinks this is an abnormal phenomenon. Indeed, for most software project teams, it is not a realistic idea to develop a performance testing tool by themselves. In view of the importance of performance testing, there is an urgent need to have experts who master mainstream performance testing tools in the testing team. If possible, we have experts in automated testing tools, we have experts in self-developed automated testing tools, etc. These are very useful. However, the order in which these experts are trained should also follow the trend, not only in a hurry but also in a hurry.

When an excellent testing team is established, the problem of "rice" is solved. At this time, it is much simpler to consider how to "cook" for a specific project. Simplicity doesn't mean you can get things straightened out with little effort. You must know that man is a complex animal, and his mood will be cloudy and sunny, and he will have emotions. I will not talk about these issues that do not match with technology. After all, my life experience has not been exciting. It can teach readers how to be a person~ Regarding the topics related to people in the planning test, we will continue to discuss in the subsequent articles of this series in conjunction with "specific things", "specific times" and so on.

(4) -

Earth As the saying goes, the right time and place are right for people. In the previous piece, the author spent a lot of saliva on how to "raise the soldiers" and "the soldiers" how to realize self-cultivation. With people, it is time to consider the issue of "geographical advantage". The so-called geographical location refers to the testing environment of the software, which is very different from the development environment, and also maintains a certain connection (nonsense).

Tests don't come out of nowhere, it's because there are so many lessons learned before that people start to take quality seriously. In this sense, organizations are testing to avoid losses, and reducing spending is actually making money, so they test for profit. On the other hand, testing is also an investment, and the testing environment will inevitably have a share of these expenditures.

Let 's first look at a sketch made by the author.


For the above picture, please briefly explain it, or "follow the picture and ask for it". In our planning of testing, we use a top-down analysis strategy.

The so-called test environment (we refer to the environment in the physical sense here) is divided into two halves with a "click" for software testing: the hardware environment and the software environment.

Take the hardware environment out, including the hardware environment that the test project depends on, and the hardware environment that the test work itself depends on.

The so-called hardware environment that the test project depends on, for example, when we test a mobile phone operating system, we must take out a mobile phone to try it out; if the tractor also needs software equipment, then a tractor is also required, and one needs to be obtained. The warehouse or at least an open space to store the tractor; the so-called hardware environment that the test work itself depends on, at least one test machine is required. For special requirements, such as developing an embedded program to monitor the concentration of indoor carbon dioxide, this time a Special studios may also be necessary, at least one tool to change the CO2 concentration, somewhere to trap this CO2. Regarding the machine, we also need to consider the configuration of the machine and so on.

Next is the software environment. Like the hardware environment, it includes the software that the test project depends on and the software that the test work itself depends on. Of course, the most important thing is to have an operating system and the application to be tested.

I won't talk about the application to be tested, think about what else we can test if there is no program to be tested, right? The software that the test project depends on, there is a little more twists and turns. First of all, some applications referenced or operated by the program to be tested must be prepared neatly. For example, an application is used to monitor how many times a person opens Outlook and how many emails are sent and received every day. If Outlook is not installed on the machine, then we only It is possible to test the performance of the application without Outlook. Although this is also a very important use case, more useful use cases still require us to be equipped with Outlook to test. Secondly, the platform for the program to be tested is running. For .NET development, you must install the corresponding .NET Framework. It is not possible for a web application to have a browser.

The software on which the test work itself depends, shows that the main point is the test tool. There are too many twists and turns, so the author will not go into it.

Operating system, for the software based on the windows operating system, we need to take into account the so many versions that Microsoft has contributed to us over the years. If we consider other operating systems, we have to take into account the contributions of Apple and so on.

To sum up, for testers, all parts (the leaf parts of the graph) do not need to be considered in the project, but the non-leaf node parts still have to be carefully thought out. For developers, a common situation is the environment of software work: when applying Team Foundation to manage team projects, project developer A references an external DLL (assuming C.DLL), and when checking in the source code At this time, the DLL is not checked into TFS, which will cause the version on the server to fail to compile, prompting an error message such as the DLL cannot be found. This is a common environment error. In addition, if the development environment used by the project team members is inconsistent, it may also lead to application integration failure or BVT failure; if the development team development environment is consistent, then when there are compatibility requirements for the application, the relevant system compatibility test is compulsory.

(5) -

Time The time has come, and this is the most tangled part of the planning and testing process. Planning is a very troublesome thing. The key point is that it is difficult to determine the length of time when planning. In a book called "Facts and Fallacies in Software Engineering", the author mentions that one of the two major factors that cause a software project to fail is inaccurate estimation, which shows how difficult it is to plan. Aaron is now also sloppy when it comes to time planning.

On the first day of the new year, I planned to watch the moon and drink with each other on the night of the full moon on the fifteenth day, but there were unforeseen circumstances. On the fifteenth day, the weather turned overcast, and the moon did not even have a shadow, not to mention the full moon. This situation is often encountered in projects. We expect that we will implement a certain function on a certain day in a certain year, but when it really arrives on that day, we find that the function we imagined can only exist and imagine. among.

So how do we plan for time? In Aaron's view, it varies according to the nature of the project. We must know that the projects we are engaged in can be roughly divided into two types: product nature and outsourcing nature. The time requirements for the two types of projects and the requirements for the test intensity are very different.

For general outsourcing projects, the testing requirements are relatively low, and the time is fixed. Most of the current teams that advertise spiral development are actually just a disguised or even metamorphic waterfall model, especially for the current state of testing. Test first, test and project start at the same time is just a slogan in most projects, because everyone knows in their hearts that slogans do not need money, so empty slogans are the cheapest way to put money in the face. It is widely used by software developers. The owners are welcomed and even admired. No nonsense, for the testing work of this kind of project, it is generally a standard segment, that is, planning testing, test case design, test case execution and bug management, test report submission and so on. This is much easier. According to experience (if you have no experience at all, there is still intuition), we convert these stages into proportions, and then divide the total test time. The thing to remind everyone is to remember to leave some points after the division. "Buffer time", otherwise it will be troublesome when something unexpected happens, remember to add a buffer period after each period of time, not the last time.

For products, the testing requirements will be relatively high, and time also needs to be considered. To use the most frequently quoted phrase in the IT industry, "in this rapidly changing era", grasping the timing is undoubtedly very important for a product. However, because many companies are reluctant to have their products born full of poisonous sores - bugs. In light of the damage to the sales of the products, in the worst cases, the production will die prematurely, and even seriously affect the company's image and even lead to serious problems such as the operation of the company. At this time, we should first divide the test into segments. For this kind of project, we first stand in the perspective of test quality, seek truth from facts, and estimate the test time according to the number of function points, difficulty, test experience, etc., and then add up the total time. If the time is sufficient , we consider adding more test surfaces, and if time is tight, we consider whether to delete some non-core functions to reduce the time cost of development and testing, thus escorting the test quality.

Going back to the "Night of the Full Moon" story clip above, this clip reminds us that there are still some issues to be aware of when planning our time. It is mentioned above that the plan failed because the external factor "full moon" did not meet the requirements, which reminds us that in the process of planning, we should minimize the dependence on the outside world. There are several contingency plans. In addition, in the process of project execution, it is also necessary to check the progress of the project in time, which can prevent us from running on the wrong road, which will only lead to further mistakes. This part does not belong to the scope of planned testing, so it is not considered. If you are interested, you can take a look at the continuous integration related materials.

(6)

- The previous article "Planning and Testing Series (5) - Time" of this project was Aaron's weakness and was poorly written, but in order to maintain completeness, Aaron posted it, and saw only a few. The number of people's visits, Aaron felt that he should work hard and write better things. Without further ado, start talking about things in the planning test series.

What does the test do? Testing is for... hurry up and stop, Aaron refers to "things" that are specific things done during the process of a testing project, not reading sentences with "Software Testing" or other classics. According to Aaron's own understanding in the previous article, the software project process or an iteration must go through the stages of planning, implementation and summary. For testing, we can subdivide each stage, and then it becomes the following:

Develop a test plan

As for the role of the plan, it will not be repeated, and the test plan, as the crystallization of the planned test activities, should be paid attention to. In the actual project, Aaron found that the test plan document he wrote was not meaningful, at least not as meaningful as the process of planning the test. In many software workshops, the test plan has been abandoned since birth. The significance of the test plan is only a means used by the workshop owner to put gold in his face. Aaron's recommended approach is to maintain a document called a test plan that is essentially more like a test design (Test Design Spec) after completing an intersecting test plan. This document acts as an outline throughout the test execution process, and any Readers can understand Aaron's ideas and testing ideas for project testing through this Test Design Spec. Aaron won the opportunity to review the Test Design Spec in his project team. Aaron told them this: Aaron was worried that he had misunderstood the meaning of PM, Aaron was worried that he was thinking differently from Dev, and Aaron wanted to clarify things first, so Aaron asked for a review.

About the content of the test plan Aaron also talked about the second part of this series of articles - "Planning Test Series (2) - Test Plan".

Test software requirements

Software requirements are where testing should cover, which is one of the reasons why many software approaches advocate that testing be involved as early as possible in the software development process. We should be skeptical about the Feature list or Feature Spec provided by PM. PM is not an expert in the field of project objects. Many project members, including testers, are required to check this list or spec together, which is called Review. For testers and others involved in the review, it should be possible to read the documentation and have knowledge of the relevant areas of the project. The Review work of Test Design Spec mentioned by Aaron has completed the task relatively well. Of course, limited to relevant business knowledge and experience, the quality of Test Design Spec will be high or low, and the effect of Reivew may be very different. Aaron recommends to constantly temper his own Test Design Spec before submitting a Review. Aaron usually doesn't dare to "share" it with PM and Dev until version V1.3.

Test case design

For test case design methods, such as equivalence class division, boundary value analysis, and even requirement matrix methods, etc., Aaron will not talk about it here. The existing documents on the Internet are much more professional than what Aaron talks about. , not to mention that these are not the purpose of this article.

Execution test

mainly refers to the execution of test cases, but it should also include the update of test cases, as well as the submission and management of bugs and so on. Aaron will also send a Weekly Test Report to project team members every week in longer iterations to help them understand the progress of the testing work in the past week (in terms of the number trend and distribution of test cases), and also report current bug-related issues. Information (total number of bugs, trends, critical bug distribution, regional distribution, etc.), which are helpful in helping the project run smoothly.

report test results

Aaron will send a Weekly Test Report to project team members every week in longer iterations to help them understand the progress of the testing work in the past week (in terms of the number and distribution of test cases), and also report current bug-related information (Total number of bugs, trends, critical bug distribution, regional distribution, etc.), these are very helpful to help the project run smoothly. Of course, after the end of an iteration or the end of the project, we also have to submit a test report, which is a summary report.

Installation test

Consider the various hardware and software environments used by the software, not only in the planning process, but also check whether the definition and introduction of the installation environment are included in the deployment documents or product manuals. Automated

testing The scope and content of automated testing are many. The introduction and implementation of automated testing according to the actual situation of the project team is the trend of software testing development. Performance testing The scope of performance testing includes stress testing, load testing, performance testing (narrow sense), large-capacity testing, etc., which must be selected and arranged according to actual needs, and reflected in the plan. Update (software change) test It mainly refers to the upgrade test of the version, especially for the software of the product nature, more attention should be paid to this aspect. The test work itself also includes many other contents, such as Failover and Switchover tests, etc., which need to be considered. Sometimes the logical relationship of the software, the physical relationship of the software, and the more common interface testing, usability testing, acceptance testing, etc. The choice of these tests and the degree of testing depends on the actual situation of the project (time, cost, etc.) and the personal experience of the tester. (7) - When do we stop? When do we stop our project? We should stop when we reach our goal. But what is the goal? Aaron believes that the so-called goal is the measurable requirement that the test should achieve, which is more commonly called the test stop criterion.

















I don’t know if any programmers will laugh at Aaron and say: Our project is just a tester, and we can successfully deliver useful products to customers even without testers. I don’t know if any testers will say: When we test Nothing but use cases, not to mention boring test stop criteria =! But Aaron tells you that if you really have such a thing in your project, it will only be good for everyone. You think, the test stop standard is the "Plum" that can quench thirst. With it, we will have the direction of struggle and hope to wait until the early days. At the same time, because some project groups also use test standards as release standards - although there are still differences between the two - so test stop standards are also useful for developers and PMs.

Of course, not all test stop criteria will have this effect. In Aaron's view, a qualified test stop standard should meet the following conditions: establish

test stop criteria as early as possible in the planning stage .

We draw the "square and circle" when we plan from the beginning, instead of waiting for a little bit to find out that the "rules" used are not the standard version, which is a waste of time.

The test stop criteria should be confirmed by the project leader.

This is especially true for project teams that are not so harmonious and project teams led by project leaders who are used to indecision. Also pay attention to the lack of evidence, so it is sometimes necessary to establish words as evidence~ Our purpose is to "fix" the rules.

For this article, there are two risks:

First, it is easy to cause discord: if the project leader signs it, it feels like the brothers are shackled to him, more like putting some responsibility on him. . (Because there is such a situation, everyone made a rule, but the top leader was not satisfied with what they did later. At this time, ordinary team members can still infer that the rules of our group are like this, no need to ask, signed at that time to confirm The project leader now has a bigger responsibility.)

The second is because of changes in requirements, because the test stop standard requires measurability (the specific content will be discussed in detail later), so it may involve more detailed things, such as what a core module should be. Later requirements change, and the "fixed" test stop criteria will have to be changed.

Although Aaron has some experience in the prevention and treatment of these risks, considering that the situation of each workshop is different, Aaron will not mislead you.

Test stopping criteria should be measurable

Aaron believes that the requirement of "measurable" is necessary for test stopping criteria, and it is meaningless to describe test stopping criteria in abstract language. For example, it is wrong to say "stop the test at the right time" in the test stop criteria. The so-called "right time" is either confusing, or it appears that "there is one in the eyes of a thousand viewers." A thousand Hamlets" is the case, so a precise definition is required.

The test stop standard is achievable.

This is easy to understand. If there is a standard that requires three or five of us to build an operating system that is exactly the same as Windows Xp within a month for customers, those who set this standard will be afraid It’s the same SB as Aaron the day before yesterday~ Don’t have impossible or hard-to-achieve goals in the test stop criteria. For example, if there is a clause in the test stop criteria that says “fix all bugs”, we have to consider the actual situation. Is it possible for us to fix all bugs, are the requirements of the project so strict - all bugs have to be fixed, not handled (fixed, delayed and reported, etc.), are some bugs allowed to be deferred?

Checkers for Test Stopping Criteria The

test stopping criteria as an acceptance criterion also need to be clearly defined as standard enforcers. If there are no rules, there will be no square circle, but if there are rules and they are not implemented, they will not be able to achieve a "square circle". Therefore, law enforcement or law protectors are needed, which is reflected here to check and verify whether our test meets the standard. Sometimes, in order to express democracy, it is also a desirable way for everyone to have the final say in a small project team.

Aaron talked about some of his own understanding, but it seemed too abstract, so he continued to talk about it in detail. The developer posted the code, but Aaron had no code, so he had to post a few sticky notes

quote

The test standard should include:
》The execution rate of effective test cases (functions) reaches X%?
》Unit test code line coverage reaches X%?
》Unit test case pass rate X%?
"Unit test case design passed the review
" Core modules (A, B, D and other modules) test coverage " All
defects found are included in the defect management system Report and other processing methods) "functional test case module, function point coverage achieved?

quote

Test stop criteria according to test type:
For example, unit testing activities can be stopped after all the following conditions are met:
》100% of the core module code has passed Code Review
》Unit test case design has passed the review
》Test case execution rate is 100%
》The latest version of the unit The test pass rate is 100%
. The unit test global code line coverage is not less than 80%
. The unit test single module code line coverage rate is not less than 70%
. /Thousand lines of code
"All found defects are included in the defect tracking system
" Priority 1 bugs are all fixed
"Priority 2 and 3 bugs are all handled (fixed or not handled and clearly indicated in the test report and passed)
" Completed unit test report and pass review
...

quote

"Standard"
testing activities that will occur in actual work need to be suspended or terminated when one of the following conditions is met:
》The new requirements change too much, the testing activities should be suspended, and continue after the requirement definition is stable;
》Testing exceeds the predetermined time, If the test time cannot continue to increase, the test should be stopped;
》The test cost increases (the bug discovery rate is lower than 1/week, and the defects found at this time are lower than the predefined upper limit);
》If the development is suspended, the corresponding test It should also be suspended, and the data at the point of suspension should be backed up;
》The software system has passed the acceptance test;
》The software project has major estimation and progress deviations in its development life cycle, and when it needs to be suspended or terminated, the test should be suspended or terminated accordingly, and the suspension should be backed up or termination point data;
》Project leader declares to stop the project;
》Team collective (development, management, testing, marketing, sales staff) agrees to stop the project (due to market and interest reasons);
 …

The above notes are from the Internet and personal practice. Aaron is only an excerpt. It must not be used directly as a template. Otherwise, it is the sin of Aaron's misunderstanding~ These notes cannot directly help readers to build a suitable one for themselves. The project's "test stop criterion" because Aaron believes in everyone's abilities. Aaron's purpose in bringing the test stop criteria into the planning test series is to remind readers to have an eye on our results when planning. Whether the project or the testing process is result-oriented, there is no final success. Even if the intermediate process is perfect, it has no meaning for the project (product) itself (in most cases, project members are swallowing failures. while at least gaining experience, so maybe someone else will enjoy the "beautiful process" of failed projects).