Testing Phases in SDLC
SDLC: software development life cycle
| Static | Dynamic | |
|---|---|---|
| Black Box | Examine MRD/PRD and specs | Data testing, State testing |
| White Box | Formal reviews | Debug, unit test |
MRD: market requirement document
PRD: product requirement document
SPECIFICATION DOCUMENT
• As we have seen, specifications are not easy to write.
• They are meticulously detailed (and often ponderous in their wording)
• Given even a simple program, rarely would two people program it the same way. We need to know what the user really wants.
• One use for the specification document is that testers can find bugs even before a line of code is written.
What Happens If No Spec
• First, try to avoid such projects!
• If you are stuck, you need to wait until you have the software.
• Treat the software as the specification and explore it feature by feature.
• Of course, you cannot tell if a feature is missing.
• Start educating your company about the need to find bugs early in the development process!
Why Examining the Spec ?
In the past, most Software organizations are faced with a compelling need to:
• Reduce cycle time
• Improve quality
• Reduce costs
• Improve productivity
By Examining The Specification, we can:
• We can find many bugs that we can’t find through testing, especially the logic design problems.
• Find defects BEFORE production began.
• Reduction of cost.
• Shortening the software development life cycle.
Why? - Cost Ratio to Fix a Defect
World-Class Defect Detection一流的
Testing the Spec - Philosophy
• This is static, black box testing
• It is static because it is not even a program yet
• It is a black box as you are not concerned with how the specs were established
- Usability studies?
- Focus groups?
- Marketing research?
- Specific customer driven?
黑盒测试
什么是黑盒测试
Black Box Testing is testing without knowledge of the internal workings of the item being tested, treats the system as a “black-box”, so it doesn’t explicitly use knowledge of the internal structure. 黑匣子测试是在不了解被测项目内部运作的情况下进行的测试，将系统视为“黑匣子”，因此不会明确使用内部结构的知识。
Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box. 黑盒测试设计通常被描述为专注于测试功能需求。 黑盒的同义词包括：行为，功能，不透明和封闭盒。
例子
For example, when black box testing is applied to software engineering, the tester would only know the “legal” inputs and what the expected outputs should be, but not how the program actually arrives at those outputs.?例如，当将黑盒测试应用于软件工程时，测试人员只会知道“合法”输入以及预期的输出应该是什么，而不知道程序实际如何实现这些输出。
Black-box testing verifies that the requirements have been met:
How is functional validity tested.
How is system behavior and performance tested.
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundaries of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system operation?
黑盒测试的优点
- The tester does not need knowledge of any specific programming languages, Tester and programmer are independent of each other 测试人员不需要任何特定的编程语言的知识，测试人员和程序员彼此独立
- The test is done from the point of view of the user, not the designer. 测试是从用户而非设计者的角度进行的。
Test cases can be designed as soon as the specs are complete. 规格一完成，就可以设计测试用例。 - More effective on larger units of code than glass box testing 在更大的代码单元上比玻璃盒测试更有效
- Will help to expose any ambiguities or inconsistencies in the specs
将有助于揭露规格中的任何歧义或不一致之处
黑盒测试的缺点
- Testing every possible input stream is unrealistic because it would take a inordinate amount of time; many program paths will go untested. 测试所有可能的输入流是不现实的，因为这将花费大量时间。 许多程序路径将未经测试。
- The test can be redundant if the software designer has run a test case, there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried .如果软件设计人员已经运行了一个测试用例，则该测试可能是多余的；如果没有将测试人员告知程序员已经尝试过的测试用例，则可能不必要重复测试输入。
- Without clear and concise specifications, test cases are hard to design 没有清晰明了的规范，很难设计测试用例
- Cannot be directed toward specific segments of code which may be very complex (and therefore more error prone) 无法定向到可能非常复杂的特定代码段（因此更容易出错
- Most testing related research has been directed toward glass box testing
大多数与测试有关的研究都针对玻璃箱测试
Document testing
- Documentation tests ensure that we have written the required documents.
- Verify the information that the specific document contains is consistent, accurate, and easy to read.
- Sometimes requirements specify the format and audience of the documentation.
Testing the Spec – Where is the beginning
- Although some forms are easier to use, the format is not really critical.
- Even if no one bothered to write specs, interview the development team.
- Write down what you learned and circulate it.
- Stress that your document is what your planning to test against.
- You’ll be astonished how fast details start coming in.
Basic rules
- Don’t start by looking for bugs in great detail.
- Play at being the customer.
- Assume nothing- question until you are viewed as a pest.
- Learn which standards and guidelines can be applied to the product being developed.
You wouldn’t pick the standards and guidelines (the project manage will), but you need to test if the appropriate ones are followed.
Performing a High-Level Review
- Check if there is logic error
- Read MRD, think more about what’s customer needed.
Who are the customers ?
Understand the customer’s expectations
- Existing Standards and Guidelines.
Corporate Terminology and Conventions
Industry Requirements
Government Standards
Graphical User Interface (GUI)
Hardware and Networking Standards
- Review and Test Similar Software
–> Scale–> Complexity–> Testability–> Quality & Reliability
Low-Level Spec Test Techniques
- Specification Attributes Checklist.
Complete? Anything missing or forgotten? thorough
Accurate? Correct solution, properly defined goals, …
Precise, unambiguous, clear? Single interpretation
Consistent? No conflict?
Relevant? Is the statement necessary to specify the feature? Extra info? Traceable to an original customer need?
Feasible? the feature be implemented with available personnel, tools, and resources with the specified budget /schedule
Code-free? Stick with defining the product and not the underlying software design, architecture, code?
Testable?
- Specification Terminology Checklist. P61
Always, Every, All, None, Never. If you see words such as these that denote something as certain or absolute, make sure that it is, indeed, certain. Put on your tester’s hat and think of cases that violate them.
Certainly, Therefore, Clearly, Obviously, Evidently. These words tend to persuade you into accepting something as a given. Don’t fall into the trap.
Some, Sometimes, Often, Usually, Ordinarily, Customarily, Most, Mostly. These words are too vague. It’s impossible to test a feature that operates sometimes.” —can’t test
Etc., And So Forth, And So On, Such As. Lists that finish with words such as these aren’t testable. Lists need to be absolute or explained so that there’s no confusion as to how the series is generated and what appears next in the list.
Good, Fast, Cheap, Efficient, Small, Stable. These are unquantifiable terms. They aren’t testable. If they appear in a specification, they must be further defined to explain exactly what they mean.
Handled, Processed, Rejected, Skipped, Eliminated. These terms can hide large amounts of functionality that need to be specified.
If…Then…(but missing Else). Look for statements that have “If…Then” clauses but don’t have a matching “Else.” Ask yourself what will happen if the “if” doesn’t happen
Software Inspection Process (SIP)软件评审过程
- Planning: Identifies work product to be inspected and sets the inspection schedule.
- Overview: Optional phase where team members who are unfamiliar with the work product to be inspected receive orientation.
- Preparation: Team members inspect the work individually looking for defects in the work product.
- Inspection: Inspection team members met to discuss possible defects in the work product.
- Rework: Thework product is revised to conform to requirements and specifications.
- Follow up: The rework is verified, final inspection data is collected and summarized, and the inspection is officially closed.
SIP Roles
