How to conduct test analysis and design - HTSM heuristic test strategy model | JD Cloud technical team

Testing, without analysis and design, loses its soul;

How should testers conduct test analysis and design before writing use cases? Last time in " The Underlying Logic of Testing " I talked about [Input and Output Test Model], and also talked about [2W+1H Test Analysis Method], but the 2W1H analysis method is a preliminary analysis method, and how to implement it in the test still needs to be implemented. Thinner design.

Today, I will introduce to you the test analysis and design model summarized by James Batch, an expert in the testing field, and the HTSM heuristic test strategy model.

What are HTSMs?

HTSM is a set of test thinking inspiration methods, designed to help testers better think about test strategies, and guide testers on how to think and what points to think about when conducting test analysis and design. HTSM includes four focus areas: testing techniques, project environment, product elements and quality standards . James Batch suggested: You can use them freely, it is common to any type of software, and when applying it, it is recommended to adjust the content of the model according to the actual scene to adapt to your own organizational environment.

James Batch原文解释：The Heuristic Test Strategy Model is a set of patterns for designing and choosing tests to perform. The immediate purpose of this model is to remind testers of what to think about during that process.

The following is a comparison between HTSM and 2W1H analysis methods. Through the comparison, we can see that HTSM can correspond to 2W1H. By analyzing the project environment, we can understand why we tested this project and understand the project background; by analyzing product elements, we determined our test scope. Clear what we want to test; then guide us how to test through testing technology and quality standards.

HTSM compared with 2W1H:

Overview of the HTSM model:

In the HTSM model, each item of project environment, product element, quality standard, and test technology lists many sub-items; among them, [project environment] and [product element] do not need to be fully analyzed according to the sub-items provided by the model. So you can use it as a reference to select and analyze the information that is useful to you.

I will also add some content based on my own experience; [Quality Standards] and [Testing Technology] will be more valuable for reference. I have translated the original text in detail; because it is only a model, the author will not understand each The detailed explanation of the test method or test standard is just a summary made by the author based on his experience. Everyone can summarize more specific methods based on their own experience; but each sub-item of test technology and quality standard can be used as a tester The reference standard for test design, the quality standard can also refer to the ISO9126 software quality model.

In short, the model is not a detailed knowledge explanation, but a collection of ideas and methods to guide everyone in test analysis and design.

ISO9126 Software Quality Model

The four areas of HTSM are explained below: project environment, product elements, quality standards, and testing techniques. You can also use HTSM in this order, analyze according to the 2W1H method, refer to the sub-items and experience provided by HTSM, and finally form a complete test plan.

The first step of the test: [Project environment], before the test, you must first understand the project background and why we do this project?

Background of the project: Why do you want to do this project? What is the background reason for the project?

Problem Solved: What problem does this project solve?

By understanding the background, we can better help us analyze the needs and understand the most original needs and purposes; only after understanding the most original needs can we analyze whether the functional design in PRD can meet the most original needs of users, and then we can better Understand the business and requirements, so that you can dig out the problems or deficiencies in the requirements documents.

Project level: Is it a strategic project? Or a technical maintenance and upgrade project? Understanding the level of the project is also the degree of importance, which also determines the resources we should invest; what level is the quality requirement, and what is the quality requirement of the user.

Expected completion time of the project: The understanding of the completion time can determine what kind of testing strategy we will adopt in the future, which testing methods must be adopted, which ones can be used or not used after going online, and so on. For example, time is very tight, and the most important thing is to guarantee the function. Other performance can be determined according to the online plan; whether it will be used by a large number of users or only a small number of seed users; whether compatibility can be implemented after the online, or for C Compatibility must be guaranteed; automated testing may not be possible, or the automation technology is very mature, with recording-related tools and rich experience, then automated recording tools or recording and playback tools can play a big role at this time effect.

System user situation: Who are the users of the system? How many users? To pay attention to the number of users, the first is to know whether the system has performance pressure testing requirements, and the second is to know how many users the system has, dozens, hundreds or thousands? The number of users is different, and its value must be different.

Who are the customers of the system: Who are the customers of the system? What is the customer's most original request?

The following is the specific content of HTSM [Project Environment], some of which I have supplemented and adjusted.

The second step of testing: [product elements] is what we plan to test; testers must ensure that all product elements are covered, not just the parts we see.

Ultimately, a product is an experience or solution provided to customers; a product has multiple dimensions, and to test well, the dimensions we cover must be comprehensive. Each dimension represents a unique aspect of the product . If the tests only cover part of the coverage, there is a risk of missing serious bugs. Analyzing the elements of the product is to analyze the scope of our testing and what aspects or content we need to test. Most people determine that the test scope comes from the requirements document. In fact, there are many things outside the requirements document that need our attention and need to be tested. The following is the [product element] of HTSM:

The HTSM model [Product Elements] lists a lot of content and is relatively complicated. These contents can be used as our reference, and it is by no means a requirement to test completely according to the following content.

Structure structure: everything included in the product;

I think this part is the easiest for us to ignore. We spend most of our time testing software, and the hardware part is easy to ignore. In addition, non-executable files are easy to ignore, such as help documents, license agreements, etc. Although many of these are business Provided, but in order to be responsible for the company, the product, and the users, we still need to check these non-executable documents; whether the help documents are easy to understand, whether there are any missing, whether there are errors or inconsistent with the system functions; because The person who is most familiar with the system before going online is not the product manager or R&D, but the tester.

In addition, I would like to share some of my experience; first of all, the determination of the test scope is very important, and most people will ignore this step, which may lead to missed tests. This step is easy to ignore, because our test basis is PRD, so the test scope is in PRD; but the actual situation is that most of the test scope is in PRD, and one part is in the design document, or outside the document .

For new projects starting from 0, the test scope can be considered from two perspectives:

1. From the perspective of products, it is basically enough to analyze the PRD; of course, some content of the PRD may be neglected and omitted, so we need to analyze the needs and dig out the hidden needs that may exist.

2. From a technical point of view , in addition to the operable functions seen by users, some functions cannot be directly used by users, or are used by users of other system technology developers; such as interface services provided externally, tasks processed asynchronously in the background of the system, Scheduled tasks, basic data refreshed before going online, pre-data, etc.

Upgrade maintenance items from 1.X to 1.(X+1) or 2.0:

In addition to the normal test scope assessment, the most important and difficult thing is the assessment of the regression test scope, that is, the scope of influence on the original system is the most difficult to assess.

The determination of the scope of regression testing is actually a game between cost and quality; for software with very high quality requirements, each test will require a full amount of regression, so in this scenario, automated regression testing is very important; time requirements For very urgent quality risks that are tolerable or controllable, the scope of regression can be narrowed to the relevant functions of this modification; however, in most scenarios, the time requirement is relatively tight, but the quality cannot be problematic. How to delineate the scope at this time, first of all, the return of the relevant functions of this modification is essential; secondly, it is necessary to keep the bottom line of the system, that is, to determine which functions of the system cannot cause problems, and these functions need to be used as the system every time. Required range for regression . In addition, you can use the function of code diff to analyze the changes and the functions affected.

In addition to the evaluation of the functional regression range, for the upgrade project, we must pay attention to the compatibility verification of the original data ; there are probably the following situations:

1) Whether the function change will affect the operation of the original ongoing data;

2) Changes in the process, whether the data in the process or the rejected and resubmitted data can be carried out normally;

3) The data transmission object (PO, VO, DTO, etc.) changes, whether the data in progress or the re-edited data can operate normally, the most important thing is to verify the transmission or storage of the data.

4) Changes in the underlying database table will affect the display and operation of the original data; whether new fields need to be refreshed; whether the function is normal after swiping.

The third step of testing: [Quality Standards], determine the specific testing strategy, and clarify what type of testing the system should conduct;

Quality standards are some requirements that the product definition should meet, and the rules for testers to judge whether a system has passed the verification; by considering different types of standards, you can better plan the test and quickly find important problems. Each of the following types can be considered a potential risk area.

Capability : Does the system function correctly and does it meet user needs?

Reliability : Will it work under any circumstances?

• Robustness (fault tolerance) : When the system fails, whether it can automatically recover or continue to operate despite the failure.
• Error handling: The product is resistant to failure in the presence of bad data, fails gracefully, and recovers easily. (In case of failure, it can also give accurate prompt information and inform the user how to deal with it)
• Data Integrity: Data in the system is protected and no data loss or data corruption can occur.
• Safety: After the system fails, it will not cause a large amount of loss.

Usability: Is it easy for real users to use the product?

• Ease of learning: the operation of the product can be quickly mastered by target users
• Easy to operate: the product can be easily operated
• Accessible: The product complies with relevant accessibility standards and works with O/S accessibility features.

Security : How protected is the product from unauthorized use or intrusion?

• Authentication: Whether the logged-in user has been authenticated by the system
• Authorization: Whether the user's permissions are controlled and authorized according to different roles or levels
• Privacy: Is customer or employee data encrypted

Scalability ( Scalability ): Whether there is a reasonable plan to deal with the growth of the system (data volume, traffic, complexity)

Compatibility ( Compatibility ): Is it compatible with external components and configurations, and works normally? Whether it can run normally on different hardware platforms, between different application software, in different operating systems, and in different network environments.

• Application Compatibility: Whether the product works with other software products.
• Operating System Compatibility: Whether the product is able to work in different types of operating systems.
• Browser Compatibility: Whether the product is compatible with different types and versions of browsers.
• Hardware Compatibility: This product works with specific hardware components and configurations.
• Backward compatibility: whether the product can be used with its earlier version at the same time, and whether the data and functions are compatible.

Performance ( Performance ): Is the system responsive enough?

Performance testing is often performed by dedicated performance testers, but functional testers must also pay attention to it; in addition to common concurrent stress testing, in fact, more scenarios are due to the large amount of data in the system, which eventually leads to queries, imports , export and other functions respond very slowly; especially if concurrency occurs at the same time, or multiple tasks are parallelized, it is very likely that an export task will eventually be completed after a few days.

Installability : Whether the system can be easily installed on the corresponding platform.

• System Requirements: Will the product recognize that some required components are missing or deficient?
• Configuration: What parts of the system will be affected by the installation? Where are files and resources stored?
• Uninstallation: When the product is uninstalled, can it be cleaned up?
• Upgrades/Patches: Can new modules be easily added or new versions upgraded? Will they affect existing configurations?
• Administration: Is the installation handled by dedicated administrators, or is it on a special schedule?

Ease of maintenance ( Development ): Is the system easy to develop, test, and maintain?

• Supportability: Whether it is possible to provide assistance and support to product users at a lower cost

• Testability: Can a quick test be done in as simple a way as possible

• Maintainability: How easy and expensive is it to build, fix, or enhance the product?

• Portability: How economical is it to port or reuse the technology elsewhere?

• Localizable: How economical is it to adapt the product elsewhere?

The fourth step of testing: [Testing technology], to determine how we will test, what means and methods are used to test, so as to ensure that the quality of the system meets the quality standards and requirements.

Testing technology is the strategy analysis of testing. There are nine commonly used testing technologies as follows.

Function Testing Functional testing: verify each function of the product, and test item by item according to _functional testing_use cases to check whether the product meets the functions required by users.

Claims Testing Constraint Testing:  Challenge every claim! Guarantee the correctness and consistency of the functions mentioned in any materials viewed by users.

1) Clarify relevant reference materials, such as SLA, advertisements, manuals, help texts, operation manuals, etc.; (_SLA_ generally refers to service level agreement. Service level agreement refers to the service agreement between the enterprise providing the service and the customer An agreement or contract mutually recognized by both parties reached in terms of quality, level, performance, etc.)

2) Referring to the above information, test each statement of the product

Flow Testing flow test: a test performed in a certain order

1. Test processes that are connected by multiple processing steps

2. Do not reset the system between related operations or processes

3. Change the time and order, and perform concurrent operations

Domain Testing field test:

1) Analyze any possible input and output of the product

2) Determine the data used in the test for testing. Examples include boundary values, typical values, commonly used values, invalid values, and most representative values.

3) Consider the combination of data tested.

Scenario Testing

1) First consider all possible actual scenarios

2) Design test cases, including meaningful functions for the product and complex interaction scenarios

3) A good scenario test is a compelling story about how an important person does something important to him

Stress Testing stress testing

1) Determine the scope of stress testing. This subsystem or function may be subject to a very large amount of data pressure, or due to resource constraints, large concurrency may cause system overload or "damage";

2) Identify data and resources related to these subsystems and functions;

3) Select or generate challenging data, or resources under limited conditions; for example, large or complex data structures, high load, long test runs, execution of a large number of test cases, low memory conditions

Automatic CheckingAutomated testing

Risk Testing Risk Testing

1) Think about what problems and risks may arise in the product?

2) Which problem is most likely to occur? Focus on those issues that are more likely to occur

3) If they exist, how would you detect and discover it?

4) List these problems and design corresponding test cases, which are specially used to mine them

5) It may also be helpful to consult relevant experts, review design documents, past defect reports, or apply risk heuristics

User TestingUser Testing

1) Identify user categories and roles

2) Determine what operations each category of users will do on a daily basis, how do they generally operate, and what functions do they focus on?

3) Obtain real user data, or introduce real users, and use the system for testing in advance

4) Systematically simulate the usage scenarios of real users (although you are not a real user, it is easy to imagine yourself as a user)

5) The best user testing involves a variety of user roles, and multi-user parallel and cross-operation, not just a single user or a user operation.

Author: JD Retail Zhang Qiang

Content source: JD Cloud developer community

{{o.name}}

{{m.name}}