Performance test scenario analysis and design? Super detailed case explanation, just read this article

Contents: Guide

foreword

The performance test scenario is actually no different from the functional test, but the focus is different.

The methods of analyzing and designing test cases, such as equivalence class boundary values, which we often use in functional testing, aim to cover business scenarios as much as possible and avoid functional logic missing or failure to meet expectations caused by omissions.

In performance testing, based on performance requirement analysis and design of performance test scenarios, the focus is on request/traffic ratio based on business scenarios and test data preparation.

Design performance test scenarios

Suppose now that we want to carry out a performance test, the requirement background and description are as follows:

Demand background: Internet e-commerce platform;
demand description: verify the performance of order-related business and order service;
expected goal: order service can support the stable operation of daily online business; expected indicators:
service level TPS>200, P0 interface 99RT<100ms , online application CPU%<=40%;

At this time, how to conduct demand analysis and test scenario design?

demand analysis

To verify the performance of the order service;
mixed scene verification is required;
the ratio of request traffic must be considered;

99RT<100ms of the P0 interface;
need to sort out the order service P0 interface;
check whether the relevant monitoring tools are connected;

Scene design

Assume that the order service has 4 P0 interfaces;
they are order creation/confirmation order/order list/order details; the
respective request traffic ratios are 35%/30%/20%/15% (other interfaces with smaller proportions are ignored here) , the real proportion should be considered in actual work);

Then the stress test scenario design is as follows:

serial number	scene name	scene type	Pressure measurement method	Pressure testing purpose	instruction manual
1	Create Order	Stand-alone single interface	Gradient increase	Look for performance inflection points and discover performance bottlenecks	Multiple pressure test verifications may be required
2	Confirm Order	Stand-alone single interface	Gradient increase	Look for performance inflection points and discover performance bottlenecks	Multiple pressure test verifications may be required
3	Order List	Stand-alone single interface	Gradient increase	Look for performance inflection points and discover performance bottlenecks	Multiple pressure test verifications may be required
4	order details	Stand-alone single interface	Gradient increase	Look for performance inflection points and discover performance bottlenecks	Multiple pressure test verifications may be required
5	mixed scene	Stand-alone service level (traffic ratio)	Gradient increase	Look for performance inflection points and discover performance bottlenecks	Multiple pressure test verifications may be required
6	mixed scene	Stand-alone service level (traffic ratio)	Stable concurrent pressure test	Verify performance within expected range	Adjust the concurrency multiple times until the performance reaches the standard
7	mixed scene	Stand-alone service level (traffic ratio)	Stability test (>12h)	Verify long-running stability of services	Use the last stable concurrent pressure measurement value pressure measurement

As shown above, about seven scenarios need to be designed to verify the performance of interface level and service level respectively.

Question: Why not directly pressure test hybrid scenarios?

Answer: Because a service has multiple interfaces, each interface may have factors that affect performance. Through a single interface pressure test, the factors that have performance problems can be quickly checked and solved, which can reduce the performance problem location analysis and Optimize verification difficulty.

data preparation

Data preparation in performance testing depends on the business scenario being tested. Taking the above requirements as an example, two aspects should be paid attention to when preparing test data:

Business logic
Whether the order product inventory is sufficient;
whether the order user has available coupons;
whether the order user coupons can be superimposed; whether the
order product participates in marketing activities ; whether
the order user needs to check the login status;
Can be superimposed;

Data volume
Quantity of the user who placed the order;
the validity period of the user login token;
whether the product inventory is sufficient for multiple uses;
whether the user coupon is sufficient (need to consider coupon write-off and recovery);
creation of marketing activities and coupons & products and marketing activities associated configuration;

After completing the above steps, the next step is to consider the follow-up actions. Subsequent pressure test preparations generally include the following items:

Environmental inspection;
DDL synchronization;
branch release of the service under test;
script development and joint debugging pass;

The examples given are for reference only. In fact, it needs to be designed in combination with specific business requirements, and it is necessary to learn to be flexible.

The following is the most complete software test engineer learning knowledge architecture system diagram in 2023 that I compiled