[Performance Test] Design relationship between business/throughput and existing data + common solutions for stress testing

---

Preface

1. Design relationship between business volume, throughput and inventory data in performance testing

1) Business volume

It does not have a time unit. When we mention business volume, we will definitely add a time unit. For example, the daily business volume is 1 million, the annual business volume is 100 million, and so on.

2) Throughput

It has its own time unit. Throughput is the number of services processed per unit time.

The relationship between business volume and throughput
So the question is, which word should we use when doing performance testing? Business volume or throughput?
In fact, we use both words. Because their connotations are different.

The goals of the business department are often the business volume in a year and the business volume in a day. These goals do not outline performance testing goals. Because what the performance test wants is the amount of business per second.

for example:

240,000 transactions a day does not mean that 10,000 transactions are processed in an hour. Maybe these 240,000 transactions are processed in an hour.

Users can often tolerate waiting for one or two seconds. If they wait for 10 seconds, they may think that the system has hung up. Therefore, the system must process business requests/messages in a timely manner without causing serious accumulation. What we are most concerned about is how many businesses are processed in one second. Not how much in an hour, or how much in a day.

There is a conversion process here.

The business requirement is to process 20 million transactions a day, so what is the throughput target?

This depends on the performance model of the system. Which period of the day has the largest business volume, and what percentage of the business volume during this period accounts for the day's business volume? Then step by step, we calculate the business volume of one second during the peak period, which is our throughput target.

3) Stock data

We have the throughput target, but we cannot start testing immediately. This is because there is a third concept, stock data.

If the database is an empty database, or the database is a database with 200 million records, then the response time of SQL addition, deletion, modification and query is completely different. The corresponding business response times are also completely different. So, how much existing data should be set in our database?

The relationship between stock data and business volume

The concept of business volume is used for the estimated stock data.
After all, inventory data is measured in years, months, and days, not seconds.

For example, the inventory data of this system is 3 years, or 3 days, not 3 seconds.

Moreover, calculating one year's inventory data is definitely not calculated using the peak business volume per second of 360024*365. Instead, the business volume mentioned by the business department is used.

For example, the business volume this year is expected to be 10 billion, and the annual growth rate is expected to be 50%. Then, if you want to test whether the system can meet the needs after 3 years, you need to calculate it like this: Assuming that the system's stock data is saved for 3 years, then the stock data after 3 years is (100+1001.5+1001.5*1.5) billion.

The throughput after three years is calculated as follows:
Assume that the hour with the largest business volume in a day accounts for 20% of the whole day's business volume, and the second with the largest business volume accounts for this hour 1/2000. Assume that the business volume per day is A.

This year’s throughput target per second is B=A20%(1/2000). Assuming the performance model remains unchanged, the throughput target per second after three years is C=B1.51.5.

2. Several common solutions for stress testing

Concurrency (stress testing) refers to the processing of multiple users trying to access the same data at the same time. The key to the problem lies in how to design the application to handle concurrency issues, especially in many current systems where multiple users access shared resources. , common solutions are as follows:

1) Conservative approach

This concurrency model adds a lock to the data. For example, when a user operates a record in the database, in an environment that allows editing, the system will reject requests from other users to read the data.

This method is most suitable for situations where it is likely that more than one user will edit the same data at the same time, although this method has a certain complexity in implementation.

The main concern in testing concurrency in this mode is to verify that the lock on the record is correctly acquired and released, and that all parts of the application that may update the record are correctly handled.

Lock acquisition:
Because only one user can enter the update state of a data record or data item at the same time, the key is that the system must correctly allocate the lock to the first one The requesting user. The operation to acquire the lock should be operational.

The specific method is:
Let two users try to enter the editing state at the same time or use a large number of requests. For the latter, we can use a script to generate multiple simultaneous requests. Edit the data request to verify that only one request was successful.

Effectiveness of the lock:
Verifying the validity of the lock must ensure that any other user cannot modify this data in any way (such as modification and deletion).

The specific verification method is:
Let a user open a record (enter edit mode and maintain this state), while other users try to edit, delete, etc. in all places in the application All methods are used to update data, and the system should reject all attempts by other users to update data.

Lock release:
Must verify: after the user editing the data releases the record, the system can allow other users to edit the record. Another aspect to note is errors Processing, that is, what operations should the system complete when the user holding the lock makes an error (such as a client crash), and the system's ability to recover from a failure to release the lock should be considered.

2) Open method

In this mode, the user is always allowed to read the data, and may even be allowed to update the data, but when the user attempts to save the data, the system will automatically check whether anyone else has updated the data since the user retrieved the data. If the data occurs changes, then the update fails.

This approach allows more users to view the data than the conservative model, so it is suitable for situations where it is unlikely that multiple people will modify the same data at the same time.

In this mode, updates are the only point of concern. The best testing method is to combine manual and automatic testing techniques. During manual testing, two testers edit the data and then try to save the data at the same time. The operation updated by one user succeeds. Finally, another user gets a message that other users have updated the data. At this time, he can only reload the data and complete the modification operation again.

When using the automatic mass testing method, in the same way, only one user can update the record, and other users will receive prompts because other users have already updated the data, so his operation is invalid.

3) No concurrency protection

It is the simplest of all modes. In layman's terms, the victory belongs to the last user. However, when two users modify a record at the same time, data corruption may occur.

In this mode, regardless of the order of update requests, all users should successfully complete the update operation. Special attention needs to be paid to data integrity and update errors, such as: when a user updates a record, it is indeed deleted. .

Also note when dealing with concurrent testing that when the same data can be updated through different interfaces or functions, all functions that may access this record should be tested.

The following is the most comprehensive software testing engineer learning knowledge architecture system diagram in 2023 that I compiled.

1. Python programming from entry to proficiency

2. Practical implementation of interface automation projects

3. Web automation project actual combat

4. Practical implementation of App automation project

5. Resumes of first-tier manufacturers

6. Test and develop DevOps system

7. Commonly used automated testing tools

8. JMeter performance test

9. Summary (little surprise at the end)

Every effort is an accumulation of success, and only those who persevere can climb to the top of life. Let us work together, realize our dreams with practical actions, and make life more exciting!

Only by facing challenges bravely can you stimulate your potential. Let us fight together, use perseverance and courage to conquer difficulties, achieve self-transcendence, and create a better future!

Struggle is the ladder to realize dreams. Only by continuous hard work and hard work can we reach the other side of success. Let us move forward bravely together, water hope with sweat, and let dreams illuminate the way forward!