Summary of database stress testing methods

I. Introduction

In the previous stress test process, the main concern was to stress test the interface and server hardware performance, and evaluate the impact of the request interface and hardware performance on the service. But for most Web applications, the bottleneck of the entire system lies in the database.

The reason is simple: other factors in web applications, such as network bandwidth, load balancing nodes, application servers (including CPU, memory, hard disk, number of connections, etc.), cache, are all easy to achieve through horizontal expansion (commonly known as adding machines) Performance improvement. For MySQL, due to data consistency requirements, it is impossible to increase the machine to disperse the pressure of writing data to the database; although the pressure can be reduced by pre-caching (Redis, etc.), separation of reads and writes, and database and table splitting, but Compared with the horizontal expansion of other components of the system, there are too many restrictions.

Two, common database pressure measurement indicators

Insert picture description here
Similar to the interface pressure measurement indicators, the database-related indicators are as follows:

TPS/QPS: measure throughput.
Response time: Including average response time, minimum response time, maximum response time, time percentage, etc. The time percentage is of great significance, such as the maximum response time of the first 95% of requests. .
Concurrency: The number of query requests processed at the same time.

Three, use sysbench for database pressure testing

1. Basic introduction

sysbench is a cross-platform benchmark test tool that supports multi-threading and multiple databases; it mainly includes the following tests:

CPU performance
Disk io performance
Scheduler performance
Memory allocation and transmission speed
POSIX thread performance
Database performance (OLTP benchmark)
This tool can automatically help you construct a large amount of data in the database, and automatically construct it for you as much data as you want How many pieces of data. At the same time, it can also simulate thousands of threads concurrently accessing the database, and simulate the use of various SQL statements, including simulating various transactions to submit to your database, and even simulate hundreds of thousands of TPS to stress the database. . Insert picture description here
2. Install sysbench:

curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash

sudo yum -y install sysbench

sysbench --version

3. Construct test table and test data based on sysbench:

First, we need to create a test library in our database. We can call it test and create a corresponding test account, which can be called root, and the password is 123456, so that this user has permission to access test.
Build 20 test tables based on sysbench, each with 1 million pieces of data, and then use 100 concurrent threads to initiate access to this database for 5 minutes, which is 300 seconds. The command is as follows:
sysbench --db-driver=mysql --time=300 --threads=100 --report-interval=1 --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user= root --mysql-password=123456 --mysql-db=test --tables=20 --table_size=1000000 oltp_read_write --db-ps-mode=disable prepare
4. Perform the test:

(1) Test the comprehensive read and write TPS of the database, using the oltp_read_write mode:

sysbench --db-driver=mysql --time=300 --threads=100 --report-interval=1 --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=test_user --mysql-password=test_user --mysql-db=test_db --tables=20 --table_size=1000000 oltp_read_write --db-ps-mode=disable run

(2) To test the read-only performance of the database, the oltp_read_only mode is used:

(3) To test the write performance of the database, the oltp_write_only mode is used:

Using the above command, the sysbench tool will construct various SQL statements according to your instructions to update or query the data in your 20 test tables, and at the same time monitor the pressure test performance indicators of your database, and finally complete the pressure After the test, you can execute the cleanup command to clean up the data.

If you are interested in software testing and want to learn more about testing knowledge, solving testing problems, and getting started guidance to help you solve the puzzles encountered in testing, we have technical experts here. If you are looking for a job or just came out of school, or have already worked but often find it difficult, you feel that you have not learned enough in the test, and you want to continue to learn, if you want to change careers, you can join us at 1079636098, in the group You can receive the latest software test interview materials and learning materials for Python automation, interfaces, and framework building!
Insert picture description here
5. Analysis of pressure test results:

According to our above command, we let it output a pressure test report every 1 second, at this time it will output something like the following every one second:

[ 10s ] thds: 100 tps: 500 qps: 5000 (r/w/o: 3500/1000/500) lat (ms, 95%): 20 err/s: 0.00 reconn/s: 0.00

thds: 100, which means that there are 100 threads in the pressure test
tps: 500, which means that 500 transactions are executed per second
qps: 5000, which means that 5000 requests can be executed per second
(r/w/o: 3500 /1000/500): This means that out of 5000 requests per second, 3500 requests are read requests, 1000 requests are write requests, and 500 requests are other requests, which means that QPS is disassembled
lat (ms, 95%): 20, which means that 95% of requests have a delay of less than 20 milliseconds
err/s: 0.00 reconn/s: 0.00: These two mean that there are 0 requests per second It failed, 0 network reconnections occurred

Concluding remarks

The sysbench tool can be used to directly evaluate the performance of the database, but there is still room for further improvement in the display of results. Of course, the results can be further processed later, such as writing into a table or drawing, which will be more intuitive.