1. Benchmarking
基准测试（benchmarking）It is a type of performance test that emphasizes the quantitative, reproducible, and comparable test of certain performance indicators of a type of test object.
To further understand, it 基准测试is to establish a known performance level (called a baseline) through a benchmark test at some point, and perform a benchmark test after the system's software and hardware environment changes to determine the impact of those changes on performance. It is also the most common use for benchmarking. Other uses include determining performance limits under certain load levels, managing system or environmental changes, discovering conditions that may cause performance problems, and so on.
2. The role of benchmarking
For most web applications, system bottlenecks tend to occur easily on the database side . The reason is simple: other factors in web applications, such as network bandwidth, load balancing nodes, application servers (including CPU, memory, hard disk lights, connection It is easy to achieve performance improvement through horizontal expansion (commonly known as adding machines). For databases MySQL, for example , due to data consistency requirements, the pressure of writing data to the database cannot be dispersed by adding machines; although the pressure can be reduced by pre-caching (Redis, etc.), separation of reads and writes, and sub-databases. But compared with the horizontal expansion of other components of the system, it is subject to too many restrictions.
The role of the database benchmark test is to analyze the performance of the database under the current configuration (including hardware configuration, OS, database settings, etc.), so as to find out the performance threshold of the database, and adjust the configuration according to the actual system requirements. In addition, benchmarking the database server is also usually used to observe how the performance of the database structure will be affected before and after the modification.
3. The difference between benchmark testing and stress testing
In many cases, benchmark testing and stress testing can easily be confused during actual use. Benchmark testing can be understood as a stress test for the system. But benchmark tests do not care about business logic, and are simpler, more direct, and easier to test. Data can be generated by tools and does not require truth; while stress tests generally consider business logic (such as shopping cart business) and require real data.
4. Benchmarking indicators and tools
Common database indicators include:
- TPS/QPS: Mainly used to measure throughput.
- Response time: Including average response time, minimum response time, maximum response time, time percentage, etc. The time percentage is of great significance, such as the maximum response time of the first 95% of requests.
- Concurrency: The number of query requests processed at the same time.
If it is only for the database server, for example, only for the MySQL database benchmark test, you can generally use a special tool, such as mysqlslap, sysbenchetc.
mysqlslapIt is the benchmark tool that comes with MySQL 5.1.4. This tool can simulate multiple clients to send queries and updates to the server at the same time, give performance test data and provide performance comparisons of multiple engines.
Among them, it sysbenchis mysqlslapmore general and more powerful. This article will introduce how to use sysbench to carry out benchmark tests. The reason why it is said to be a test artifact is that in addition to using it to perform benchmark tests on various infrastructure objects, it can also Use its implementation ideas to help us generate tens of millions of test data, look down, let's reveal the secret together~
5. sysbench benchmark test
sysbenchIt is a modular, cross-platform, multi-threaded benchmark test tool, mainly used to evaluate and test the database load under various system parameters. It mainly includes the following tests:
- cpu performance
- Disk io performance
- Scheduler performance
- Memory allocation and transmission speed
- POSIX thread performance
- Database performance (OLTP benchmark test)
At present the main support sysbench MySQL, pgsql, Oraclethese three types of database
MySQL is supported by default . If you need to test Oracle/PostgreSQL, you need to add –with-oracleor –with-pgsqlparameters when configuring .
The biggest highlight of benchmarking the database through the sysbench tool is that it can automatically construct a large amount of data in the database for you, and automatically construct as many data as you want. At the same time, it can also simulate thousands of threads concurrently accessing the database, and simulate the use of various SQL statements, including simulating various transactions to submit to your database, and even simulate hundreds of thousands of TPS to stress the database. .
6. Basic syntax of sysbench
1. Before use, you need to install it first. It is recommended to install sysbench under Linux
yum install -y sysbench
2.The basic syntax of sysbench is as follows:
sysbench [options]... [testname] [command]
Among them, command is the command sysbenchto be executed, including prepare, runand cleanup.
As the name suggests:
- prepare is to prepare data in advance for testing
- run is to perform a formal test
- Cleanup is to clean up the database after the test is completed.
The testname specifies the test to be performed. In the old version of sysbench, the test script can be specified through the --test parameter; in the new version, the --test parameter has been declared as obsolete, so you can not use --test, but Specify the script directly.
The following two methods have the same effect:
sysbench --test=./tests/include/oltp_legacy/oltp.lua
sysbench ./tests/include/oltp_legacy/oltp.lua
The script used in the test is a lua script. You can use sysbench's own script or develop it yourself. For most applications, the script that comes with sysbench is sufficient. In different versions of sysbench, the location of the lua script may be different, you can use the find command to search under the sysbench path oltp.lua.
Most data services are of the oltp type. If you do not understand what oltp is, then there is a high probability that your data service is of the oltp type.
7. sysbench use practice (one-click to build millions of data)
Next, I will show you how to use sysbench to benchmark MySQL and how to use sysbenchone command to build millions of test data.
Note: sysbenchBenchmark test skills, in the author's 全栈测试开发训练营database section, have a special introduction to the trainees of the training camp, this article can be regarded as a welfare for the public account readers.
Operation method:
1. In the benchmark preparation stage, first create a default test library, for example sysbench_test:
mysql> create database sysbench_test; #创建数据库
As here, create a sysbench_test database
2. It takes a long time to prepare the data. This article is mainly to demonstrate the method, so the table_size is set smaller
sysbench /usr/share/sysbench/oltp_read_write.lua --tables=5 --table_size=100 --mysql-user=root --mysql-password=xxx --mysql-host=192.168.0.103 --mysql-port=3306 --mysql-db=sysbench_test prepare
Among them, several key parameters:
--tables: Specify the number of generated tables. 5 tables are set here, indicating that 5 test tables are generated. Readers can adjust this value according to actual needs.--table_size: Specify the amount of data generated in the generated table. The above example shows that each table generates 100 test data. Actually, the quoted value can be adjusted as needed. For example, if it is adjusted to 1000000, it means that one million test data is generated.--mysql-db: The name of the connected test database.
The other parameters are relatively simple, so I won't introduce them one by one.
After executing the above command, the output is as follows:
After the command is executed successfully, you can open the database to check whether the generated data corresponds to the setting:
As you can see, 100 pieces of test data have been automatically generated. If you need to generate millions of pieces of test data with one click, you only need to --table_size=100 modify it --table_size=1000000to.
3. Select a lua script to test, such as verifying read and write performance, execute the command as follows:
sysbench /usr/share/sysbench/oltp_read_write.lua --mysql-user=root --mysql-password=xxx --mysql-host=192.168.0.103 --mysql-port=3306 --mysql-db=sysbench_test --tables=5 --table_size=100 --threads=10 --time=30 --report-interval=3 run
The above command indicates that 10 concurrent threads are used, the execution time is 30 seconds, and the test information is output every 3 seconds.
Actual readers can increase the --threads=10sum --time=30value.
The output information is similar to the following: 
Among them, the more important information for us includes:
queries: Total number of queries and qpstransactions: Total number of transactions and tpsLatency-95th percentile: Response time for the first 95% of requests.
4. After executing the test and getting the required test results, remember to clean up the data in the last step, otherwise the subsequent tests may be affected.
sysbench /usr/share/sysbench/oltp_read_write.lua --tables=5 --table_size=100 --mysql-user=root --mysql-password=xxx --mysql-host=192.168.0.103 --mysql-port=3306 --mysql-db=sysbench_test cleanup
If you find it useful, you can use one-click triple connection