Foreword: Recently, a batch of new machines have to be tested. The main test ranges include CPU test, disk IO test, thread test, OLTP test, etc., then sysbench can meet our pressure test needs. Let's take a brief look at the installation and use of sysbench and the interpretation of pressure test results.
One, sysbench installation
The sysbench installation is actually very simple, you can refer to git , in fact, the MySQL official website can also be downloaded, but the version is relatively old, the latest version is sysbench1.1.0, here we talk about the source code compilation and installation.
Upload sysbench to the server, decompress and install:
[root@localhost tmp]# unzip sysbench-master.zip
[root@localhost tmp]# cd sysbench-master
[root@localhost sysbench-master]# ./autogen.sh
[root@localhost sysbench-master]# ./configure
[root@localhost sysbench-master]# make -j
[root@localhost sysbench-master]# make install
In fact, you can check README.md, which also writes how to install
## Build and Install
shell
./autogen.sh
# Add --with-pgsql to build with PostgreSQL support
./configure
make -j
make install
Second, the pressure test process and pressure test results interpretation
1.cpu test
[root@localhost storage]# sysbench --test=cpu --cpu-max-prime=2000000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Prime numbers limit: 2000000
Initializing worker threads...
Threads started!
CPU speed:
events per second: 0.63
Throughput:
events/s (eps): 0.6334
time elapsed: 11.0520s
total number of events: 7
Latency (ms):
min: 1571.86
avg: 1578.84
max: 1620.53
95th percentile: 1618.78
sum: 11051.91
Threads fairness:
events (avg/stddev): 7.0000/0.00
execution time (avg/stddev): 11.0519/0.00
The cpu test is mainly for the addition of prime numbers. In our example above, the maximum number of prime generators is specified as 2000000. It can be seen that the server's test execution time is approximately 11.05 seconds
2. Disk IO test
[root@localhost storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
128 files, 245760Kb each, 30720Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
Creating file test_file.5
Creating file test_file.6
Creating file test_file.7
Creating file test_file.8
Creating file test_file.9
Creating file test_file.10
Creating file test_file.11
Creating file test_file.12
Creating file test_file.13
Creating file test_file.14
Creating file test_file.15
Creating file test_file.16
Creating file test_file.17
Creating file test_file.18
Creating file test_file.19
Creating file test_file.20
Creating file test_file.21
Creating file test_file.22
Creating file test_file.23
Creating file test_file.24
Creating file test_file.25
Creating file test_file.26
Creating file test_file.27
Creating file test_file.28
Creating file test_file.29
Creating file test_file.30
Creating file test_file.31
Creating file test_file.32
Creating file test_file.33
Creating file test_file.34
Creating file test_file.35
Creating file test_file.36
Creating file test_file.37
Creating file test_file.38
Creating file test_file.39
Creating file test_file.40
Creating file test_file.41
Creating file test_file.42
Creating file test_file.43
Creating file test_file.44
Creating file test_file.45
Creating file test_file.46
Creating file test_file.47
Creating file test_file.48
Creating file test_file.49
Creating file test_file.50
Creating file test_file.51
Creating file test_file.52
Creating file test_file.53
Creating file test_file.54
Creating file test_file.55
Creating file test_file.56
Creating file test_file.57
Creating file test_file.58
Creating file test_file.59
Creating file test_file.60
Creating file test_file.61
Creating file test_file.62
Creating file test_file.63
Creating file test_file.64
Creating file test_file.65
Creating file test_file.66
Creating file test_file.67
Creating file test_file.68
Creating file test_file.69
Creating file test_file.70
Creating file test_file.71
Creating file test_file.72
Creating file test_file.73
Creating file test_file.74
Creating file test_file.75
Creating file test_file.76
Creating file test_file.77
Creating file test_file.78
Creating file test_file.79
Creating file test_file.80
Creating file test_file.81
Creating file test_file.82
Creating file test_file.83
Creating file test_file.84
Creating file test_file.85
Creating file test_file.86
Creating file test_file.87
Creating file test_file.88
Creating file test_file.89
Creating file test_file.90
Creating file test_file.91
Creating file test_file.92
Creating file test_file.93
Creating file test_file.94
Creating file test_file.95
Creating file test_file.96
Creating file test_file.97
Creating file test_file.98
Creating file test_file.99
Creating file test_file.100
Creating file test_file.101
Creating file test_file.102
Creating file test_file.103
Creating file test_file.104
Creating file test_file.105
Creating file test_file.106
Creating file test_file.107
Creating file test_file.108
Creating file test_file.109
Creating file test_file.110
Creating file test_file.111
Creating file test_file.112
Creating file test_file.113
Creating file test_file.114
Creating file test_file.115
Creating file test_file.116
Creating file test_file.117
Creating file test_file.118
Creating file test_file.119
Creating file test_file.120
Creating file test_file.121
Creating file test_file.122
Creating file test_file.123
Creating file test_file.124
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
32212254720 bytes written in 55.25 seconds (556.00 MiB/sec).
[root@localhost storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 16
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 240MiB each
30GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...
Threads started!
Throughput:
read: IOPS=68491.05 1070.17 MiB/s (1122.16 MB/s)
write: IOPS=45660.96 713.45 MiB/s (748.11 MB/s)
fsync: IOPS=146103.57
Latency (ms):
min: 0.00
avg: 0.06
max: 3.61
95th percentile: 0.31
sum: 156128.82
It can be seen that the read IOPS of the disk can reach 68491.05, the write IOPS can reach 45660.96, and the fsync can reach 146103.57
3. Thread test
[root@localhost storage]# sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 64
Initializing random number generator from current time
Initializing worker threads...
Threads started!
Throughput:
events/s (eps): 14851.1802
time elapsed: 10.0036s
total number of events: 148565
Latency (ms):
min: 0.02
avg: 4.31
max: 24.49
95th percentile: 10.46
sum: 639889.93
Threads fairness:
events (avg/stddev): 2321.3281/42.94
execution time (avg/stddev): 9.9983/0.00
(Send 64 times / test thread request, each time / thread request generates / generates 100 numbers, and the number of locks per thread is 2), test execution time is 10 seconds
4. Memory test
[root@localhost storage]# sysbench --test=memory --memory-block-size=8k --memory-total-size=40G run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Running memory speed test with the following options:
block size: 8KiB
total size: 40960MiB
operation: write
scope: global
Initializing worker threads...
Threads started!
Total operations: 5242880 (1443403.42 per second)
40960.00 MiB transferred (11276.59 MiB/sec)
Throughput:
events/s (eps): 1443403.4239
time elapsed: 3.6323s
total number of events: 5242880
Latency (ms):
min: 0.00
avg: 0.00
max: 0.02
95th percentile: 0.00
sum: 2926.10
Threads fairness:
events (avg/stddev): 5242880.0000/0.00
execution time (avg/stddev): 2.9261/0.00
The above parameters specify that the entire process of this test is to transfer 40G of data in memory, and the size of each block is 8K. The test results show that: the
execution time is 3.6323 seconds, the transmission speed per second is 11276.59Mb per second
5. OLTP test
(1) 100 threads
Data preparation:
[root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=100 --max-requests=0 prepare
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)
Initializing worker threads...
Creating table 'sbtest1'...
Creating table 'sbtest4'...
Creating table 'sbtest2'...
Creating table 'sbtest9'...
Creating table 'sbtest6'...
Creating table 'sbtest8'...
Creating table 'sbtest7'...
Creating table 'sbtest5'...
Creating table 'sbtest10'...
Creating table 'sbtest3'...
Inserting 20000000 records into 'sbtest1'
Inserting 20000000 records into 'sbtest7'
Inserting 20000000 records into 'sbtest4'
Inserting 20000000 records into 'sbtest9'
Inserting 20000000 records into 'sbtest6'
Inserting 20000000 records into 'sbtest8'
Inserting 20000000 records into 'sbtest2'
Inserting 20000000 records into 'sbtest5'
Inserting 20000000 records into 'sbtest3'
Inserting 20000000 records into 'sbtest10'
Creating a secondary index on 'sbtest4'...
Creating a secondary index on 'sbtest1'...
Creating a secondary index on 'sbtest10'...
Creating a secondary index on 'sbtest3'...
Creating a secondary index on 'sbtest6'...
Creating a secondary index on 'sbtest8'...
Creating a secondary index on 'sbtest7'...
Creating a secondary index on 'sbtest9'...
Creating a secondary index on 'sbtest5'...
Creating a secondary index on 'sbtest2'...
start testing:
[root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua --mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=100 --time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_100.log
During the test, you can view the output by tail -f /tmp/40_100.log
Test result: only the statistics part is shown here
SQL statistics:
queries performed:
read: 143195290
write: 40912940
other: 20456470
total: 204564700
transactions: 10228235 (2841.15 per sec.)
queries: 204564700 (56823.07 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)
Throughput:
events/s (eps): 2841.1536
time elapsed: 3600.0289s
total number of events: 10228235
Latency (ms):
min: 2.49
avg: 35.19
max: 2017.54
95th percentile: 39.65
sum: 359973625.38
Threads fairness:
events (avg/stddev): 102282.3500/5194.20
execution time (avg/stddev): 3599.7363/0.01
(2) 200 thread
data preparation:
[root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=200 --max-requests=0 prepare
start testing:
[root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua \
--mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc \
--mysql-password=tpcc --table_size=20000000 --tables=10 --threads=200 --time=3600 \
--report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_200.log
Test Results:
SQL statistics:
queries performed:
read: 142876258
write: 40821788
other: 20410894
total: 204108940
transactions: 10205447 (2834.80 per sec.)
queries: 204108940 (56696.02 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)
Throughput:
events/s (eps): 2834.8009
time elapsed: 3600.0577s
total number of events: 10205447
Latency (ms):
min: 2.39
avg: 70.55
max: 2011.65
95th percentile: 80.03
sum: 719978372.22
Threads fairness:
events (avg/stddev): 51027.2350/3840.19
execution time (avg/stddev): 3599.8919/0.02
(3) 300 thread
data preparation:
[root@localhost storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=300 --max-requests=0 prepare
start testing:
sysbench /usr/local/share/sysbench/oltp_read_write.lua \
--mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest \
--mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=300 \
--time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_300.log
Test Results:
SQL statistics:
queries performed:
read: 144281438
write: 41223268
other: 20611634
total: 206116340
transactions: 10305817 (2862.66 per sec.)
queries: 206116340 (57253.12 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)
Throughput:
events/s (eps): 2862.6561
time elapsed: 3600.0891s
total number of events: 10305817
Latency (ms):
min: 2.64
avg: 104.79
max: 4576.60
95th percentile: 123.28
sum: 1079986824.04
Threads fairness:
events (avg/stddev): 34352.7233/2459.96
execution time (avg/stddev): 3599.9561/0.03
Generate pressure test chart
Install gnuplot
plot 'D:\40_100.log' using 9 with linepoint title '100 threads','D:\40_200.log' using 9 with linepoint title '200 threads','D:\40_300.log' using 9 with linepoint title '300 threads';
Multi-threaded QPS comparison chart
Multi-threaded TPS comparison chart
3. Precautions for pressure measurement
1. During the OLTP test, after the database is initialized, the data directory is cold standby. After each test, the current data directory is deleted, and the data directory is copied to restart, in order to maintain the same environment before each test.
2. After the test is completed, you need to do some cleanup work at the system layer
shell> sync
#refresh dirty data to disk shell> echo 3> / proc / sys / vm / drop_cache #clear cache
shell> swapoff -a && swapon -a release swap
3. The parameter max_prepared_stmt_count needs to be modified during the stress test by default to 16382, to 65536
4. If it is an SSD hard drive, the two parameters innodb_io_capacity and innodb_io_capacity_max should also be adjusted accordingly. It is recommended to adjust it to more than 50000.
5. For the preparation of test data, we observe free buffers through show engine innodb status, try to simulate the production environment, and let the test data fill the entire buffer pool. Here I set the 40G innodb_buffer_pool_size, the number of test tables is 10, and the data of each table is 20 million, which almost fills the buffer pool.
The next article will bring you the stress test of InnoDB engine and TokuDB engine.
Author: charley scavengers
link: https: //www.jianshu.com/p/9823b4aa445a
Source: Jane books
are copyrighted by the author. For commercial reproduction, please contact the author for authorization, and for non-commercial reproduction, please indicate the source.
