欢迎关注MySQL 8.0必知必会系列课程。
MySQL8.0必知必会-自动化部署 https://edu.51cto.com/course/16368.html
MySQL8.0必知必会之参数标准化配置 https://edu.51cto.com/course/16358.html
1 介绍
本文主要介绍sysbench1.0的版本和sysbench-tpcc
真实测试场景中，数据表建议不低于10个，单表数据量不低于500万行，当然了，要视服务器硬件配置而定。如果是配备了SSD或者PCIE SSD这种高IOPS设备的话，则建议单表数据量最少不低于1亿行。
2 部署
2.1 官方主页
2.2 安装
Percona安装源（yum源） [root@zhumh ~]# wget https://www.percona.com/redir/downloads/percona-release/redhat/0.1-6/percona-release-0.1-6.noarch.rpm [root@zhumh ~]# yum -y install percona-release-0.1-6.noarch.rpm [root@zhumh ~]# yum -y install sysbench [root@zhumh ~]# yum -y install sysbench-tpcc |
3 帮助文档
[root@zhumh ~]# sysbench --help Usage: sysbench [options]... [testname] [command] Commands implemented by most tests: prepare run cleanup help General options: --threads=N number of threads to use [1] --events=N limit for total number of events [0] --time=N limit for total execution time in seconds [10] --forced-shutdown=STRING number of seconds to wait after the --time limit before forcing shutdown, or 'off' to disable [off] --thread-stack-size=SIZE size of stack per thread [64K] --rate=N average transactions rate. 0 for unlimited rate [0] --report-interval=N periodically report intermediate statistics with a specified interval in seconds. 0 disables intermediate reports [0] --report-checkpoints=[LIST,...] dump full statistics and reset all counters at specified points in time. The argument is a list of comma-separated values representing the amount of time in seconds elapsed from start of test when report checkpoint(s) must be performed. Report checkpoints are off by default. [] --debug[=on|off] print more debugging info [off] --validate[=on|off] perform validation checks where possible [off] --help[=on|off] print help and exit [off] --version[=on|off] print version and exit [off] --config-file=FILENAME File containing command line options --tx-rate=N deprecated alias for --rate [0] --max-requests=N deprecated alias for --events [0] --max-time=N deprecated alias for --time [0] --num-threads=N deprecated alias for --threads [1] Pseudo-Random Numbers Generator options: --rand-type=STRING random numbers distribution {uniform,gaussian,special,pareto} [special] --rand-spec-iter=N number of iterations used for numbers generation [12] --rand-spec-pct=N percentage of values to be treated as 'special' (for special distribution) [1] --rand-spec-res=N percentage of 'special' values to use (for special distribution) [75] --rand-seed=N seed for random number generator. When 0, the current time is used as a RNG seed. [0] --rand-pareto-h=N parameter h for pareto distibution [0.2] Log options: --verbosity=N verbosity level {5 - debug, 0 - only critical messages} [3] --percentile=N percentile to calculate in latency statistics (1-100). Use the special value of 0 to disable percentile calculations [95] --histogram[=on|off] print latency histogram in report [off] General database options: --db-driver=STRING specifies database driver to use ('help' to get list of available drivers) --db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto] --db-debug[=on|off] print database-specific debug information [off] Compiled-in database drivers: mysql - MySQL driver mysql options: --mysql-host=[LIST,...] MySQL server host [localhost] --mysql-port=[LIST,...] MySQL server port [3306] --mysql-socket=[LIST,...] MySQL socket --mysql-user=STRING MySQL user [sbtest] --mysql-password=STRING MySQL password [] --mysql-db=STRING MySQL database name [sbtest] --mysql-ssl[=on|off] use SSL connections, if available in the client library [off] --mysql-ssl-cipher=STRING use specific cipher for SSL connections [] --mysql-compression[=on|off] use compression, if available in the client library [off] --mysql-debug[=on|off] trace all client library calls [off] --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205] --mysql-dry-run[=on|off] Dry run, pretend that all MySQL client API calls are successful without executing them [off] Compiled-in tests: fileio - File I/O test cpu - CPU performance test memory - Memory functions speed test threads - Threads subsystem performance test mutex - Mutex performance test See 'sysbench <testname> help' for a list of options for each test. |
[root@zhumh ~]# sysbench fileio help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) fileio options: --file-num=N number of files to create [128] --file-block-size=N block size to use in all IO operations [16384] --file-total-size=SIZE total size of files to create [2G] --file-test-mode=STRING test mode {seqwr, seqrewr, seqrd, rndrd, rndwr, rndrw} --file-io-mode=STRING file operations mode {sync,async,mmap} [sync] --file-async-backlog=N number of asynchronous operatons to queue per thread [128] --file-extra-flags=STRING additional flags to use on opening files {sync,dsync,direct} [] --file-fsync-freq=N do fsync() after this number of requests (0 - don't use fsync()) [100] --file-fsync-all[=on|off] do fsync() after each write operation [off] --file-fsync-end[=on|off] do fsync() at the end of test [on] --file-fsync-mode=STRING which method to use for synchronization {fsync, fdatasync} [fsync] --file-merged-requests=N merge at most this number of IO requests if possible (0 - don't merge) [0] --file-rw-ratio=N reads/writes ratio for combined test [1.5] [root@zhumh ~]# sysbench memory help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) memory options: --memory-block-size=SIZE size of memory block for test [1K] --memory-total-size=SIZE total size of data to transfer [100G] --memory-scope=STRING memory access scope {global,local} [global] --memory-hugetlb[=on|off] allocate memory from HugeTLB pool [off] --memory-oper=STRING type of memory operations {read, write, none} [write] --memory-access-mode=STRING memory access mode {seq,rnd} [seq] [root@zhumh ~]# sysbench threads help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) threads options: --thread-yields=N number of yields to do per request [1000] --thread-locks=N number of locks per thread [8] [root@zhumh ~]# sysbench mutex help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) mutex options: --mutex-num=N total size of mutex array [4096] --mutex-locks=N number of mutex locks to do per thread [50000] --mutex-loops=N number of empty loops to do inside mutex lock [10000] [root@zhumh ~]# sysbench cpu help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) cpu options: --cpu-max-prime=N upper limit for primes generator [10000] |
4 使用方法
4.1 命令格式
sysbench [options]... [testname] [command]
4.2 Command
Commands implemented by most tests: prepare run cleanup help
prepare --生成测试需要的数据run --进行性能测试cleanup --清除测试数据
4.3 Testname
包括两种：内建脚本和lua脚本
内建脚本如下：
fileio - File I/O test
cpu - CPU performance test
memory - Memory functions speed test
threads - Threads subsystem performance test
mutex - Mutex performance test
lua脚本如下：
[root@zhumh ~]# ll /usr/share/sysbench/ total 64 -rwxr-xr-x 1 root root 1452 Feb 17 18:01 bulk_insert.lua -rw-r--r-- 1 root root 13756 Feb 17 18:01 oltp_common.lua -rwxr-xr-x 1 root root 1116 Feb 17 18:01 oltp_delete.lua -rwxr-xr-x 1 root root 1957 Feb 17 18:01 oltp_insert.lua -rwxr-xr-x 1 root root 1265 Feb 17 18:01 oltp_point_select.lua -rwxr-xr-x 1 root root 1649 Feb 17 18:01 oltp_read_only.lua -rwxr-xr-x 1 root root 1824 Feb 17 18:01 oltp_read_write.lua -rwxr-xr-x 1 root root 1118 Feb 17 18:01 oltp_update_index.lua -rwxr-xr-x 1 root root 1127 Feb 17 18:01 oltp_update_non_index.lua -rwxr-xr-x 1 root root 1440 Feb 17 18:01 oltp_write_only.lua -rwxr-xr-x 1 root root 1919 Feb 17 18:01 select_random_points.lua -rwxr-xr-x 1 root root 2118 Feb 17 18:01 select_random_ranges.lua drwxr-xr-x 4 root root 4096 Apr 13 14:04 tests |
4.4 Options
4.4.1 全局参数
General options: --threads=N 指定测试使用多少线程数,默认为1 --events=N 测试执行的总events数量，默认0 --time=N 最大执行时间，单位为秒。默认是10 --forced-shutdown=STRING 超过max-time强制中断, 默认是 off [off] --thread-stack-size=SIZE 每个线程的堆栈大小, 默认是 64k --rate=N average transactions rate. 0 for unlimited rate [0] --report-interval=N 在N秒内输出一次统计数据。默认0禁用 --report-checkpoints=[LIST,...] 转储完全统计信息并在指定时间点复位所有计数器。 参数是逗号分隔值的列表，表示从必须执行报告检查点的测试开始所经过的时间（以秒为单位）。 默认情况下，报告检查点处于关闭状态 --debug[=on|off] 是否显示更多的调试信息, 默认是off --validate[=on|off] 在可能情况下执行验证检查, 默认是off。 --help[=on|off] 输出 help 信息, 并退出 --version[=on|off] 输出版本信息, 并退出 --config-file=FILENAME 配置文件 --tx-rate=N deprecated alias for --rate [0] --max-requests=N deprecated alias for --events [0] --max-time=N deprecated alias for --time [0] --num-threads=N deprecated alias for --threads [1] |
4.4.2 伪随机数
Pseudo-Random Numbers Generator options: --rand-type=STRING 分布的随机数{uniform(均匀分布),Gaussian(高斯分布),special(空间分布)}。默认是special --rand-spec-iter=N 产生数的迭代次数。默认是12 --rand-spec-pct=N 值的百分比被视为’special’ (for special distribution)。默认是1 --rand-spec-res=N 'special'的百分比值。默认是75 --rand-seed=N seed for random number generator. When 0, the current time is used as a RNG seed. [0] --rand-pareto-h=N 参数h用于 pareto 分布[0.2] |
4.4.3 日志
Log options: --verbosity=N 日志级别,默认为3,5=debug,0=只包含重要信息 --percentile=N 表示设定采样比例，默认是 95%，即丢弃5%的长请求，在剩余的95%里取最大值 [95] --histogram[=on|off] print latency histogram in report [off] |
4.4.4 数据库
General database options: --db-driver=STRING specifies database driver to use ('help' to get list of available drivers) --db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto] --db-debug[=on|off] print database-specific debug information [off] |
4.4.5 Mysql
Compiled-in database drivers: mysql - MySQL driver mysql options: --mysql-host=[LIST,...] MySQL server host [localhost] --mysql-port=[LIST,...] MySQL server port [3306] --mysql-socket=[LIST,...] MySQL socket --mysql-user=STRING MySQL user [sbtest] --mysql-password=STRING MySQL password [] --mysql-db=STRING MySQL database name [sbtest] --mysql-ssl[=on|off] use SSL connections, if available in the client library [off] --mysql-ssl-cipher=STRING use specific cipher for SSL connections [] --mysql-compression[=on|off] use compression, if available in the client library [off] --mysql-debug[=on|off] trace all client library calls [off] --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205] --mysql-dry-run[=on|off] Dry run, pretend that all MySQL client API calls are successful without executing them [off] |
4.4.6 fileio
[root@zhumh ~]# sysbench fileio help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) fileio options: --file-num=N 指定测试生成多少文件[128] --file-block-size=N --指定测试块的大小(通常测试mysql默认为16k[16384],或8k[8192])[16384] --file-total-size=SIZE --指定测试生成总文件大小(file-total-size/file-num=文件个数)[2G] --file-test-mode=STRING --指定测试磁盘IO的模式 seqwr --顺序写 seqrewr --顺序读写 seqrd --顺序读 rndrd --随机读 rndwr --随机写 rndrw --随机读写 --file-io-mode=STRING 文件操作模式{sync(同步),async(异步),fastmmap(快速map映射),slowmmap(慢map映射)}。默认是sync --file-async-backlog=N 对应每个线程队列的异步操作数，默认为128。 --file-extra-flags=STRING 使用额外的标志来打开文件(sync,dsync,direct),mysql通常用direct方式 --file-fsync-freq=N --执行fsync()的频率,默认为100,0为不使用 (0 - don't use fsync()) [100] --file-fsync-all[=on|off] do fsync()--每次执行完一次写操作就执行一次fsync(),默认为off --file-fsync-end[=on|off] --在测试结束时才执行fsync。默认是on --file-fsync-mode=STRING 用什么样的模式来同步文件fsync, fdatasync (see above) fsync --file-merged-requests=N --如果可以，合并最多的IO请求数(0 – 表示不合并)。默认是0 --file-rw-ratio=N --测试时的读写比例。默认是1.5 生成测试用的数据文件，要求生成的数据文件至少要比内存大，否则数据由于被操作系统缓存而无法体现 I/O 密集型工作负载。 |
4.4.7 cpu
[root@zhumh ~]# sysbench cpu help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) cpu options: --cpu-max-prime=N upper limit for primes generator [10000] cpu 测试为测试计算素数直到某个最大值所需的时间。–cpu-max-prime指定这个最大素数值 |
4.4.8 memory
[root@zhumh ~]# sysbench memory help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) memory options: –memory-block-size=SIZE 测试内存块的大小，默认为1K |
4.4.9 threads
[root@zhumh ~]# sysbench threads help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) threads options: 1. --thread-yields=N number of yields to do per request [1000] 每个请求产生多少个线程，默认[1000] 2. --thread-locks=N number of locks per thread [8] 每个线程的锁数量，默认 [8] 线程调度：线程并发执行，循环响应信号量花费的时间{越少越好} |
4.4.10 mutex
[root@zhumh ~]# sysbench mutex help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) mutex options: 1. --mutex-num=N total size of mutex array [4096] 数组互斥总大小，默认[4096] 2. --mutex-locks=N number of mutex locks to do per thread [50000] 每个线程互斥锁的数量，默认 [50000] 3. --mutex-loops=N number of empty loops to do inside mutex lock [10000] 内部互斥锁的空循环数量，默认 [10000 互斥锁：并发线程同时申请互斥锁循环一定次数花费的时间{越少越好} 测试互斥锁的性能，方式是模拟所有线程在同一时刻并发运行，并都短暂请求互斥锁 |
4.4.11 OLTP
[root@zhumh ~]# sysbench /usr/share/sysbench/oltp_common.lua help sysbench 1.0.2 (using bundled LuaJIT 2.1.0-beta2) oltp_common.lua options: --distinct_ranges=N Number of SELECT DISTINCT queries per transaction [1] --sum_ranges=N Number of SELECT SUM() queries per transaction [1] --skip_trx[=on|off] Don't start explicit transactions and execute all queries as in the AUTOCOMMIT mode [off] --secondary[=on|off] Use a secondary index in place of the PRIMARY KEY [off] --create_secondary[=on|off] Create a secondary index in addition to the PRIMARY KEY [on] --index_updates=N Number of UPDATE index queries per transaction [1] --range_size=N Range size for range SELECT queries [100] --auto_inc[=on|off] Use AUTO_INCREMENT column as Primary Key (for MySQL), or its alternatives in other DBMS. When disabled, use client-generated IDs [on] --delete_inserts=N Number of DELETE/INSERT combination per transaction [1] --tables=N Number of tables [1] --mysql_storage_engine=STRING Storage engine, if MySQL is used [innodb] --non_index_updates=N Number of UPDATE non-index queries per transaction [1] --table_size=N Number of rows per table [10000] --pgsql_variant=STRING Use this PostgreSQL variant when running with the PostgreSQL driver. The only currently supported variant is 'redshift'. When enabled, create_secondary is automatically disabled, and delete_inserts is set to 0 --simple_ranges=N Number of simple range SELECT queries per transaction [1] --order_ranges=N Number of SELECT ORDER BY queries per transaction [1] --range_selects[=on|off] Enable/disable all range SELECT queries [on] --point_selects=N Number of point SELECT queries per transaction [10] |
5 测试样例
5.1 Mysql
5.1.1 准备
创建sbtest数据库 Create database sbtest; |
5.1.2 测试脚本
sysbench --threads=10 --tables=10 --table-size=100000 --rand-type=uniform --report-interval=1 --time=60 --mysql-user=root --mysql-password=root --mysql-socket=/data/mysql/db_zhumh/mysql.sock /usr/share/sysbench/oltp_read_write.lua prepare sysbench --threads=10 --tables=10 --table-size=100000 --rand-type=uniform --report-interval=1 --time=60 --mysql-user=root --mysql-password=root --mysql-socket=/data/mysql/db_zhumh/mysql.sock /usr/share/sysbench/oltp_read_write.lua run |
5.2 IO
--创建10G的文件,分成4个,测试16K块大小,使用direct方式读,测试600秒(10分钟),启用64个线程,每3秒输出一次结果 #sysbench --file-num=4 --file-block-size=16384 --file-total-size=10G --file-test-mode=rndrd --file-extra-flags=direct --max-requests=0 --max-time=600 --num-threads=64 --report-interval=3 fileio prepare #sysbench --file-num=4 --file-block-size=16384 --file-total-size=10G --file-test-mode=rndrd --file-extra-flags=direct --max-requests=0 --max-time=600 --num-threads=64 --report-interval=3 fileio run #sysbench --file-num=4 --file-block-size=16384 --file-total-size=10G --file-test-mode=rndrd --file-extra-flags=direct --max-requests=0 --max-time=600 --num-threads=64 --report-interval=3 fileio cleanup |
磁盘：S3610 * 6 raid10, 内存128G 测试出相关场景下的极限IOPS 随机读写（3:2 oltp场景） sysbench --num-threads=16 --report-interval=3 --max-requests=0 --max-time=300 --test=fileio --file-num=200 --file-total-size=200G --file-test-mode=rndrw --file-block-size=16384 --file-extra-flags=direct run fileio_oltp_32 随机读写（5：1 oltp场景） sysbench --num-threads=16 --report-interval=3 --max-requests=0 --max-time=300 --test=fileio --file-num=200 --file-total-size=200G --file-test-mode=rndrw --file-block-size=16384 --file-extra-flags=direct --file-rw-ratio=5 run fileio_oltp_51 随机写 sysbench --num-threads=16 --report-interval=3 --max-requests=0 --max-time=300 --test=fileio --file-num=200 --file-total-size=200G --file-test-mode=rndwr --file-block-size=16384 --file-extra-flags=direct run fileio_rndwr 随机读 sysbench --num-threads=16 --report-interval=3 --max-requests=0 --max-time=300 --test=fileio --file-num=200 --file-total-size=200G --file-test-mode=rndrd --file-block-size=16384 --file-extra-flags=direct run |
6 Tpcc
6.1 Option
[root@localhost sysbench]# sysbench /usr/share/sysbench/tpcc.lua help sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) tpcc.lua options: --enable_purge=STRING Use purge transaction (yes, no) [no] --force_pk=N Force using auto-inc PK on history table [0] --mysql_storage_engine=STRING Storage engine, if MySQL is used [innodb] --mysql_table_options=STRING Extra table options, if MySQL is used. e.g. 'COLLATE latin1_bin' [] --report_csv=STRING Report output in csv (yes, no) [no] --scale=N Scale factor (warehouses) [100] --tables=N Number of tables [1] --trx_level=STRING Transaction isolation level (RC, RR or SER) [RR] --use_fk=N Use foreign keys [1] |
6.2 测试脚本
./tpcc.lua --mysql-socket=/mysqldb/db_master/mysql.sock --mysql-user=root --mysql-password=zabbix --mysql-db=sbt --time=30 --threads=64 --report-interval=1 --tables=10 --scale=10 --db-driver=mysql prepare ./tpcc.lua --mysql-socket=/mysqldb/db_master/mysql.sock --mysql-user=root --mysql-password=zabbix --mysql-db=sbt --time=30 --threads=64 --report-interval=1 --tables=10 --scale=10 --db-driver=mysql run ./tpcc.lua --mysql-socket=/mysqldb/db_master/mysql.sock --mysql-user=root --mysql-password=zabbix --mysql-db=sbt --time=30 --threads=64 --report-interval=1 --tables=10 --scale=10 --db-driver=mysql cleanup |