pt-query-digest是Percona工具包的一部分,用于对查询进行分析。可以通过以下任何方式收集插叙:
慢查询日志
通用查询日志
进程列表
二进制日志
TCP转储
慢查询日志
首先要开启慢查询日志并收集查询,然后就可以通过传递慢查询日志来运行pt-query-digest。
通过修改配置文件,打开慢查询日志
做几次查询,生成慢查询日志后,使用工具进行分析
摘要报告(digest report)中的查询按照查询执行的次数与查询时间的乘积排列。所有查询的详细信
息,例如查询校验和(每种查询类型有一个唯一的值)、平均时间、百分比时间和执行次数等都会显示
出来。可以通过搜索查询校验和来深入研究特定查询。
摘要报告的内容如下所示:
[root@www mysql]# vim /etc/my.cnf
[mysqld]
#slow_log
slow_query_log=on #开启
slow_query_log_file=/var/lib/mysql/www-slow.log #日志文件
long_query_time = 0.001 #为了测试,定义超过0.001秒的就记录
log_queries_not_using_indexes = on #设置对未使用索引的SQL进行记录
[root@www mysql]# pt-query-digest /var/lib/mysql/www-slow.log >
/root/query_digest
[root@www ~]# cat query_digest
# 40ms user time, 420ms system time, 26.11M rss, 224.43M vsz
# Current date: Sun Oct 6 16:28:28 2019
# Hostname: www.lnmp.com
# Files: /var/lib/mysql/www-slow.log
# Overall: 2 total, 1 unique, 0.03 QPS, 0.00x concurrency ________________
# Time range: 2019-10-06T08:21:13 to 2019-10-06T08:22:31
# Attribute total min max avg 95% stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 247ms 73ms 174ms 123ms 174ms 72ms 123ms
# Lock time 331us 124us 207us 165us 207us 58us 165us
# Rows sent 38.74k 5.53k 33.21k 19.37k 33.21k 19.57k 19.37k
# Rows examine 585.98k 292.99k 292.99k 292.99k 292.99k 0 292.99k
# Query size 146 73 73 73 73 0 73
#对此查询的分析如下所示:
云计算-legolas制作
通用查询日志
可以使用pt-query-digest通过传递参数--type genlog来分析通用查询日志。由于通用日志不报告查询的
次数,因此只显示累计的数字:
# Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== ============= ===== ====== ===== ===============
# 1 0x8D07475FE113B1C0 0.2466 100.0% 2 0.1233 0.04 SELECT employees
#对于#1查询(0x8D07475FE113B1C0),其所有的执行累计响应时间为0.2466秒,
#占全部查询的累计响应时间的100%。执行次数为2次
#平均查询时间为0.1233秒
# Query 1: 0.03 QPS, 0.00x concurrency, ID 0x8D07475FE113B1C0 at byte 0 __
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.04
# Time range: 2019-10-06T08:21:13 to 2019-10-06T08:22:31
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 100 2
# Exec time 100 247ms 73ms 174ms 123ms 174ms 72ms 123ms
# Lock time 100 331us 124us 207us 165us 207us 58us 165us
# Rows sent 100 38.74k 5.53k 33.21k 19.37k 33.21k 19.57k 19.37k
# Rows examine 100 585.98k 292.99k 292.99k 292.99k 292.99k 0 292.99k
# Query size 100 146 73 73 73 73 0 73
# String:
# Databases employees
# Hosts localhost
# Users root
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms ################################################################
# 100ms ################################################################
# 1s
# 10s+
# Tables
# SHOW TABLE STATUS FROM `employees` LIKE 'employees'\G
# SHOW CREATE TABLE `employees`.`employees`\G
# EXPLAIN /*!50100 PARTITIONS*/
select first_name,last_name from employees where hire_date > ‘1998-01-01’\G
云计算-legolas制作
做几次查询后,生成日志并使用工具进行分析
结果如下:
mysql> set global general_log = ‘ON’;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like ‘%general_log%’;
±-----------------±-----------------------+
| Variable_name | Value |
±-----------------±-----------------------+
| general_log | ON |
| general_log_file | /var/lib/mysql/www.log |
±-----------------±-----------------------+
2 rows in set (0.00 sec)
[root@www mysql]# pt-query-digest --type genlog /var/lib/mysql/www.log >
/root/general_query_digest
[root@www ~]# cat general_query_digest
# 110ms user time, 10ms system time, 25.58M rss, 220.06M vsz
# Current date: Sun Oct 6 16:58:45 2019
# Hostname: www.lnmp.com
# Files: /var/lib/mysql/www.log
# Overall: 7 total, 4 unique, 0 QPS, 0x concurrency ______________________
# Time range: 2019-10-06T08:55:42.298155Z to 2019-10-06T08:57:26.314967Z
# Attribute total min max avg 95% stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 0 0 0 0 0 0 0
# Query size 489 29 209 69.86 202.40 57.22 34.95
# Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== ============= ===== ====== ===== ===============
# 1 0x7DD5F6760F2D2EBB 0.0000 0.0% 3 0.0000 0.00 SHOW VARIABLES
# 2 0x8D07475FE113B1C0 0.0000 0.0% 2 0.0000 0.00 SELECT employees
# 3 0x956C52D06C06E02E 0.0000 0.0% 1 0.0000 0.00 SET
# 4 0xE71D7852C493A0A8 0.0000 0.0% 1 0.0000 0.00 SET
# Query 1: 0 QPS, 0x concurrency, ID 0x7DD5F6760F2D2EBB at byte 955 ______
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: 2019-10-06T08:55:42.298155Z to 2019-10-06T08:57:26.314967Z
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 42 3
# Exec time 0 0 0 0 0 0 0 0
# Query size 21 105 35 35 35 35 0 35
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
show variables like ‘%general_log%’\G
# Query 2: 0 QPS, 0x concurrency, ID 0x8D07475FE113B1C0 at byte 483 ______
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: 2019-10-06T08:55:52.258705Z to 2019-10-06T08:56:06.280872Z
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 28 2
# Exec time 0 0 0 0 0 0 0 0
# Query size 29 146 73 73 73 73 0 73
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
# Tables
# SHOW TABLE STATUS LIKE 'employees'\G
# SHOW CREATE TABLE `employees`\G
# EXPLAIN /*!50100 PARTITIONS*/
select first_name,last_name from employees where hire_date > ‘1985-01-01’\G
# Query 3: 0 QPS, 0x concurrency, ID 0x956C52D06C06E02E at byte 700 ______
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: all events occurred at 2019-10-06T08:57:15.264670Z
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 14 1
# Exec time 0 0 0 0 0 0 0 0
# Query size 42 209 209 209 209 209 0 209
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
set global general_log = ‘off’
/usr/sbin/mysqld, Version: 8.0.16 (MySQL Community Server - GPL). started with:
Tcp port: 3306 Unix socket: /var/lib/mysql/mysql.sock
Time Id Command Argument\G
# Query 4: 0 QPS, 0x concurrency, ID 0xE71D7852C493A0A8 at byte 629 ______
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: all events occurred at 2019-10-06T08:56:48.591288Z
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 14 1
进程列表
可以使用pt-query-digest而非日志文件从进程列表(process list)中读取查询:
run-time指定每次迭代应该运行多长时间。以上例子中该工具会每分钟生成一份报告,并持续10分钟。
二进制日志
要使用pt-query-digest分析二进制日志,应该先用mysqlbinlog工具将其转换为文本格式:
TCP转储
可以使用tcpdump命令捕获TCP流量,并将其发送给pt-query-digest进行分析:
在pt-query-digest中有很多选项可供选择,例如用特定时间窗口筛选查询,筛选某个特定的查询以及生
成报告。更多信息请参阅Percona文档。
# Exec time 0 0 0 0 0 0 0 0
# Query size 5 29 29 29 29 29 0 29
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
set global general_log = ‘ON’\G
shell> pt-query-digest --processlist h=localhost --iterations 10 --run-time 1m -
u -p
[root@www binlogs]# mysqlbinlog /data/mysql/binlogs/server1.000040 >
/root/binlog.000040
[root@www binlogs]# pt-query-digest --type binlog /root/binlog.000040 >
binlog_digest
shell> tcpdump -s 65535 -x -nn -q -tttt -i any -c 1000 port 3306 > mysql.tcp.txt
shell> pt-query-digest --type tcpdump mysql.tcp.txt> tcpdump_digest