环境是mysql8.0
之前调研过mysql audit 插件，均没有发现能适合mysql8.0环境的，除了企业版mysql审计插件，于是想了一个办法，把mysql慢日志时间改为0，这样所有sql都会被记录到慢日志里面,然后在用pt工具解析，过滤，然后写入到数据库。
my.cnf慢日志配置：
slow_query_log_file = /database/mysql/mysql-slow.log slow_query_log = 1 log_slow_extra=1 long_query_time=0 log_slow_admin_statements=1 log_slow_slave_statements=1
pt-query-digest脚本：
#!/bin/bash ## ================================================================================== ## 让读书成为一种生活方式。就像吃喝拉撒每天必须要干的事， ## 终有一天你的举止、言谈、气质会不一样。 ## —- async ## ## Created Date: Sunday, 2020-05-16, 3:48:04 pm ## copyright (c): SZWW Tech. LTD. ## Engineer: async ## Module Name: ## Revision: v0.01 ## Description: ## ## Revision History : ## Revision editor date Description ## v0.01 async 2020-05-16 File Created ## ================================================================================== DIR="$( cd "$( dirname "$0" )" && pwd )" cd $DIR dt=`date --date='5 min ago' "+%Y-%m-%d %H:%M:%S"` #配置数据库的连接地址 monitor_db_host="172.18.1.xx" monitor_db_port=33601 monitor_db_user="xxxxx" monitor_db_password="xxxxx" monitor_db_database="xxxx" dba="xxxxx" #实例慢日志位置 slowquery_file=/database/mysql/mysql-slow.log pt_query_digest="/usr/bin/pt-query-digest" #实例连接信息 hostname="`hostname`" # 和实例配置内容保持一致，用于做筛选 #获取上次分析时间，初始化时请删除last_analysis_time_$hostname文件，可分析全部日志数据 if [ -s ddl_last_analysis_time_$hostname ]; then last_analysis_time=`cat ddl_last_analysis_time_$hostname` else last_analysis_time='1000-01-01 00:00:00' fi #收集日志 /usr/bin/cat $slowquery_file|grep -vE 'gtid_executed_v3|java' >/database/slow_log/mysql_slow.log && \ $pt_query_digest \ --user=$monitor_db_user --password=$monitor_db_password --port=$monitor_db_port \ --history h=$monitor_db_host,D=$monitor_db_database,t=review_history \ --no-report --limit=100% --charset=utf8mb4 \ --filter="\$event->{add_column}=length(\$event->{arg}) and \$event->{fingerprint}=~ m/^(create|alter|drop|rename|truncate|comment)/i \ and \$event->{client}=\$event->{ip}" /database/slow_log/mysql_slow.log > /tmp/analysis_slow_query.log if [ $? -ne 0 ]; then echo "failed"|/usr/bin/mailx -s "$0 failed on $(hostname) at $(date) " ${dba} else echo `date +"%Y-%m-%d %H:%M:%S"`> ddl_last_analysis_time_$hostname fi
我的日志里面有一些错误格式的日志，所以我把gtid_executed_v3这种过滤掉了，然后过滤出ddl ~ m/^(create|alter|drop|rename|truncate|comment)/i ，过滤dml为 ~ m/^(update|insert|delete)/i ，我主要过滤我关注的几类，然后写入表 review_history
review_history建表语句:
CREATE TABLE review_history ( id int(11) NOT NULL AUTO_INCREMENT, hostname_max varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs DEFAULT NULL, client_max varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs DEFAULT NULL, user_max varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs DEFAULT NULL, db_max varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs DEFAULT NULL, checksum char(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs DEFAULT NULL, sample longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs, ts_min timestamp NULL DEFAULT NULL, ts_max timestamp NULL DEFAULT NULL, ts_cnt float DEFAULT NULL, Query_time_sum float DEFAULT NULL, Query_time_min float DEFAULT NULL, Query_time_max float DEFAULT NULL, Query_time_pct_95 float DEFAULT NULL, Query_time_stddev float DEFAULT NULL, Query_time_median float DEFAULT NULL, Lock_time_sum float DEFAULT NULL, Lock_time_min float DEFAULT NULL, Lock_time_max float DEFAULT NULL, Lock_time_pct_95 float DEFAULT NULL, Lock_time_stddev float DEFAULT NULL, Lock_time_median float DEFAULT NULL, Rows_sent_sum float DEFAULT NULL, Rows_sent_min float DEFAULT NULL, Rows_sent_max float DEFAULT NULL, Rows_sent_pct_95 float DEFAULT NULL, Rows_sent_stddev float DEFAULT NULL, Rows_sent_median float DEFAULT NULL, Rows_examined_sum float DEFAULT NULL, Rows_examined_min float DEFAULT NULL, Rows_examined_max float DEFAULT NULL, Rows_examined_pct_95 float DEFAULT NULL, Rows_examined_stddev float DEFAULT NULL, Rows_examined_median float DEFAULT NULL, Rows_affected_sum float DEFAULT NULL, Rows_affected_min float DEFAULT NULL, Rows_affected_max float DEFAULT NULL, Rows_affected_pct_95 float DEFAULT NULL, Rows_affected_stddev float DEFAULT NULL, Rows_affected_median float DEFAULT NULL, Rows_read_sum float DEFAULT NULL, Rows_read_min float DEFAULT NULL, Rows_read_max float DEFAULT NULL, Rows_read_pct_95 float DEFAULT NULL, Rows_read_stddev float DEFAULT NULL, Rows_read_median float DEFAULT NULL, Merge_passes_sum float DEFAULT NULL, Merge_passes_min float DEFAULT NULL, Merge_passes_max float DEFAULT NULL, Merge_passes_pct_95 float DEFAULT NULL, Merge_passes_stddev float DEFAULT NULL, Merge_passes_median float DEFAULT NULL, InnoDB_IO_r_ops_min float DEFAULT NULL, InnoDB_IO_r_ops_max float DEFAULT NULL, InnoDB_IO_r_ops_pct_95 float DEFAULT NULL, InnoDB_IO_r_ops_stddev float DEFAULT NULL, InnoDB_IO_r_ops_median float DEFAULT NULL, InnoDB_IO_r_bytes_min float DEFAULT NULL, InnoDB_IO_r_bytes_max float DEFAULT NULL, InnoDB_IO_r_bytes_pct_95 float DEFAULT NULL, InnoDB_IO_r_bytes_stddev float DEFAULT NULL, InnoDB_IO_r_bytes_median float DEFAULT NULL, InnoDB_IO_r_wait_min float DEFAULT NULL, InnoDB_IO_r_wait_max float DEFAULT NULL, InnoDB_IO_r_wait_pct_95 float DEFAULT NULL, InnoDB_IO_r_wait_stddev float DEFAULT NULL, InnoDB_IO_r_wait_median float DEFAULT NULL, InnoDB_rec_lock_wait_min float DEFAULT NULL, InnoDB_rec_lock_wait_max float DEFAULT NULL, InnoDB_rec_lock_wait_pct_95 float DEFAULT NULL, InnoDB_rec_lock_wait_stddev float DEFAULT NULL, InnoDB_rec_lock_wait_median float DEFAULT NULL, InnoDB_queue_wait_min float DEFAULT NULL, InnoDB_queue_wait_max float DEFAULT NULL, InnoDB_queue_wait_pct_95 float DEFAULT NULL, InnoDB_queue_wait_stddev float DEFAULT NULL, InnoDB_queue_wait_median float DEFAULT NULL, InnoDB_pages_distinct_min float DEFAULT NULL, InnoDB_pages_distinct_max float DEFAULT NULL, InnoDB_pages_distinct_pct_95 float DEFAULT NULL, InnoDB_pages_distinct_stddev float DEFAULT NULL, InnoDB_pages_distinct_median float DEFAULT NULL, QC_Hit_cnt float DEFAULT NULL, QC_Hit_sum float DEFAULT NULL, Full_scan_cnt float DEFAULT NULL, Full_scan_sum float DEFAULT NULL, Full_join_cnt float DEFAULT NULL, Full_join_sum float DEFAULT NULL, Tmp_table_cnt float DEFAULT NULL, Tmp_table_sum float DEFAULT NULL, Tmp_table_on_disk_cnt float DEFAULT NULL, Tmp_table_on_disk_sum float DEFAULT NULL, Filesort_cnt float DEFAULT NULL, Filesort_sum float DEFAULT NULL, Filesort_on_disk_cnt float DEFAULT NULL, Filesort_on_disk_sum float DEFAULT NULL, Bytes_sum float DEFAULT NULL, Bytes_min float DEFAULT NULL, Bytes_max float DEFAULT NULL, Bytes_pct_95 float DEFAULT NULL, Bytes_stddev float DEFAULT NULL, Bytes_median float DEFAULT NULL, PRIMARY KEY (id), UNIQUE KEY checksum (checksum,ts_min,ts_max), KEY idx_hostname_max_ts_min (hostname_max,ts_min) ) ENGINE=InnoDB AUTO_INCREMENT=797 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_as_cs ROW_FORMAT=COMPRESSED
参考官方链接https://www.percona.com/doc/percona-toolkit/3.0/pt-query-digest.html
然后加入crontab，每个几分钟执行一次，后来发现这样会漏记录一些sql，频繁去解析slow_log对操作系统io也有一定消耗，思考以后，改成了每隔6小时全量执行一次，这样就会有一点延时，不过这个主要用于审计sql,延时也能接受，如果要实时，在想想办法。不增量以后，没再发现漏记sql。
主要就是pt的filter参数：
--filter="\$event->{add_column}=length(\$event->{arg}) and \$event->{hostname}=\"$hostname\" and \$event->{client}=\$event->{ip}" \
一开始是这样写的，记录的太多，查表筛选的时候慢，于是只过滤了自己需要的类型ddl与dml：
--filter="\$event->{add_column}=length(\$event->{arg}) and \$event->{fingerprint}=~ m/^(create|alter|drop|rename|truncate|comment|delete|insert|update)/i \ and \$event->{client}=\$event->{ip}"
然后修改下脚本就实现了mysql的sql审计，然后再配合init-connect，可以实现比较精准快速的排查问题。