数据库难免会被误操作，比如delete,drop等有害的语句被执行后，该如何恢复对dba是一个考验，如果备份可用的话，可以从备份中恢复数据来，不过这样做一般的成本有些大。
binlog2sql工具是大众点评团队用python工具写的一个工具，通过解析binlog的sql，将sql语句反解析达到恢复数据的目的。如果没有这个工具，当然我们可以通过复杂的sed命令来手动生成恢复的语句，不过binglog2sql将这一步操作做了简便处理。
这篇blog是通过手动生成恢复语句：http://www.cnblogs.com/gomysql/p/3582058.html
binlog2sql官方文档：https://github.com/danfengcao/binlog2sql
下面进行安装和试用：
1.先配置好yum后安装git,yum的配置参考：https://www.cnblogs.com/yizhichun/p/6339742.html
2.安装binlogs2sql之前的准备工作
首先需要安装好git和pip工具
git的安装比较简单，网上有很多资料
[root@qht131 yum.repos.d]# yum install git-core
不过在安装pip时出现了各种问题，好在最后安装成功了。
官方安装文档：https://pip.pypa.io/en/stable/installing/
安装pip之前需要先升级python文档：https://blog.csdn.net/see_you_see_me/article/details/78550977
如果安装过程中出错了处理方法参考：http://www.zhimengzhe.com/linux/251710.html
[root@qht131 binlog2sql]# python -V Python 2.7.14
[root@qht131 binlog2sql]# git --version git version 1.7.1
[root@qht131 binlog2sql]# pip -V pip 10.0.1 from /usr/local/python2.7/lib/python2.7/site-packages/pip (python 2.7)
3.安装binglog2sql工具
shell> git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql
我在用git下载原文件的时候出错，通过安装curl及curl-devel来解决的
yum install curl curl-devel
[root@qht131 binlog2sql]# pip install -r requirements.txt
4.设置参数及帐户权限
[root@qht131 binlog2sql]# cat /etc/my.cnf ... server-id=10000 log_bin = /u01/mysql/mysql_bin max_binlog_size=1G binlog_format = row binlog_row_image = full
mysql> create user btwos@'localhost' identified by '123456'; Query OK, 0 rows affected (0.06 sec) mysql> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* to btwos@localhost; Query OK, 0 rows affected (0.00 sec)
准备测试数据
mysql> create table test (
-> id int not null auto_increment,
-> name varchar(20) not null,
-> create_time datetime not null,
-> primary key (id));
Query OK, 0 rows affected (0.07 sec)
mysql> insert into test(name,create_time) values ('Xiaolin','2015-10-1'),('WangMac','2016-08-25'),('Jackli','2017-04-10');
Query OK, 3 rows affected (0.21 sec)
Records: 3 Duplicates: 0 Warnings: 0
mysql> select * from test;
+----+---------+---------------------+
| id | name | create_time |
+----+---------+---------------------+
| 1 | Xiaolin | 2015-10-01 00:00:00 |
| 2 | WangMac | 2016-08-25 00:00:00 |
| 3 | Jackli | 2017-04-10 00:00:00 |
+----+---------+---------------------+
3 rows in set (0.02 sec)
5.测试delete以及update操作
mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql_bin.000040 | 1212 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.01 sec)
mysql> update test set create_time='2016-10-10' where name='WangMac'; Query OK, 1 row affected (0.03 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> delete from test where name='Jackli'; Query OK, 1 row affected (0.00 sec) mysql> commit; Query OK, 0 rows affected (0.00 sec) mysql> select * from test; +----+---------+---------------------+ | id | name | create_time | +----+---------+---------------------+ | 1 | Xiaolin | 2015-10-01 00:00:00 | | 2 | WangMac | 2016-10-10 00:00:00 | +----+---------+---------------------+ 2 rows in set (0.00 sec)
mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql_bin.000040 | 1772 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
修改了一笔数据，以及删除了最后一笔数据。
先查看一下mysqlbinlog解析出来的日志，从日志可以看到从建表到update和delete的所有操作。
[root@qht131 mysql]# mysqlbinlog --no-defaults -v -v mysql_bin.000040 > 000040.log [root@qht131 mysql]# cat 000040.log # at 706 #180426 11:01:35 server id 10000 end_log_pos 906 CRC32 0x6f60578e Query thread_id=3 exec_time=0 error_code=0 use `l5m`/*!*/; SET TIMESTAMP=1524711695/*!*/; create table test ( id int not null auto_increment, name varchar(20) not null, create_time datetime not null, primary key (id)) /*!*/; # at 906 #180426 11:03:43 server id 10000 end_log_pos 971 CRC32 0x91159a2a Anonymous_GTIDlast_committed=3 sequence_number=4 rbr_only=yes /*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 971 #180426 11:03:43 server id 10000 end_log_pos 1042 CRC32 0xb26843e4 Query thread_id=3 exec_time=0 error_code=0 SET TIMESTAMP=1524711823/*!*/; BEGIN /*!*/; # at 1042 #180426 11:03:43 server id 10000 end_log_pos 1093 CRC32 0x72c273fe Table_map: `l5m`.`test` mapped to number 108 # at 1093 #180426 11:03:43 server id 10000 end_log_pos 1181 CRC32 0xb0a6183b Write_rows: table id 108 flags: STMT_END_F BINLOG ' j0HhWhMQJwAAMwAAAEUEAAAAAGwAAAAAAAEAA2w1bQAEdGVzdAADAw8SAzwAAAD+c8Jy j0HhWh4QJwAAWAAAAJ0EAAAAAGwAAAAAAAEAAgAD//gBAAAAB1hpYW9saW6Zl0IAAPgCAAAAB1dh bmdNYWOZmjIAAPgDAAAABkphY2tsaZmcVAAAOximsA== '/*!*/; ### INSERT INTO `l5m`.`test` ### SET ### @1=1 /* INT meta=0 nullable=0 is_null=0 */ ### @2='Xiaolin' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2015-10-01 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */ ### INSERT INTO `l5m`.`test` ### SET ### @1=2 /* INT meta=0 nullable=0 is_null=0 */ ### @2='WangMac' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2016-08-25 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */ ### INSERT INTO `l5m`.`test` ### SET ### @1=3 /* INT meta=0 nullable=0 is_null=0 */ ### @2='Jackli' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2017-04-10 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */ # at 1181 #180426 11:03:43 server id 10000 end_log_pos 1212 CRC32 0x9e3baa9e Xid = 21 COMMIT/*!*/; # at 1212 #180426 11:18:57 server id 10000 end_log_pos 1277 CRC32 0x7d82517c Anonymous_GTIDlast_committed=4 sequence_number=5 rbr_only=yes /*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 1277 #180426 11:18:57 server id 10000 end_log_pos 1348 CRC32 0x0ebea2ac Query thread_id=3 exec_time=0 error_code=0 SET TIMESTAMP=1524712737/*!*/; BEGIN /*!*/; # at 1348 #180426 11:18:57 server id 10000 end_log_pos 1399 CRC32 0x86e375ff Table_map: `l5m`.`test` mapped to number 108 # at 1399 #180426 11:18:57 server id 10000 end_log_pos 1471 CRC32 0xfb2db159 Update_rows: table id 108 flags: STMT_END_F BINLOG ' IUXhWhMQJwAAMwAAAHcFAAAAAGwAAAAAAAEAA2w1bQAEdGVzdAADAw8SAzwAAAD/deOG IUXhWh8QJwAASAAAAL8FAAAAAGwAAAAAAAEAAgAD///4AgAAAAdXYW5nTWFjmZoyAAD4AgAAAAdX YW5nTWFjmZqUAABZsS37 '/*!*/; ### UPDATE `l5m`.`test` ### WHERE ### @1=2 /* INT meta=0 nullable=0 is_null=0 */ ### @2='WangMac' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2016-08-25 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */ ### SET ### @1=2 /* INT meta=0 nullable=0 is_null=0 */ ### @2='WangMac' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2016-10-10 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */ # at 1471 #180426 11:18:57 server id 10000 end_log_pos 1502 CRC32 0x0f213682 Xid = 25 COMMIT/*!*/; # at 1502 #180426 11:19:11 server id 10000 end_log_pos 1567 CRC32 0x15dd429a Anonymous_GTIDlast_committed=5 sequence_number=6 rbr_only=yes /*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 1567 #180426 11:19:11 server id 10000 end_log_pos 1638 CRC32 0x926675d7 Query thread_id=3 exec_time=0 error_code=0 SET TIMESTAMP=1524712751/*!*/; BEGIN /*!*/; # at 1638 #180426 11:19:11 server id 10000 end_log_pos 1689 CRC32 0x83a25989 Table_map: `l5m`.`test` mapped to number 108 # at 1689 #180426 11:19:11 server id 10000 end_log_pos 1741 CRC32 0xbb2e56b2 Delete_rows: table id 108 flags: STMT_END_F BINLOG ' L0XhWhMQJwAAMwAAAJkGAAAAAGwAAAAAAAEAA2w1bQAEdGVzdAADAw8SAzwAAACJWaKD L0XhWiAQJwAANAAAAM0GAAAAAGwAAAAAAAEAAgAD//gDAAAABkphY2tsaZmcVAAAslYuuw== '/*!*/; ### DELETE FROM `l5m`.`test` ### WHERE ### @1=3 /* INT meta=0 nullable=0 is_null=0 */ ### @2='Jackli' /* VARSTRING(60) meta=60 nullable=0 is_null=0 */ ### @3='2017-04-10 00:00:00' /* DATETIME(0) meta=0 nullable=0 is_null=0 */
下面使用binlog2sql来解析：
[root@qht131 binlog2sql]# python binlog2sql.py -h127.0.0.1 -P3306 -ubtwos -p '123456' -dl5m -ttest --start-file='mysql_bin.000040'
CREATE USER 'btwos'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'btwos'@'localhost';
USE l5m;
create table test (
id int not null auto_increment,
name varchar(20) not null,
create_time datetime not null,
primary key (id));
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2015-10-01 00:00:00', 1, 'Xiaolin'); #start 906 end 1181 time 2018-04-26 11:03:43
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2016-08-25 00:00:00', 2, 'WangMac'); #start 906 end 1181 time 2018-04-26 11:03:43
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2017-04-10 00:00:00', 3, 'Jackli'); #start 906 end 1181 time 2018-04-26 11:03:43
UPDATE `l5m`.`test` SET `create_time`='2016-10-10 00:00:00', `id`=2, `name`='WangMac' WHERE `create_time`='2016-08-25 00:00:00' AND `id`=2 AND `name`='WangMac' LIMIT 1; #start 1212 end 1471 time 2018-04-26 11:18:57
DELETE FROM `l5m`.`test` WHERE `create_time`='2017-04-10 00:00:00' AND `id`=3 AND `name`='Jackli' LIMIT 1; #start 1502 end 1741 time 2018-04-26 11:19:11
完美呈现了所有以test的操作，并且还备注了在binlog中的position以及执行的时间
下面重点来了，解析出回滚的sql
[root@qht131 binlog2sql]# python binlog2sql.py --flashback -h127.0.0.1 -P3306 -ubtwos -p '123456' -dl5m -ttest --start-file='mysql_bin.000040'
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2017-04-10 00:00:00', 3, 'Jackli'); #start 1502 end 1741 time 2018-04-26 11:19:11
UPDATE `l5m`.`test` SET `create_time`='2016-08-25 00:00:00', `id`=2, `name`='WangMac' WHERE `create_time`='2016-10-10 00:00:00' AND `id`=2 AND `name`='WangMac' LIMIT 1; #start 1212 end 1471 time 2018-04-26 11:18:57
DELETE FROM `l5m`.`test` WHERE `create_time`='2017-04-10 00:00:00' AND `id`=3 AND `name`='Jackli' LIMIT 1; #start 906 end 1181 time 2018-04-26 11:03:43
DELETE FROM `l5m`.`test` WHERE `create_time`='2016-08-25 00:00:00' AND `id`=2 AND `name`='WangMac' LIMIT 1; #start 906 end 1181 time 2018-04-26 11:03:43
DELETE FROM `l5m`.`test` WHERE `create_time`='2015-10-01 00:00:00' AND `id`=1 AND `name`='Xiaolin' LIMIT 1; #start 906 end 1181 time 2018-04-26 11:03:43
解析出来的sql是根据时间戳倒序的，delete语句被flashback成了insert语句，update语句也可以成功将数据恢复到之前的状态，insert语句被flashback成了delete语句。
如果需要恢复数据的话只需要直接执行这些语句就可以了。
6.测试delete,truncate以及drop操作
mysql> flush logs; Query OK, 0 rows affected (0.01 sec) mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql_bin.000041 | 154 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
mysql> use l5m
Database changed
mysql> delete from test where id<3;
Query OK, 2 rows affected (0.03 sec)
mysql> select * from test;
Empty set (0.00 sec)
mysql> insert into test (name,create_time) values('ChunkA','2019-10-08');
Query OK, 1 row affected (0.00 sec)
mysql> select * from test;
+----+--------+---------------------+
| id | name | create_time |
+----+--------+---------------------+
| 4 | ChunkA | 2019-10-08 00:00:00 |
+----+--------+---------------------+
1 row in set (0.00 sec)
mysql> truncate table test;
Query OK, 0 rows affected (0.01 sec)
mysql> select * from test;
Empty set (0.00 sec)
mysql> drop table test;
Query OK, 0 rows affected (0.02 sec)
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql_bin.000041 | 1043 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
直接用binlog2sql来解析binlog
[root@qht131 binlog2sql]# python binlog2sql.py -h127.0.0.1 -P3306 -ubtwos -p '123456' -dl5m -ttest --start-file='mysql_bin.000041' USE l5m; truncate table test; USE l5m; DROP TABLE `test` /* generated by server */;
发现解析不出来truncate之前delete和insert操作，应该是由于information_schema.tables已没有test表的信息，导致解析不出来。
那如果我重新把test表建立好之后再用binlog2sql来解析行不行？
mysql> create table test (
-> id int not null auto_increment,
-> name varchar(20) not null,
-> create_time datetime not null,
-> primary key (id));
Query OK, 0 rows affected (0.03 sec)
[root@qht131 binlog2sql]# python binlog2sql.py -h127.0.0.1 -P3306 -ubtwos -p '123456' -dl5m -ttest --start-file='mysql_bin.000041'
DELETE FROM `l5m`.`test` WHERE `create_time`='2015-10-01 00:00:00' AND `id`=1 AND `name`='Xiaolin' LIMIT 1; #start 4 end 412 time 2018-04-26 11:55:27
DELETE FROM `l5m`.`test` WHERE `create_time`='2016-10-10 00:00:00' AND `id`=2 AND `name`='WangMac' LIMIT 1; #start 4 end 412 time 2018-04-26 11:55:27
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2019-10-08 00:00:00', 4, 'ChunkA'); #start 443 end 682 time 2018-04-26 11:56:10
USE l5m;
truncate table test;
USE l5m;
DROP TABLE `test` /* generated by server */;
USE l5m;
create table test (
id int not null auto_increment,
name varchar(20) not null,
create_time datetime not null,
primary key (id));
没有问题，binlog2sql倒序着把binlog的所有操作都解析出来了。
接着就好flashback了生成回滚sql了：
[root@qht131 binlog2sql]# python binlog2sql.py --flashback -h127.0.0.1 -P3306 -ubtwos -p '123456' -dl5m -ttest --start-file='mysql_bin.000041'
DELETE FROM `l5m`.`test` WHERE `create_time`='2019-10-08 00:00:00' AND `id`=4 AND `name`='ChunkA' LIMIT 1; #start 443 end 682 time 2018-04-26 11:56:10
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2016-10-10 00:00:00', 2, 'WangMac'); #start 4 end 412 time 2018-04-26 11:55:27
INSERT INTO `l5m`.`test`(`create_time`, `id`, `name`) VALUES ('2015-10-01 00:00:00', 1, 'Xiaolin'); #start 4 end 412 time 2018-04-26 11:55:27
可以看到，此时批量 delete 和 insert 的语句被解析出来了，并且能够成功生成回滚的 SQL。
但是，DDL 语句，在整个测试过程中都是无法被回滚的，这也在我们的预期中，至此，MySQL 闪回工具 -- binlog2sql 的基本测试就告一段落了。
参考：https://www.cnblogs.com/glon/p/6856192.html
https://github.com/danfengcao/binlog2sql