0.环境
CentOS Linux release 7.6.1810 (Core)
mysql Ver 8.0.16
python 3.8.1 （下面步骤安装）
pymysql 0.9.3 （下面步骤安装）
1. Binlog2sql 安装
1.1 安装python
1.依赖包安装 yum install -y wget zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc libffi-devel 2.安装python3.8.1（20191230最新稳定版） wget https://www.python.org/ftp/python/3.8.1/Python-3.8.1.tgz tar -xzvf Python-3.8.1.tgz cd Python-3.8.1 ./configure --prefix=/etc/python3.8 --enable-optimizations make && make install [root@mysql1 python3.8]# ll /usr/bin/python* lrwxrwxrwx. 1 root root 7 Jun 3 2019 /usr/bin/python -> python2 lrwxrwxrwx. 1 root root 9 Jun 3 2019 /usr/bin/python2 -> python2.7 -rwxr-xr-x. 1 root root 7216 Oct 31 2018 /usr/bin/python2.7 [root@mysql1 python3.8]# unlink /usr/bin/python [root@mysql1 python3.8]# ln -s /etc/python3.8/bin/python3.8 /usr/bin/python [root@mysql1 python3.8]# ll /usr/bin/python* lrwxrwxrwx 1 root root 28 Dec 30 10:22 /usr/bin/python -> /etc/python3.8/bin/python3.8 lrwxrwxrwx. 1 root root 9 Jun 3 2019 /usr/bin/python2 -> python2.7 -rwxr-xr-x. 1 root root 7216 Oct 31 2018 /usr/bin/python2.7 [root@mysql1 python3.8]# python -V Python 3.8.1
1.1.1 yum修复
问题
因为yum需要使用python2，将/usr/bin/python改为python3后，yum就不能正常运行了,报错如下
[root@mysql1 python3.8]# yum list File "/usr/bin/yum", line 30 except KeyboardInterrupt, e: ^ SyntaxError: invalid syntax
修复
sed -i '1s/usr\/bin\/python/usr\/bin\/python2/' /usr/bin/yum sed -i '1s/usr\/bin\/python/usr\/bin\/python2/' /usr/libexec/urlgrabber-ext-down
1.2 安装Python-pip
yum -y install epel-release yum -y install python-pip pip install --upgrade pip [root@mysql4 Python-3.8.1]# pip --version pip 19.3.1 from /usr/lib/python2.7/site-packages/pip (python 2.7) 修改pip 使用3.8的python python -m ensurepip python -m pip install --upgrade pip sed -i '1s/usr\/bin\/python2/usr\/bin\/python/' /usr/bin/pip [root@mysql4 Python-3.8.1]# pip --version pip 19.3.1 from /etc/python3.8/lib/python3.8/site-packages/pip (python 3.8)
1.3 安装pymysql 模块
[root@mysql4 Python-3.8.1]# pip install PyMySQL [root@mysql4 Python-3.8.1]# pip install --upgrade PyMySQL (不升级会遇到报错 KeyError: 255 ) Collecting PyMySQL Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB) |████████████████████████████████| 51kB 402kB/s Installing collected packages: PyMySQL Successfully installed PyMySQL-0.9.3 [root@mysql4 Python-3.8.1]# pip show pymysql Name: PyMySQL Version: 0.9.3 Summary: Pure Python MySQL Driver Home-page: https://github.com/PyMySQL/PyMySQL/ Author: yutaka.matsubara Author-email: [email protected] License: "MIT" Location: /etc/python3.8/lib/python3.8/site-packages Requires: Required-by: [root@mysql4 Python-3.8.1]# pip list Package Version ---------- ------- pip 19.3.1 PyMySQL 0.9.3 setuptools 41.2.0
1.4 安装binlog2sql
yum install -y git git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql pip install -r requirements.txt
以上完成了binlog2Mysql 的安装。
2. Binlog2Mysql 使用测试
2.1 检查数据库参数
是否包含如下参数，没有则添加
[mysqld] server_id=1 log_bin = binlog.index max_binlog_size = 1G binlog_format = row binlog_row_image = full (默认,保存了变更前和变更后的所有列镜像）
重启数据库使生效
systemctl stop mysqld
systemctl stop mysqld
2.2 创建测试用例
创建测试库cymdb, 测试用户cym, 测试表cymtable
--1.创建测试库 cymdb create database cymdb; --2. 创建测试用户 create user cym identified by "Hello3306"; GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO cym; --3.创建测试表 use cymdb; create table cymtable(id int,name varchar(10),addtime datetime default now()); insert into cymtable values(1,'赵',now()),(2,'钱',now()),(3,'孙','2022-01-12 12:12:12'),(4,'李','2000-12-12 1:00:00'); mysql> select * from cymtable; +------+------+---------------------+ | id | name | addtime | +------+------+---------------------+ | 1 | 赵 | 2019-12-30 13:18:20 | | 2 | 钱 | 2019-12-30 13:18:20 | | 3 | 孙 | 2022-01-12 12:12:12 | | 4 | 李 | 2000-12-12 01:00:00 | +------+------+---------------------+ 4 rows in set (0.00 sec) mysql> select now(); +---------------------+ | now() | +---------------------+ | 2019-12-30 13:18:39 | +---------------------+ 1 row in set (0.00 sec) --4.删除数据 mysql> delete from cymtable; Query OK, 4 rows affected (0.00 sec) mysql> select * from cymtable; Empty set (0.00 sec) mysql> select now(); +---------------------+ | now() | +---------------------+ | 2019-12-30 13:21:15 | +---------------------+ 1 row in set (0.00 sec)
以上案例：
数据正常时间：2019-12-30 13:18:39
数据丢失时间：2019-12-30 13:21:15
面临数据丢失，我们要如何恢复？
2.3 通过binlog2mysql 获取 Redo SQL 和 Undo SQL
2.3.1 获取binlog 位置
mysql> show variables like 'log_bin%'; +---------------------------------+------------------------------------+ | Variable_name | Value | +---------------------------------+------------------------------------+ | log_bin | ON | | log_bin_basename | /u01/app/mysql/mydata/binlog | | log_bin_index | /u01/app/mysql/mydata/binlog.index | | log_bin_trust_function_creators | OFF | | log_bin_use_v1_row_events | OFF | +---------------------------------+------------------------------------+ 5 rows in set (0.00 sec)
2.3.2 检查binlog 的最后修改时间，获取需要的binlog
[root@mysql4 mydata]# ls -ltrh /u01/app/mysql/mydata/binlog* -rw-r----- 1 mysql mysql 178 Dec 27 14:31 /u01/app/mysql/mydata/binlog.000001 -rw-r----- 1 mysql mysql 1.2K Dec 27 14:37 /u01/app/mysql/mydata/binlog.000002 -rw-r----- 1 mysql mysql 521 Dec 27 17:02 /u01/app/mysql/mydata/binlog.000003 -rw-r----- 1 mysql mysql 218 Dec 27 23:18 /u01/app/mysql/mydata/binlog.000004 -rw-r----- 1 mysql mysql 218 Dec 27 23:20 /u01/app/mysql/mydata/binlog.000005 -rw-r----- 1 mysql mysql 218 Dec 27 23:21 /u01/app/mysql/mydata/binlog.000006 -rw-r----- 1 mysql mysql 218 Dec 30 12:38 /u01/app/mysql/mydata/binlog.000007 -rw-r----- 1 mysql mysql 128 Dec 30 12:39 /u01/app/mysql/mydata/binlog.index -rw-r----- 1 mysql mysql 1.9K Dec 30 13:21 /u01/app/mysql/mydata/binlog.000008
已知数据丢失时间为 2019-12-30 13:18:39 ~ 2019-12-30 13:21:15
/u01/app/mysql/mydata/binlog.000008 的时间周期是 Dec 30 12:38 ~ Dec 30 13:21 是我们需要的binlog。
2.3.3 从binlog中解析出Redo SQL 和Undo SQL
2.3.3.1 解析 Redo SQL
cd binlog2sql/
[root@mysql4 binlog2sql]# python binlog2sql/binlog2sql.py -h127.0.0.1 -P3306 -ucym -pHello3306 -dcymdb -t cymtable --start-file='binlog.000008' --start-datetime='2019-12-30 13:18:39' --stop-datetime='2019-12-30 13:21:15' DELETE FROM `cymdb`.`cymtable` WHERE `addtime`=1 AND `id`='赵' AND `name`='2019-12-30 13:18:20' LIMIT 1; #start 1585 end 1817 time 2019-12-30 13:21:06 DELETE FROM `cymdb`.`cymtable` WHERE `addtime`=2 AND `id`='钱' AND `name`='2019-12-30 13:18:20' LIMIT 1; #start 1585 end 1817 time 2019-12-30 13:21:06 DELETE FROM `cymdb`.`cymtable` WHERE `addtime`=3 AND `id`='孙' AND `name`='2022-01-12 12:12:12' LIMIT 1; #start 1585 end 1817 time 2019-12-30 13:21:06 DELETE FROM `cymdb`.`cymtable` WHERE `addtime`=4 AND `id`='李' AND `name`='2000-12-12 01:00:00' LIMIT 1; #start 1585 end 1817 time 2019-12-30 13:21:06
这里看到对应时间段的cymtable表的 RedoSQL 已经被解析出来，但是注意 `addtime`=1 AND `id`='赵' AND `name`='2019-12-30 13:18:20' 顺序是错误的。
2.3.3.2 解析 Undo SQL
[root@mysql4 binlog2sql]# python binlog2sql/binlog2sql.py --flashback -h127.0.0.1 -P3306 -ucym -pHello3306 -dcymdb -t cymtable --start-file='binlog.000008' --start-datetime='2019-12-30 13:18:39' --stop-datetime='2019-12-30 13:21:15' INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (4, '李', '2000-12-12 01:00:00'); #start 1585 end 1817 time 2019-12-30 13:21:06 INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (3, '孙', '2022-01-12 12:12:12'); #start 1585 end 1817 time 2019-12-30 13:21:06 INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (2, '钱', '2019-12-30 13:18:20'); #start 1585 end 1817 time 2019-12-30 13:21:06 INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (1, '赵', '2019-12-30 13:18:20'); #start 1585 end 1817 time 2019-12-30 13:21:06
顺序同样是错的,下面使用Linux 命令修复
[root@mysql4 binlog2sql]# python binlog2sql/binlog2sql.py --flashback -h127.0.0.1 -P3306 -ucym -pHello3306 -dcymdb -t cymtable --start-file='binlog.000008' --start-datetime='2019-12-30 13:18:39' --stop-datetime='2019-12-30 13:21:15'| cut -d '#' -f1 >undo.sql [root@mysql4 binlog2sql]# more undo.sql INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (4, '李', '2000-12-12 01:00:00'); INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (3, '孙', '2022-01-12 12:12:12'); INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (2, '钱', '2019-12-30 13:18:20'); INSERT INTO `cymdb`.`cymtable`(`addtime`, `id`, `name`) VALUES (1, '赵', '2019-12-30 13:18:20'); [root@mysql4 binlog2sql]# sed -i 's/addtime/vcol1/g' undo.sql [root@mysql4 binlog2sql]# sed -i 's/name/addtime/g' undo.sql [root@mysql4 binlog2sql]# sed -i 's/id/name/g' undo.sql [root@mysql4 binlog2sql]# sed -i 's/vcol1/id/g' undo.sql [root@mysql4 binlog2sql]# more undo.sql INSERT INTO `cymdb`.`cymtable`(`id`, `name`, `addtime`) VALUES (4, '李', '2000-12-12 01:00:00'); INSERT INTO `cymdb`.`cymtable`(`id`, `name`, `addtime`) VALUES (3, '孙', '2022-01-12 12:12:12'); INSERT INTO `cymdb`.`cymtable`(`id`, `name`, `addtime`) VALUES (2, '钱', '2019-12-30 13:18:20'); INSERT INTO `cymdb`.`cymtable`(`id`, `name`, `addtime`) VALUES (1, '赵', '2019-12-30 13:18:20'); #修复成功
2.3.3.3 数据库应用Undo、
[root@mysql4 binlog2sql]# mysql -uroot -p123456 cymdb <undo.sql mysql: [Warning] Using a password on the command line interface can be insecure. [root@mysql4 binlog2sql]# mysql -uroot -p123456 cymdb -e 'select * from cymtable' mysql: [Warning] Using a password on the command line interface can be insecure. +------+------+---------------------+ | id | name | addtime | +------+------+---------------------+ | 4 | 李 | 2000-12-12 01:00:00 | | 3 | 孙 | 2022-01-12 12:12:12 | | 2 | 钱 | 2019-12-30 13:18:20 | | 1 | 赵 | 2019-12-30 13:18:20 | +------+------+---------------------+
至此完成了mysql 误删除数据的恢复。
以上操作是在联网环境下完成的，安装过程中遇到的错误还好都解决了。如果是离线环境估计又有很多问题需要处理。