Foreword

"Delete library on foot" programmers often talk about the topic, today, I'm going to teach you how to delete! Library! run! road!

Joke, the subject of today's article is how to use the built-in Binlog Mysql log on to recover mistakenly deleted data, reading this article, you can learn:

What is MySQL binlog log? Usually they used to do?
A simulated operation mistakenly deleted the data, and using binlog log restoring accidentally deleted data.

The original intention of writing this article, there is a time I really almost to a test database table to delete, then cold sweats. The reason is due to the Spring JPA configuration, there is a spring.jpa.properties.hibernate.hbm2ddl.auto=create-drop, and its purpose is 每次加载hibernate时根据model类生成表，但是sessionFactory一关闭,表就自动删除。, this configuration can not just go up, put your direct-existing table to drop it!

Well, back to the topic, this article is to want to reassure everybody , MySQL even if the operation were mistakenly deleted, they can basically salvaged. Especially within large companies, you want to delete the data can not be deleted, there are countless privileges / backup stop you.

text

Binlog Introduction

binlog database table is a record of all structural changes (e.g., CREATE, ALTER TABLE ...) and table data modifications (INSERT, UPDATE, DELETE ...) binary log.
binlog not record SELECT and SHOW this type of operation, because such operations on the data itself is not modified, but you can view all statements that MySQL executed by querying the general log.

Read binlog defined above, we should be able to infer binlog roughly three purposes:

Data recovery: the focus of today say
Database replication: master-slave database is passed through the binlog from the library, from the library there are two threads, one I / O thread, a SQL thread, I / O thread reads the main library binlog pass over the content and write to the relay log, SQL thread reads the relay log contents from inside, into the database from the library.
Audit: Users can be audited by the binary log information, determine whether the database injection attacks.

所以说，想要能够恢复数据，首先，你得打开Mysql的binlog，在平常你自己安装的单机Mysql中，默认情况下不会开启。下面就一步步地实践下如何开启你服务器上的Binlog日志。

在MySQL中开启Binlog

首先进入数据库控制台，运行指令：

mysql> show variables like'log_bin%';
+---------------------------------+-------+
| Variable_name                   | Value |
+---------------------------------+-------+
| log_bin                         | OFF   |
| log_bin_basename                |       |
| log_bin_index                   |       |
| log_bin_trust_function_creators | OFF   |
| log_bin_use_v1_row_events       | OFF   |
+---------------------------------+-------+
5 rows in set (0.00 sec)

可以看到我们的binlog是关闭的，都是OFF。接下来我们需要修改Mysql配置文件，执行命令：

sudo vi /etc/mysql/mysql.conf.d/mysqld.cnf

在文件末尾添加：

log-bin=/var/lib/mysql/mysql-bin

保存文件，重启mysql服务：

sudo service mysql restart

重启完成后，查看下mysql的状态：

systemctl status mysql.service

这时，如果你的mysql版本在5.7或更高版本，就会报错：

Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.190791Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.190839Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.359713Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (se
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.361395Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.28-0ubuntu0.16.04.2-log) starting as process 5930 ...
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.363017Z 0 [ERROR] You have enabled the binary log, but you haven't provided the mandatory server-id. Please refer to the proper server
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.363747Z 0 [ERROR] Aborting
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.363922Z 0 [Note] Binlog end
Jan 06 15:49:58 VM-0-11-ubuntu mysqld[5930]: 2020-01-06T07:49:58.364108Z 0 [Note] /usr/sbin/mysqld: Shutdown complete
Jan 06 15:49:58 VM-0-11-ubuntu systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE

You have enabled the binary log, but you haven’t provided the mandatory server-id. Please refer to the proper server

之前我们的配置，对于5.7以下版本应该是可以的。但对于高版本，我们需要指定server-id。

如果你不是分布式的部署Mysql，这个server-id随机给个数字就可以。

server-id=123454

模拟删除数据并恢复

首先新建数据库mytest，新建一张表table1，结构见下方SQL代码

CREATE DATABASE `test` ;

USE `test`;

DROP TABLE IF EXISTS `table1`;

CREATE TABLE `table2` (
  `id` int(11) DEFAULT NULL,
  `name` varchar(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

插入两条数据，分别是 (1,‘A’),(2,‘B’)

INSERT INTO `table1` VALUES (1,'A'),(2,'B');

我们看一下binlog日志的状态，使用show master status

mysql> show master status
    -> ;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      690 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

binlog日志特征：每当我们重启MySQL一次，会自动生成一个binlog文件，当然，我们也可以手动的来刷新binlog文件，通过 flush logs,同样会新创建一个binlog文件。实际上当服务器在重启时，也会调用flush logs操作。

上图代码中可以看到，现在我们正在使用 mysql-bin.0000001 ，并且这个文件现在正在记录到690行。

然后，使用flush logs来主动刷新一次binlog

mysql> flush logs;
Query OK, 0 rows affected

mysql> show master status
    -> ;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000002 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

可以看到，现在日志文件在 mysql-bin.000002 文件中，位置为154。也就是我们主动刷新了一次binlog，生成了新的000002，而000001则已经归档了，不会再写入新的日志进去了。

接下来我们在插入两条数据

insert into table1 values (3,'C');
insert into table1 values (4,'D');

mysql> select * from table1;
+----+----+
| id |name|
+----+----+
|  1 | A  |
|  2 | B  |
|  3 | C  |
|  4 | D  |
+----+----+

这时候我们已经有了四条数据，我们再次flush logs，把mysql-bin.000002日志存档，开启新的mysql-bin.000003日志，这样，每次我们插入的数据彼此独立。实际情况下，binlog会比较复杂，这里也是做了简化，为了理解更方便。

mysql> flush logs;
Query OK, 0 rows affected

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000003 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

然后我们删除id为4的数据（4，D），并且再次刷新binlog，如此一来，binlog.000003里面只有一条删除操作。

mysql> delete from table1 where id = 4;
Query OK, 1 row affected

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000003 |      423 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

mysql> flush logs;
Query OK, 0 rows affected

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000004 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

让我们来好好观察下mysql-bin.00002和mysql-bin00003两个binlog，使用命令：show binlog events in 'mysql-bin.000003'

mysql> show binlog events in 'mysql-bin.000003';
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
| Log_name         | Pos | Event_type     | Server_id | End_log_pos | Info                                                   |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
| mysql-bin.000003 |   4 | Format_desc    |    123456 |         123 | Server ver: 5.7.28-0ubuntu0.16.04.2-log, Binlog ver: 4 |
| mysql-bin.000003 | 123 | Previous_gtids |    123456 |         154 |                                                        |
| mysql-bin.000003 | 154 | Anonymous_Gtid |    123456 |         219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS'                   |
| mysql-bin.000003 | 219 | Query          |    123456 |         293 | BEGIN                                                  |
| mysql-bin.000003 | 293 | Table_map      |    123456 |         343 | table_id: 108 (test.table1)                              |
| mysql-bin.000003 | 343 | Delete_rows    |    123456 |         392 | table_id: 108 flags: STMT_END_F                        |
| mysql-bin.000003 | 392 | Xid            |    123456 |         423 | COMMIT /* xid=39 */                                    |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
7 rows in set

mysql> show binlog events in 'mysql-bin.000002';
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
| Log_name         | Pos | Event_type     | Server_id | End_log_pos | Info                                                   |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
| mysql-bin.000002 |   4 | Format_desc    |    123456 |         123 | Server ver: 5.7.28-0ubuntu0.16.04.2-log, Binlog ver: 4 |
| mysql-bin.000002 | 123 | Previous_gtids |    123456 |         154 |                                                        |
| mysql-bin.000002 | 154 | Anonymous_Gtid |    123456 |         219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS'                   |
| mysql-bin.000002 | 219 | Query          |    123456 |         293 | BEGIN                                                  |
| mysql-bin.000002 | 293 | Table_map      |    123456 |         343 | table_id: 108 (test.table1)                              |
| mysql-bin.000002 | 343 | Write_rows     |    123456 |         390 | table_id: 108 flags: STMT_END_F                        |
| mysql-bin.000002 | 390 | Xid            |    123456 |         421 | COMMIT /* xid=34 */                                    |
| mysql-bin.000002 | 421 | Anonymous_Gtid |    123456 |         486 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS'                   |
| mysql-bin.000002 | 486 | Query          |    123456 |         560 | BEGIN                                                  |
| mysql-bin.000002 | 560 | Table_map      |    123456 |         610 | table_id: 108 (test.table1)                              |
| mysql-bin.000002 | 610 | Write_rows     |    123456 |         659 | table_id: 108 flags: STMT_END_F                        |
| mysql-bin.000002 | 659 | Xid            |    123456 |         690 | COMMIT /* xid=35 */                                    |
| mysql-bin.000002 | 690 | Rotate         |    123456 |         737 | mysql-bin.000003;pos=4                                 |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------+
13 rows in set

虽然有很多看似复杂的指令，但是还是不难看出，在02里，有两条写操作，03里有一条删除操作。

一条插入操作的完整日志是这样：

| mysql-bin.000002 | 154 | Anonymous_Gtid |    123456 |         219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS'                   |
| mysql-bin.000002 | 219 | Query          |    123456 |         293 | BEGIN                                                  |
| mysql-bin.000002 | 293 | Table_map      |    123456 |         343 | table_id: 108 (test.table1)                              |
| mysql-bin.000002 | 343 | Write_rows     |    123456 |         390 | table_id: 108 flags: STMT_END_F                        |
| mysql-bin.000002 | 390 | Xid            |    123456 |         421 | COMMIT /* xid=34 */                                    |

我们的目的是恢复误删的数据，其实就是将binlog.000002日志的两条插入记录重演一遍，而不需要取理会binlog.000003的操作（因为删除是一个误操作）

所以现在能理解为什么我们频繁刷新binlog了吧，当然，在实际的线上环境中，我们肯定需要将binlog导出后，仔细筛选出误操作，并将其排除，之后再运行binlog。

在本文中，我们只做一个恢复两条插入语句的操作，执行语句：

sudo mysqlbinlog /var/lib/mysql/mysql-bin.000002 --start-position 154 --stop-position 690 | mysql -uroot -p mytest

注意：这里填写的路径/var/lib/mysql/mysql-bin.000002需要具体到你的binlog目录，网上大部分文章只写到mysql-bin.000002，如果你不在目录里，mysqlbinlog命令并不会自动定位binlog所在路径。

参数描述：

--start-datetime：从二进制日志中读取指定等于时间戳或者晚于本地计算机的时间

--stop-datetime：从二进制日志中读取指定小于时间戳或者等于本地计算机的时间 取值和上述一样

--start-position：从二进制日志中读取指定position 事件位置作为开始。

--stop-position：从二进制日志中读取指定position 事件位置作为事件截至

执行成功后，再次查看表table1，可以看到两条新的id=3和4的数据被插入了进来。恢复成功了。

mysql> select * from table1;
+----+----+
| id |name|
+----+----+
|  1 | A  |
|  2 | B  |
|  3 | C  | 
|  3 | C  |
|  4 | D  |
+----+----+
6 rows in set

延伸思考

Binlog在什么情况下无法恢复数据？

结语

删库跑路不用怕，其他开发运维都等着恢复你的数据呢，多好的练手机会是不是。

当然，看完binlog日志恢复数据的原理，希望大家以后在定期备份数据库的脚本里，也能够加上刷新binlog日志的命令，这样一旦某天丢失数据，可以将当天binlog数据单独拿出来还原，做到清晰可辨，也加快恢复效率。

参考

https://www.cnblogs.com/rjzheng/p/9721765.html

https://blog.csdn.net/king_kgh/article/details/74890381

https://www.jianshu.com/p/564fcc2b5e31

https://blog.csdn.net/king_kgh/article/details/74833539

关注我

我是一名后端开发工程师。

主要关注后端开发，数据安全，爬虫，物联网，边缘计算等方向，欢迎交流。

各大平台都可以找到我

微信公众号：后端技术漫谈
Github：@qqxx6661
CSDN：@后端技术漫谈
知乎：@后端技术漫谈
简书：@后端技术漫谈
掘金：@后端技术漫谈

原创博客主要内容

后端开发相关技术文章
Java面试知识点复习全手册
设计模式/数据结构
Leetcode/剑指offer 算法题解析
SpringBoot/SpringCloud 入门实战系列
爬虫相关技术文章
逸闻趣事/好书分享/个人兴趣

个人公众号：后端技术漫谈

公众号：后端技术漫谈.jpg

如果文章对你有帮助，不妨收藏，投币，转发，在看起来~

后端技术漫谈博客专家

发布了259 篇原创文章 · 获赞 136 · 访问量 38万+

他的留言板关注

[40] use Binlog foot library log data to recover accidentally deleted MySQL