MySQL利用binlog实现增量备份
MySQL的二进制文件记录了所有DDL和DML语句，所以可以结合dmp文件实现mysql的增量备份，下面进行实验。
试验环境
操作系统：CentOS Linux release 7.3.1611 (Core)
数据库：MySQL Community Server 8.0.15
开启binlog日志
MySQL5.x版本中，binlog日志默认是关闭状态。而8.0版本中binlog默认是开启状态：
mysql> show variables like 'log_%';
+----------------------------------------+----------------------------------------+
| Variable_name | Value |
+----------------------------------------+----------------------------------------+
| log_bin | ON |
| log_bin_basename | /var/lib/mysql/binlog |
| log_bin_index | /var/lib/mysql/binlog.index |
| log_bin_trust_function_creators | OFF |
| log_bin_use_v1_row_events | OFF |
| log_error | /var/log/mysqld.log |
| log_error_services | log_filter_internal; log_sink_internal |
| log_error_suppression_list | |
| log_error_verbosity | 2 |
| log_output | FILE |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | ON |
| log_slow_admin_statements | OFF |
| log_slow_extra | OFF |
| log_slow_slave_statements | OFF |
| log_statements_unsafe_for_binlog | ON |
| log_throttle_queries_not_using_indexes | 0 |
| log_timestamps | UTC |
+----------------------------------------+----------------------------------------+
18 rows in set (0.01 sec)
且默认的存放路径为/var/lib/mysql/binlog
binlog相关基本操作
查看所有binlog日志
将以此列出所有binlog的名称，大小，以及是否加密
mysql> show master logs;
+---------------+-----------+-----------+
| Log_name | File_size | Encrypted |
+---------------+-----------+-----------+
| binlog.000001 | 499 | No |
| binlog.000002 | 178 | No |
| binlog.000003 | 155 | No |
| binlog.000004 | 178 | No |
| binlog.000005 | 155 | No |
| binlog.000006 | 178 | No |
| binlog.000007 | 1540 | No |
| binlog.000008 | 5291 | No |
+---------------+-----------+-----------+
8 rows in set (0.00 sec)
查看当前master状态
将展示最新一个binlog的名称，pos值
mysql> show master status;
+---------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+---------------+----------+--------------+------------------+-------------------+
| binlog.000008 | 5291 | | | |
+---------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
刷新binlog
会将缓存中的数据写入binlog，并开始一个新的binlog（像oracle的checkpoint）
mysql> flush logs;
Query OK, 0 rows affected (0.01 sec)
mysql> show master status;
+---------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+---------------+----------+--------------+------------------+-------------------+
| binlog.000009 | 155 | | | |
+---------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
mysql> show master logs;
+---------------+-----------+-----------+
| Log_name | File_size | Encrypted |
+---------------+-----------+-----------+
| binlog.000001 | 499 | No |
| binlog.000002 | 178 | No |
| binlog.000003 | 155 | No |
| binlog.000004 | 178 | No |
| binlog.000005 | 155 | No |
| binlog.000006 | 178 | No |
| binlog.000007 | 1540 | No |
| binlog.000008 | 5335 | No |
| binlog.000009 | 155 | No |
+---------------+-----------+-----------+
9 rows in set (0.00 sec)
重置binlog
将清空所有binlog，其文件也会被删除
mysql> reset master;
Query OK, 0 rows affected (0.02 sec)
mysql> show master logs;
+---------------+-----------+-----------+
| Log_name | File_size | Encrypted |
+---------------+-----------+-----------+
| binlog.000001 | 155 | No |
+---------------+-----------+-----------+
1 row in set (0.00 sec)
增量备份实验
1.创建测试数据
创建测试数据库test
mysql> create database test;
Query OK, 1 row affected (0.01 sec)
在test库中创建测试表t
mysql> use test;
Database changed
mysql> create table t(id int);
Query OK, 0 rows affected (0.03 sec)
写入测试数据
mysql> insert into t values(1);
Query OK, 1 row affected (0.01 sec)
mysql> insert into t values(2);
Query OK, 1 row affected (0.01 sec)
mysql> insert into t values(3);
Query OK, 1 row affected (0.00 sec)
mysql> select * from t;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
+------+
3 rows in set (0.00 sec)
2.对测试数据库全备
为方便起见，使用mysqldump对test库进行逻辑备份：
[root@localhost home]# mysqldump -uroot -poracle --single-transaction --flush-logs --log-error=/home/mydmp.log --databases test >/home/test_bak.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.
[root@localhost home]# ls /home/
dba mydmp.log test_bak.sql
3.继续向测试表写数据
再次向测试表t插入数据
mysql> insert into t values(4);
Query OK, 1 row affected (0.00 sec)
mysql> select * from t;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
+------+
4 rows in set (0.00 sec)
4.删除测试库
模拟故障，删除测试库test
mysql> drop database test;
Query OK, 1 row affected (0.03 sec)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.00 sec)
5.使用全备恢复数据库
备份最新的binlog：
cp /var/lib/mysql/binlog.000002 /home/
先使用全备恢复数据库：
[root@localhost home]# mysql -uroot -poracle </home/test_bak.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
检查数据库状态，发现全备后插入的数据丢失了
[root@localhost home]# mysql -uroot -poracle
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 56
Server version: 8.0.15 MySQL Community Server - GPL
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select * from test.t;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
+------+
3 rows in set (0.00 sec)
6.使用binlog恢复
先查看当前binlog状态：
mysql> show master logs;
+---------------+-----------+-----------+
| Log_name | File_size | Encrypted |
+---------------+-----------+-----------+
| binlog.000001 | 1540 | No |
| binlog.000002 | 2064 | No |
+---------------+-----------+-----------+
2 rows in set (0.00 sec)
再查看最新binlog的日志分析
mysql> show binlog events in 'binlog.000002';
+---------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+---------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| binlog.000002 | 4 | Format_desc | 1 | 124 | Server ver: 8.0.15, Binlog ver: 4 |
| binlog.000002 | 124 | Previous_gtids | 1 | 155 | |
| binlog.000002 | 155 | Anonymous_Gtid | 1 | 234 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 234 | Query | 1 | 309 | BEGIN |
| binlog.000002 | 309 | Table_map | 1 | 356 | table_id: 146 (test.t) |
| binlog.000002 | 356 | Write_rows | 1 | 396 | table_id: 146 flags: STMT_END_F |
| binlog.000002 | 396 | Xid | 1 | 427 | COMMIT /* xid=418 */ |
| binlog.000002 | 427 | Anonymous_Gtid | 1 | 504 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 504 | Query | 1 | 608 | drop database test /* xid=420 */ |
| binlog.000002 | 608 | Anonymous_Gtid | 1 | 687 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 687 | Query | 1 | 889 | CREATE DATABASE /*!32312 IF NOT EXISTS*/ `test` /*!40100 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci */ /* xid=435 */ |
| binlog.000002 | 889 | Anonymous_Gtid | 1 | 966 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 966 | Query | 1 | 1093 | use `test`; DROP TABLE IF EXISTS `t` /* generated by server */ |
| binlog.000002 | 1093 | Anonymous_Gtid | 1 | 1172 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 1172 | Query | 1 | 1373 | use `test`; CREATE TABLE `t` (
`id` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci /* xid=443 */ |
| binlog.000002 | 1373 | Anonymous_Gtid | 1 | 1450 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 1450 | Query | 1 | 1578 | use `test`; /*!40000 ALTER TABLE `t` DISABLE KEYS */ /* xid=446 */ |
| binlog.000002 | 1578 | Anonymous_Gtid | 1 | 1657 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 1657 | Query | 1 | 1732 | BEGIN |
| binlog.000002 | 1732 | Table_map | 1 | 1779 | table_id: 168 (test.t) |
| binlog.000002 | 1779 | Write_rows | 1 | 1829 | table_id: 168 flags: STMT_END_F |
| binlog.000002 | 1829 | Xid | 1 | 1860 | COMMIT /* xid=447 */ |
| binlog.000002 | 1860 | Anonymous_Gtid | 1 | 1937 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' |
| binlog.000002 | 1937 | Query | 1 | 2064 | use `test`; /*!40000 ALTER TABLE `t` ENABLE KEYS */ /* xid=448 */ |
+---------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------+
24 rows in set (0.00 sec)
可观察到drop database命令在
| binlog.000002 | 504 | Query | 1 | 608 | drop database test /* xid=420 */
所以只用恢复到pos节点504之前即可：
[root@localhost home]# mysqlbinlog --stop-position=504 --database=test /home/binlog.000002 |mysql -uroot -poracle test
mysql: [Warning] Using a password on the command line interface can be insecure.
WARNING: The option --database has been used. It may filter parts of transactions, but will include the GTIDs in any case. If you want to exclude or include transactions, you should use the options --exclude-gtids or --include-gtids, respectively, instead.
检查数据库状态
检查test库中数据状态：
[root@localhost home]# mysql -uroot -poracle
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 60
Server version: 8.0.15 MySQL Community Server - GPL
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select * from test.t;
+------+
| id |
+------+
| 1 |
| 2 |
| 3 |
| 4 |
+------+
4 rows in set (0.00 sec)
数据库已恢复正常