mysqldump (MDP) application:
- Introduction, classification: Logical backup tool backup results: SQL statement, Create database, Create table,
Insert into statement - Backup logic:
RR level to
obtain consistent snapshots
A. 元数据备份
FTWRL ---->
Flush tables with read lock; ---> MDL
Flush tables ---> close tables,阻止所有commit
read lock ---> all read lock
show create database .. ---> 建库语句
show create table .. ---> 建表语句
unlock tables;
B. 备份数据行
select --->
InnoDB 可以进行快照备份
拼接成insert
非InnoDB表,锁表备份的.
适合的场景
单节点数据量,100G以内,单表2000w以内.可以使用MDP
恢复时间一般是备份时间的3-5倍.
Basic backup parameters:
-A : 全库备份
mysqldump -uroot -p'密码' -S(指定socket文件位置) -A >/(你要保存到的文件位置)
-B : 单库或者多库备份
mysqldump -uroot -p'密码' -S(指定socket文件位置) -B 选择备份的库1 选择备份的库2 >/(你要保存到的文件位置)
备份单表或者多表
mysqldump -uroot -p'密码' -S(指定socket文件位置) 库名称 选择库下的某张表1 选择库下的某张表2 >/(你要保存到的文件位置)
说明: 以上方法,在恢复时都需要先建立数据库,use到库中,然后再source导入
mysqldump special backup parameters:
1. --master-data=2
解:
自动记录binlog信息
自动进行加GRL锁,配合--single-transaction会有不一样的效果
2. --single-transaction
解:
对于innodb表进行一致性快照备份
备份期间如果有DDL操作会备份失败
3. --max_allowed_packet=128M
解:
允许备份最大数据页的内存大小
4. -R -E --triggers
解:
备份时,同时将 存储过程\函数\事件\触发器 等高级对象备份走
标准化备份:
mysqldump -uroot -p -A --master-data=2 --single-transaction --max_allowed_packet=128M -R -E --triggers >/tmp/full.sql
DIY case:
案例：通过mysqldump全备+binlog实现PIT数据恢复
环境背景： 小型的业务数据库，50G，每天23:00全备，定期binlog异地备份。
故障场景： 周三下午2点，开发Navicat连接数据库实例错误，导致生产数据被误删除（DROP）
Case ideas:
1. 挂维护页。
2. 检查备份、日志可用。
3. 如果只是部分损坏，建议找一个应急库进行恢复
a. 全备恢复
b. 日志截取并恢复
4. 恢复后数据校验 （业务测试部门验证）
5. 立即备份（停机冷备）
6. 恢复架构系统
7. 撤维护页，恢复业务
Case solution:
模拟故障:
1. 模拟测试数据
mysql> create database pit;
mysql> use pit
mysql> create table t1 (id int);
mysql> insert into t1 values(1),(2),(3);
mysql> commit;
2. 全备
[root@db01 tmp]# mysqldump -uroot -p -A --master-data=2 --single-transaction -R -E --triggers --max_allowed_packet=64M >/tmp/full_2300.sql
3. 模拟周三白天的操作
mysql> use pit
mysql> insert into t1 values(11),(22),(33);
mysql> commit;
4. 模拟周三下午2:00,删库操作
mysql> drop database pit;
恢复过程 :
1. 恢复全备
source /tmp/full_2300.sql
2. 截取二进制日志
起点: 21105555
[root@db01 tmp]# grep "\-- CHANGE MASTER TO" /tmp/full_2300.sql
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000009', MASTER_LOG_POS=21105555;
终点:21105836
a. show master status ;
b. show binlog events in 'mysql-bin.000008';
mysql> pager grep -i "drop database pit" -B 10
|
| mysql-bin.000009 | 21105805 | Xid | 51 | 21105836 | COMMIT /* xid=6232 */ |
| mysql-bin.000009 | 21105836 | Gtid | 51 | 21105913 | SET @@SESSION.GTID_NEXT= '95972e36-43fe-11eb-a366-000c2905f029:70' |
| mysql-bin.000009 | 21105913 | Query | 51 | 21106014 | drop database pit /* xid=6234 */
[root@db01 tmp]# mysqlbinlog --skip-gtids --start-position=21105555 --stop-position=21105836 /data/3306/binlog/mysql-bin.000009 >/tmp/bin.sql
3. 恢复日志
mysql> set sql_log_bin=0;
mysql> source /tmp/bin.sql
mysql> set sql_log_bin=1;
Percona Xrabackup application:
Introduce
physical backup tools. Faster backup and recovery.
Support full backup and incremental backup
Full backup logic
a. Before 8.0
Preparation: Initialize processes and threads, and allocate dedicated memory.
Backup:
0. Connect to the target database to obtain database information.
a. Current LSN number record —> show engine InnoDB stutus; Main: Last checkpoint at NO .
b. non-InnoDB data: frm, csv, MYI, MYI ----> FTWRL
c. unlock tables;
d. Copy Innodb data: ibd ibdata1 undo tmp, can allow DML
e. redo interception and backup during backup, And record the LSN number.
End:
record the current position of the binlog, and record all the backup information to a dedicated log file.
b. Before 8.0
Preparation: Initialize processes and threads, and allocate dedicated memory.
Backup:
0. Connect to the target database to obtain database information.
a. Current LSN number record —> show engine InnoDB stutus; Main: Last checkpoint at NO.
b. LOCK INSTANCE FOR BACKUP; UNLOCK INSTANCE;
c. Copy Innodb data: ibd ibdata1 undo tmp, you can allow DML
d. Redo interception and backup during backup, and record the LSN number.
End:
record the current position of the binlog , and record all backup information To the dedicated log file.
Incremental backup logic
a. For the first incremental backup, use full backup as the "reference" to save the changed data pages and redo generated during the backup process.
b. All subsequent incrementals are based on the previous backup As a reference. Save the changed data pages and the redo generated during the backup process.
c. Incremental backups are only valid for InnoDB tables, so before 8.0, they are fully prepared for non-InnoDB tables.
Recovery process:
例如:备份策略为,FULL+inc1+inc2....
a. prepare 全备 (CR)
应用redo前滚
应用undo回滚(省略)
b. 合并所有增量到全备并且prepare
应用redo前滚
应用undo回滚(除了最后一次增量,这步省略)
c. 合并后的全备prepare
d. 恢复备份
PXB的版本兼容性
MySQL 5.6 ,5.7 : PXB 2.4版本
MySQL 8.0.11 ~ 8.0.19 : PXB 8.0 稳定版.
MySQL 8.0.20 : PXB 8.0.12+
Full backup\data recovery:
安装
yum install -y percona-xtrabackup-80-8.0.13-1.el7.x86_64.rpm
全量备份
mkdir -p /data/backup
xtrabackup --defaults-file=/etc/my.cnf --user=root --password=123 --backup --target-dir=/data/backup/full
数据恢复：
a 搞破坏
[root@db01 ~]# pkill mysqld
[root@db01 ~]# rm -rf /data/3306/data/*
[root@db01 ~]# rm -rf /data/3306/logs/*
[root@db01 ~]# rm -rf /data/3306/binlog/*
b 准备：(CR)
xtrabackup --prepare --target-dir=/data/backup/full
说明： 模拟CR过程，将redo前滚，undo回滚，让备份数据是一致状态
c 拷回数据：
xtrabackup --copy-back --target-dir=/data/backup/full
d 修改权限并启动数据库
[root@db01 data]# chown -R mysql.mysql /data/*
[root@db01 data]# /etc/init.d/mysqld start
Table space migration.
Requirements: Migrate the test.t100w table of 3306 (source end) to test of 3307 (target end)
1. 目标数据库3307中创建一个源端结构一致空表(t100w)
mysql> create database test;
mysql> use test
mysql> CREATE TABLE `t100w` ( `id` int DEFAULT NULL, `num` int DEFAULT NULL, `k1` char(2) DEFAULT NULL, `k2` char(4) DEFAULT NULL, `dt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, KEY `idx` (`k1`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
2. 目标端删除表空间,保留表结构,删除表空间文件
mysql> alter table t100w discard tablespace;
3. 源端锁定需要迁移的表
mysql> flush table t100w with read lock;
4. 拷贝ibd文件到目标端目录下
[root@db01 test]# cp -a /data/3306/data/test/t100w.ibd /data/3307/data/test/
[root@db01 test]# pwd
/data/3307/data/test
[root@db01 test]# ll
total 94208
-rw-r----- 1 mysql mysql 96468992 Dec 28 17:38 t100w.ibd
5. 目标端导入表空间文件
mysql> alter table t100w import tablespace;
6. 释放源端表锁
mysql> unlock tables;
PXB full filing DIY case list:
(500G database, PXB full preparation on Sunday + Monday-Friday binlog, complete damage on Wednesday, simulation and recovery)
mysql> use pxb;
mysql> create table t1(id int);
mysql> insert into t1 values(1),(2),(3);
mysql> commit;
[root@gzl full]#xtrabackup --user=root --password=123 --backup --target-dir=/data/backup/full
mysql> use pxb;
mysql> insert into t1 values(11),(22);
mysql> commit;
[root@gzl full]# pkill mysqld
[root@gzl full]# rm -rf /data/3306/data/*
[root@gzl full]# xtrabackup --prepare --target-dir=/data/backup/full
[root@gzl full]# cat xtrabackup_binlog_info
mysql-bin.000018 276 5fb7d9f5-3db7-11eb-b613-000c29d516ad:1-19,d81850da-45ed-11eb-a0a8-000c29d516ad:1-39,d8673e6e-48f8-11eb-84ad-000c29d516ad:1-4
[root@gzl full]# mysqlbinlog --skip-gtids --start-position=276 /data/3306/binlog/mysql-bin.000018 >/tmp/bin.sql
[root@gzl full]# mv /data/3306/binlog/* /tmp
[root@gzl full]# rm -rf /data/3306/data/*
[root@gzl full]# xtrabackup --copy-back --target-dir=/data/backup/full
[root@gzl full]# chown -R mysql. /data/
[root@gzl full]# /etc/init.d/mysqld start
mysql -uroot -p123
mysql> source /tmp/bin.sql
Incremental backup walkthrough:
全量备份的目录为： mkdir -p /data/backup/full
增量备份的目录为： mkdir -p /data/backup/inc
# 1.备份操作：
# 1.1.全量备份：
xtrabackup --defaults-file=/etc/my.cnf --user=root --password=123 --backup --parallel=4 --target-dir=/data/backup/full (/data/backup/full下不能有文件)
mysql> create database pxb;
mysql> use pxb
mysql> create table t1 (id int);
mysql> insert into t1 values(1),(2),(3);
mysql> commit;
# 1.2.增量备份：
xtrabackup --defaults-file=/etc/my.cnf --user=root --password=123 --backup --parallel=4 --target-dir=/data/backup/inc --incremental-basedir=/data/backup/full
# 模拟损坏
[root@db01 ~]# pkill mysqld
[root@db01 ~]# rm -rf /data/3306/data/*
# 2.恢复操作：
# 2.1 准备全备份的日志：
xtrabackup --prepare --apply-log-only --target-dir=/data/backup/full
# 2.2 准备增量备份的日志：
xtrabackup --prepare --apply-log-only --target-dir=/data/backup/full --incremental-dir=/data/backup/inc
# 2.3 全备份准备：
rm -rf /data/3306/binlog/*
rm -rf /data/3306/data/*
# xtrabackup --prepare --target-dir=/data/backup/full
# 2.4 拷回数据：
xtrabackup --copy-back --target-dir=/data/backup/full
#2.5 修改数据目录的权限和属性：
chown -R mysql:mysql /data/*