MySQL-- full amount, incremental backup and recovery

The importance of data backup

1, in a production environment, data security is crucial, any loss of data could have serious consequences

2, the cause of data loss

Program error
human error
computer failed
disk failure
disaster and theft

Classification database backup

Physical and logical point of view, the backup can be divided into:

1, the physical backup: physical file system, database operations (such as data files, log files, etc.) of backup
physical backup can be divided into an offline backup (cold backup) and online backup (hot standby):
cold backup: is in the closed when the database for
hot backup: database is running, this method relies on a database backup log file
2, logical backup: backup database logic components (e.g., a database table or the like object)
from the perspective of the database backup strategy, the backup can be divided into:
full backup: every time a complete backup of the data
differential backup: back up those since the last full backup of the file has been modified
incremental backup: only those in the last full backup or incremental backup after modification. files will be backed up
Note: differential and incremental complement each other

MySQL full backup

1、完全备份是对整个数据库的备份、数据库结构和文件结构的备份
2、完全备份保存的是备份完成时刻的数据库
3、完全备份是增量备份的基础
（1）完全备份的优点
●备份与恢复操作简单方便
（2）完全备份的缺点
●数据存在大量的重复
●占用大量的备份空间
●备份与恢复时间长

mysqldump backup library

1、MySQL数据库的备份可以采用用多种方式

●直接打包数据库文件夹，如/usr/local/mysql/data
●使用专用备份工具mysqldump

2、mysqldump命令

●MySQL自带的备份工具，相当方便对MySQL进行备份
●通过该命令工具可以将指定的库、表或全部的库导出为SQL脚本，在需要恢复时可进行数据恢复

3、mysqldump命令对单个库进行完全备份

mysqldump -u用户名-p [密码] [选项] [数据库名] > /备份路径/备份文件名

单库备份例子
mysqldump -u root -p auth > /backup/auth.sql
mysqldump -u root -p mysql > /bakcup/mysql.sql

4、mysqldump命令对多个库进行完全备份

mysqldump -u 用户名 -p [密码] [选项] --databases 库名1 [库名2]... >/备份路径/备份文件名

多库备份例子
mysqldump -u root -p --databases autth mysql > /backup/databases-auth-mysql.sql

5、对所有库进行完全备份

mysqldump -u用户名-p [密码] [选项] --all-databases > /备份路径/备份文件名

所有库备份例子
mysqldump -u root -p --opt --all-databases > /backup/all-data.sql

mysqldump backup table

1、在实际生产环境中，存在对某个特定表的维护操作，此时mysqldump同样发挥重大作用

2、使用mysqldump备份表的操作
mysqldump -u 用户名 -p [密码] [选项] 数据库名 表名 > /备份路径/备份文件名

备份表的例子
mysqldump -u root -p mysql user > /backup/mysql-user.sql

3、使用mysqldump备份表结构的操作
mysqldump -u 用户名 -p [密码] [选项] -d 数据库名 表名 > /备份路径/备份文件名

备份表结构的例子
mysqldump -u root -p mysql -d user > /backup/mysql-user.sql

Database recovery

1、使用mysqldump命令导出的SQL备份脚本，在进行数据恢复时可使用以下方法导入
●source命令          数据库模式中运行
●mysq|命令           Linux模式中运行

2、使用source恢复数据库的步骤
●登录到MySQL数据库
●执行source备份sq|脚本的路径（绝对路径）

source 恢复例子
MySQL [(none)] > source /backup/all-data.sql

3、使用mysq|命令恢复数据
mysql -u用户名-p [密码] <库备份脚本的路径

mysql命令恢复例子
mysql -u root -p < /backup/all-data.sql

Recovery operation table

1、恢复表时同样可以使用source或者mysql命令进行

2、source恢复表的操作与恢复库的操作相同

3、当备份文件中只包含表的备份，而不包括创建库的语句时，必须指定库名，且目标库必须存在
mysql -u用户名-p [密码] < 表备份脚本的路径
mysql -u root -p mysql < /backup/mysql-user.sql

4、在生产环境中，可以使用shell脚本自动实现定期备份

MySQL backup ideas

1、定期实施备份，制定备份计划或者策略，并严格遵守
2、除了进行完全备份，开启MySQL服务器的日志功能是很重要的
●完全备份加上日志，可以对MySQL进行最大化还原
MySQL-bin：MySQL数据库的二进制日志,用于记录用户对数据库操作的SQL语句((除了数据查询语句)信息。可以使用mysqlbin命令查看二进制日志的内容
3、使用统一的和易理解的备份文件名称
●不要使用backup1、backup2等这样没有意义的名字
●推荐使用库名或者表名加上时间的命名规则

MySQL incremental backup

1、使用mysqldump进行完全备份的存在的问题
●备份数据中有重复数据
●备份时间与恢复时间长
2、增量备份就是备份自上一次备份之后增加或变化的文件或者内容
3、增量备份的特点
●没有重复数据，备份量不大，时间短
●恢复麻烦：需要上次完全备份及完全备份之后所有的增量备份才能恢复，而且要对所有增量备份进行逐个反推恢复
4、MySQL没有提供直接的增量备份方法
5、可以通过MySQL提供的二进制日志(binary logs)间接实现增量备份
6、MySQL二进制日志对备份的意义
●二进制日志保存了所有更新或者可能更新数据库的操作
●二进制日志在启动MySQL服务器后开始记录，并在文件达到max_ binlog_size所设置的大小或者接收到flush logs命令后重新创建新的日志文件
●只需定时执行flush logs方法重新创建新的日志，生成二进制文件序列，并及时把这些旧的日志保存到安全的地方就完成了一个时间段的增量备份
7、一般恢复
添加数据——进行完全备份——录入新的数据——进行增量备份——模拟故障——恢复操作
8、基于位置回复
就是将某个起始时间的二进制日志导入数据库中，从而跳过某个发生错误的时间点实现数据的恢复
9、基于时间点恢复
使用基于时间点的恢复，可能会出现在一个时间点里既同时存在正确的操作又存在错误的操作，所以我们需要一种更为精确的恢复方式
10、增量恢复的方法
（1）一般恢复
mysqlbinlog [--no-defaults]增量备份文件 | mysql -u用户名 -p
（2）基于位置的恢复
●恢复数据到指定位置
mysqlbinlog --stop-position=操作'id' 1进制日志 | mysql -u用户名 -p 密码
●从指定的位置开始恢复数据
mysqlbinlog --start-position=操作'id'二进制日志 | mysql -u用户名 -p 密码
（3）基于时间点的恢复
●从日志开头截止到某个时间点的恢复
mysqlbinlog [--no-defaults] --stop-datetime='年-月-日 小时:分钟:秒'二进制日志 | mysql -u用户名 -p 密码
●从某个时间点到日志结尾的恢复
mysqlbinlog [--no defaults] --start-datetime='年-月-日 小时:分钟:秒'二进制日志 | mysql -u用户名 -p 密码
●从某个时间点到某个时间点的恢复
mysqlbinlog [--no defaults] --start-datetime='年-月-日 小时:分钟:秒' --stop-datetime='年-月-日 小时:分钟:秒'二进制日志 | mysql -u用户名 -p 密码
查看二进制日志文件（解码）
mysqlbinlog --no-defaults --base64-output=decode-rows -V mysql-bin.000002 > /opt/ bak. txt

Full backup and restore practical operation

1, access to the database, create tables, insert data table

[root@master2 ~]# mysql -uroot -p   ##进入数据库
Enter password: 

mysql> create database school;    ##创建数据库
Query OK, 1 row affected (0.01 sec)

mysql> use school;   ##使用数据库
Database changed
mysql> create table info(    ##创建表
        -> id int(3) not null primary key auto_increment,
        -> name varchar(10) not null,
        -> score decimal(4,1) not null);
Query OK, 0 rows affected (0.02 sec)

mysql> desc info;   ##查看表结构
+-------+--------------+------+-----+---------+----------------+
| Field | Type         | Null | Key | Default | Extra          |
+-------+--------------+------+-----+---------+----------------+
| id    | int(3)       | NO   | PRI | NULL    | auto_increment |
| name  | varchar(10)  | NO   |     | NULL    |                |
| score | decimal(4,1) | NO   |     | NULL    |                |
+-------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

mysql> insert into info (name,score) values ('stu01',88),('stu02',77);   ##插入表数据
Query OK, 2 rows affected (0.02 sec)
Records: 2  Duplicates: 0  Warnings: 0

mysql> select * from info;   ##查看表内容
+----+-------+-------+
| id | name  | score |
+----+-------+-------+
|  1 | stu01 |  88.0 |
|  2 | stu02 |  77.0 |
+----+-------+-------+
2 rows in set (0.01 sec)

mysql> select * from info limit 1;   ##只显示表中的前1行
+----+-------+-------+
| id | name  | score |
+----+-------+-------+
|  1 | stu01 |  88.0 |
+----+-------+-------+
1 row in set (0.00 sec)

2, a full backup of the database physical

[root@master2 ~]# cd /usr/local/mysql/data/   ##切换到数据库的数据目录下
[root@master2 data]# ls
auto.cnf        ibdata1      ib_logfile1  mysql               school  test
ib_buffer_pool  ib_logfile0  ibtmp1       performance_schema  sys
[root@master2 data]# cd school/
[root@master2 school]# ls   ##数据中的文件
db.opt  info.frm  info.ibd
[root@master2 school]# cd ..
[root@master2 data]# tar Jcvf /opt/mysql-$(date +%F).tar.xz /usr/local/mysql/data/ ##用xz格式压缩
[root@master2 data]# cd /opt/
[root@master2 opt]# ls
mysql-2019-11-26.tar.xz  mysql-5.7.20  rh

3, a single logical database backup

[root@master2 opt]# mysqldump -uroot -p school > /opt/school.sql  ##逻辑备份单个数据库
Enter password: 
[root@master2 opt]# ls
mysql-2019-11-26.tar.xz  mysql-5.7.20  rh  school.sql
[root@master2 opt]# vim school.sql  ##查看备份数据库脚本

...
CREATE TABLE `info` (
    `id` int(3) NOT NULL AUTO_INCREMENT,
    `name` varchar(10) NOT NULL,
    `score` decimal(4,1) NOT NULL,
    PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;
...
LOCK TABLES `info` WRITE;
/*!40000 ALTER TABLE `info` DISABLE KEYS */;
INSERT INTO `info` VALUES (1,'stu01',88.0),(2,'stu02',77.0);

4, to back up multiple databases

[root@master2 opt]# mysqldump -uroot -p --databases school mysql > /opt/db_school_mysql.sql
##备份多个数据库
Enter password: 
[root@master2 opt]# ls
db_school_mysql.sql  mysql-2019-11-26.tar.xz  mysql-5.7.20  rh  school.sql

5, make a full backup of the database

[root@master2 opt]# mysqldump -uroot -p --opt --all-databases > /opt/all.sql ##完全备份
Enter password: 
[root@master2 opt]# ls
all.sql              mysql-2019-11-26.tar.xz  rh
db_school_mysql.sql  mysql-5.7.20             school.sql

6, on the table in the database backup

[root@master2 opt]# mysqldump -uroot -p school info > /opt/school_info.sql ##对数据库中的表进行备份
Enter password: 
[root@master2 opt]# ls
all.sql              mysql-2019-11-26.tar.xz  rh               school.sql
db_school_mysql.sql  mysql-5.7.20             school_info.sql

7, a table structure in the database backup

[root@master2 opt]# mysqldump -uroot -p -d school info > /opt/school_info_desc.sql  
##对表结构进行备份
Enter password: 
[root@master2 opt]# ls
all.sql                  mysql-5.7.20          school_info.sql
db_school_mysql.sql      rh                    school.sql
mysql-2019-11-26.tar.xz  school_info_desc.sql

8, script-based recovery database

[root@master2 opt]# mysql -uroot -p     ##进入数据库
Enter password: 

mysql> show databases;   ##查看数据库
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| school             |
| sys                |
| test               |
+--------------------+
6 rows in set (0.00 sec)

mysql> use school;    ##使用数据库
Database changed

mysql> show tables;   ##查看表
+------------------+
| Tables_in_school |
+------------------+
| info             |
+------------------+
1 row in set (0.00 sec)

mysql> drop table info;    ##删除表
Query OK, 0 rows affected (0.01 sec)

mysql> show tables;        ###查看表
Empty set (0.00 sec)

mysql> source /opt/school.sql  ##恢复数据库脚本文件

mysql> show tables;    ##查看表
+------------------+
| Tables_in_school |
+------------------+
| info             |
+------------------+
1 row in set (0.00 sec)

9, based on an external MySQL database recovery command

mysql> drop table info;   ##删除表
Query OK, 0 rows affected (0.01 sec)

mysql> show tables;   ##查看表
Empty set (0.00 sec)

mysql> quit   ##退出
Bye
[root@master2 opt]# mysql -uroot -p123123 school < /opt/school.sql   ##利用mysql命令进行恢复
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@master2 opt]# mysql -uroot -p123123    ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;    ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;   ##查看表
+------------------+
| Tables_in_school |
+------------------+
| info             |
+------------------+
1 row in set (0.00 sec)

MySQL incremental backup and restore practical operation

1, open the binary log file

[root@master2 opt]# vim /etc/my.cnf  ##开启二进制日志文件
[mysqld]
user = mysql
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
character_set_server=utf8
pid-file = /usr/local/mysql/mysql.pid
socket = /usr/local/mysql/mysql.sock
log-bin=mysql-bin   ##开启二进制日志文件
server-id = 1
[root@master2 opt]# systemctl restart mysqld.service   ##重启mysql服务
[root@master2 opt]# cd /usr/local/mysql/data/   ##切换到mysql站点
[root@master2 data]# ls     ##查看二进制日志文件
auto.cnf        ib_logfile0  mysql             performance_schema  test
ib_buffer_pool  ib_logfile1  mysql-bin.000001  school
ibdata1         ibtmp1       mysql-bin.index   sys

2, a full backup

[root@master2 data]# mysqldump -uroot -p123123 school > /opt/school.sql ##一次完全备份
mysqldump: [Warning] Using a password on the command line interface can be insecure.
[root@master2 data]# ls
auto.cnf        ib_logfile0  mysql             performance_schema  test
ib_buffer_pool  ib_logfile1  mysql-bin.000001  school
ibdata1         ibtmp1       mysql-bin.index   sys
[root@master2 data]# mysqladmin -uroot -p123123 flush-logs  ##刷新二进制日志文件
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
[root@master2 data]# ls     ##生成新的二进制日志文件，接下来的操作会保存在mysql-bin.000002中
auto.cnf        ib_logfile0  mysql             mysql-bin.index     sys
ib_buffer_pool  ib_logfile1  mysql-bin.000001  performance_schema  test
ibdata1         ibtmp1       mysql-bin.000002  school

3, into the database, simulation misuse

[root@master2 data]# mysql -uroot -p123123  ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;    ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from info;  ##查看表
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
+----+------+-------+
2 rows in set (0.00 sec)

mysql> insert into info (name,score) values ('by01',66);   ##正确操作
Query OK, 1 row affected (0.00 sec)

mysql> select * from info;
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
+----+------+-------+
3 rows in set (0.00 sec)

mysql> delete from info where name='st01';   ##错误操作
Query OK, 1 row affected (0.00 sec)

mysql> insert into info (name,score) values ('by02',99); ##正确操作
Query OK, 1 row affected (0.00 sec)

mysql> select * from info;
+----+------+-------+
| id | name | score |
+----+------+-------+
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
|  4 | by02 |  99.0 |
+----+------+-------+
3 rows in set (0.00 sec)

[root@master2 data]# mysqladmin -uroot -p123123 flush-logs    ##刷新二进制日志文件        
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
[root@master2 data]# mysqlbinlog --no-defaults --base64-output=decode-rows -v mysql-bin.000002 > /opt/bak.txt
##用64位解码器查看二进制日志文件，并生成一个文件
[root@master2 data]# cd /opt/
[root@master2 opt]# ls
bak.txt  mysql-5.7.20  rh  school.sql
[root@master2 opt]# vim bak.txt    ##查看二进制日志文件

#at 1084
#191127 20:14:01 server id 1  end_log_pos 1132 CRC32 0xdcc90eb5         Write_rows: table id 221 flags: STMT_END_F
###INSERT INTO `school`.`info`   ##第一次正确操作的时间和位置
###SET
###@1=3
###@2='by01'
###@3=66.0
...
#at 1302   ##停止位置点
#191127 20:14:46 server id 1  end_log_pos 1357 CRC32 0x6648509a         Table_map: `school`.`info` mapped to number 221
#at 1357
#191127 20:14:46 server id 1  end_log_pos 1405 CRC32 0x1eeb752b         Delete_rows: table id 221 flags: STMT_END_F
###DELETE FROM `school`.`info`   ##第二次执行错误操作的时间和位置191127 20:14:46
###WHERE
###@1=1
###@2='st01'
###@3=88.0
#at 1405  ##开始位置点
#191127 20:14:46 server id 1  end_log_pos 1436 CRC32 0xf1c8d903         Xid = 54
...
#at 1630
#191127 20:15:16 server id 1  end_log_pos 1678 CRC32 0x08d9b0f4         Write_rows: table id 221 flags: STMT_END_F
###INSERT INTO `school`.`info`   ##第二次正确操作的时间和位置191127 20:15:16
###SET
###@1=4
###@2='by02'
###@3=99.0

4, a break point in time based recovery

[root@master2 opt]# mysql -uroot -p123123   ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;  ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> drop table info;   ##删除数据库
Query OK, 0 rows affected (0.01 sec)

mysql> select * from info;   ##查看表
ERROR 1146 (42S02): Table 'school.info' doesn't exist
mysql> source /opt/school.sql   ##恢复完全备份数据库脚本
...
mysql> show tables;   ##查看表
+------------------+
| Tables_in_school |
+------------------+
| info             |
+------------------+
1 row in set (0.00 sec)

mysql> select * from info;  ##查看表数据
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
+----+------+-------+
2 rows in set (0.00 sec)

[root@master2 opt]# mysqlbinlog --no-defaults --stop-datetime='2019-11-27 20:14:46' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p123123
##恢复bin.000002中前一个正确的执行语句（从第二个错误语句时间点停止）
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@master2 opt]# mysql -uroot -p123123   ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;  ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from info;   ##查看表数据，恢复了第一次正确操作
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
+----+------+-------+
3 rows in set (0.00 sec)

[root@master2 opt]# mysqlbinlog --no-defaults --start-datetime='2019-11-27 20:15:16' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p123123   
##跳过错误节点，恢复最后一个正确的操作（从最后一个正确的操作时间点开始）
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@master2 opt]# mysql -uroot -p123123  ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;   ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from info;    ##查看表数据，恢复了第二次正确操作，跳过了错误的操作
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
|  4 | by02 |  99.0 |
+----+------+-------+
4 rows in set (0.00 sec)

5, breakpoint recovery point based on location

mysql> delete from info where name='by01';    ##为实验方便直接删除
Query OK, 1 row affected (0.01 sec)

mysql> delete from info where name='by02';    ##删除
Query OK, 1 row affected (0.00 sec)

mysql> select * from info;  ##完全备份的初始状态
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
+----+------+-------+
2 rows in set (0.00 sec)

mysql> quit
Bye

[root@master2 opt]# mysqlbinlog --no-defaults --stop-position='1302' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p123123                           
##跳过错误操作的位置点从上一个位置点开始
[root@master2 opt]# mysql -uroot -p123123   ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;    ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from info;   ##查看表数据，恢复了第一次正确的操作
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
+----+------+-------+
3 rows in set (0.00 sec)

mysql> quit
Bye

[root@master2 opt]# mysqlbinlog --no-defaults --start-position='1405' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p123123   
##从错误的位置后一个位置点开始，跳过错误操作的位置点
[root@master2 opt]# mysql -uroot -p123123   ##进入数据库
mysql: [Warning] Using a password on the command line interface can be insecure.

mysql> use school;   ##使用数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from info;    ##查看表数据，跳过错误操作，恢复第二次正确操作数据
+----+------+-------+
| id | name | score |
+----+------+-------+
|  1 | st01 |  88.0 |
|  2 | st02 |  77.0 |
|  3 | by01 |  66.0 |
|  4 | by02 |  99.0 |
+----+------+-------+
4 rows in set (0.00 sec)

6, to restore all the incremental backups

[root@master2 opt]# mysqlbinlog --no-defaults  /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p123123
##全部增量恢复