Mysql backup and recovery (full backup, incremental backup)

1. Data backup overview

1.1 The importance of data backup

• In a production environment, data security is critical
• Any loss of data may have the consequences of verification
• Causes of data loss:
  Program errors, think operation errors (mostly), operation errors, disk failures, disasters (such as earthquakes and fires) and theft

1.2 Classification of database backup

• From a physical and logical point of view, it can be divided into:
  physical backup: the backup of the physical files of the database operating system (such as data files, log files, etc.)
  Logical backup: the backup of the database logical components (such as tables and other database objects)
• Physical backup method:
  cold backup (offline backup): it is performed when the database is closed.
  Hot backup (online backup): the database is running and depends on the database log file.
  Warm backup: database lock table (not writable but Readable) status for backup operations

1.3 Commonly used backup methods

• Physical backup The
  database is closed during the backup, and the database files are directly packaged. The
  backup speed is fast and the recovery is the easiest
• Dedicated backup tool mydump or mysqlhotcopy
  mysqldump commonly used logical backup tool
  mysqlhotcopy only has backup MyISAM and ARCHIVE tables
• Enable binary log for incremental backup To
  perform incremental backup, you need to refresh the binary log
• Third-party tools
  Free Mysql hot backup software Percona XtraBackup

1.4 Mysql full backup

• It is a backup of the entire database, database structure and file structure.
• Save the database when the backup is completed
• It is the basis of differential backup and incremental backup
• Advantages: simple and convenient backup and recovery operations
• Disadvantages: There is a lot of duplication of data,
  taking up a lot of backup space,
  and long backup and recovery time

1.4.1 Database full backup classification

• Physical cold backup and recovery
  Close the Mysql database
  Use the tar command to directly package the database folder and
  directly replace the existing Mysql directory
• mysqldump backup and restore
  MySQL comes with a backup tool, which can facilitate the backup of MySQL.
  You can export the specified library and table as SQL script.
  Use the command mysql to import the backup data

Two, MySQL physical cold backup and recovery

• Backup

[root@localhost ~]# systemctl stop mysqld
[root@localhost ~]# tar zcf /opt/mysql_all-$(date +%F).tar.gz /usr/local/mysql/data/

• reduction

[root@localhost mnt]# mkdir /bak/
[root@localhost mnt]# mv /usr/local/mysql/data/ /mnt/bak/        ## 将出现问题的数据移/mnt/bak里面
[root@localhost opt]# tar zxf /opt/mysql_all-2020-08-23.tar.gz -C /mnt/   ## 解压备份的文件到mnt下
[root@localhost mnt]# mv usr/local/mysql/data/  /usr/local/mysql/  ## 将备份文件移动至库文件夹下
[root@localhost ~]# systemctl start  mysqld  ## 重新启动服务

Three, mysqldump backup database

• The mysqldump command makes a full backup of a single library
• Basic format

mysqldump -u 用户名 -p [密码][选项][数据库] > /备份路径/备份文件名
例：
[root@localhost mysql]# mysqldump -uroot -p students > /opt/students.sql
Enter password: 

• Full backup of the database

mysqldump -u 用户名 -p [密码][选项] --all-databases >  /备份路径/备份文件名
例：
[root@localhost mysql]# mysqldump -uroot -p --all-databases > /opt/all_data.sql

• Back up specific tables in the library

mysqldump -u 用户名 -p [密码][选项] 数据库名 表名 > /备份路径/备份文件名
例：
[root@localhost mysql]# mysqldump -uroot -p students test  > /opt/students-test.sql

Fourth, restore the database or table

4.1 Restore the database

• Use mysqldump to export the script, you can use the import method
  source command mysql command

• Steps to restore the database using source

• Log in to the MySQL database

• The path to execute the source backup sql script

• Example of source recovery

mysql> source /opt/students.sql

• Example of mysql recovery

[root@localhost mysql]# mysql -uroot -pabc123  students < /opt/students.sql   ## mysql单个库的还原需要指定库，
## 两个或两个以上的恢复，就不需要，因为此时本分的文件里有了创建数据库的操作，单个的话没有

4.2 Recovery table

• You can also use source or mysql commands when restoring tables
• The operation of the source recovery table is the same as that of the recovery library
• When the backup file contains only the backup of the table and does not include the statement to create the library, the library name must be specified and the target library must exist

mysql -u [用户名] -p [密码] < 表备份脚本的路径

Five, incremental backup

It is the file or content added or changed after the last backup (the basis of incremental backup is the last full backup)

• Features
  There is no duplication of data, the amount of backup is not large, and the time period.
  Recovery requires the last full backup and all incremental backups after the full backup to restore, and all incremental backups must be reversed and restored one by one.

mysql has no direct backup method, it can be backed up indirectly through the binary log provided by mysql

• The meaning of mysql binary log for backup : the
  binary log saves all updates or operations that may update the database. The
  binary log starts to record after starting the mysql server, and recreates a new file after the file reaches the size set by max_binlog_size or receives the flush logs command the log file
  where only the timing of the implementation flush logs method to re-create a new log file generated binary sequence and in a timely manner to save these logs to a secure incremental backups completed a period of time

5.1 MySQL database incremental recovery

5.1.1 General recovery

General recovery: recover all binary log contents

• Basic format

mysqlbinlog [--no-defaults] 增量备份文件 | mysql -u  用户名 -p

5.1.2 Recovery based on location

• Recover based on position: Recover in a certain wrong operation segment, skip the wrong operation to recover

• Restore data to the specified location (that is, stop at the specified location)

mysqlbinlog [--no-defaults]  --stop-position='操作 id' 二进制文件  | mysql -u  用户名 -p 密码

• Recover data from the specified location

mysqlbinlog [--no-defaults]  --start-position='操作 id' 二进制文件  | mysql -u  用户名 -p 密码

5.1.3 Recovery based on point in time

Point-in-time recovery: skip the point in time when there is an error operation

• Recovery from the beginning of the log to a certain point in time

mysqlbinlog  [--no-defaults] --stop-datetime='年-月-日 小时:分钟:秒'  二进制日志 | mysql -u 用户名 -p  密码``

• Recovery from a certain point in time to the end of the log

mysqlbinlog  [--no-defaults] --start-datetime='年-月-日 小时:分钟:秒'  二进制日志 | mysql -u 用户名 -p  密码

• Recovery from a certain point in time to a certain time

mysqlbinlog  [--no-defaults] --start-datetime='年-月-日 小时:分钟:秒'  --stop-datetime='年-月-日 小时:分钟:秒' 二进制日志 | mysql -u 用户名 -p  密码

5.1.4 Binary log file

• Enable binary log function

vim   /etc/cnf
在mysqld 中加 log-bin=mysql-bin

• View binary log files

mysqlbinlog --no-defaults  mysql-bin.000001   ## mysql-bin.000001为日志增量名

• Decode binary files

mysqlbinlog --no-defaults --base64-output=decode-rows -v  mysql-bin.000001 >/opt/bk02.txt

• The operation is complete to generate a new binary log file

mysqladmin -uroot -p flush-logs  ## 刷新日志，生成新的存储日志的文件，备份增量日志

5.2 Incremental recovery of files

5.2.1 Enable binary log function

[root@localhost ~]# systemctl stop mysqld  ## 先关闭数据库
[root@localhost ~]# vim /etc/my.cnf
[mysqld]
省略不写……
log-bin=mysql-bin  ## 在mysqld模块下添加二进制日志功能
[root@localhost ~]# ls /usr/local/mysql/data/    ##   mysql-bin.000001 发现二进制日志文件
auto.cnf        ibdata1      ib_logfile1  mysql             mysql-bin.index     students
ib_buffer_pool  ib_logfile0  ibtmp1       mysql-bin.000001  performance_schema  sys

5.2.2 Operation

[root@localhost ~]# mysqldump -uroot -p students > /mnt/students.sql ## 进行数据库students的备份
[root@localhost data]# mysqladmin -uroot -pabc123 flush-logs  ## 生成新的增量备份文件
mysql> select * from xuesheng;          ## 查看未操作的表
+----+----------+-------+----------+-------+
| id | name     | score | address  | hobby |
+----+----------+-------+----------+-------+
|  1 | zhangsan |    90 | nanjing  |     1 |
|  2 | lisi     |    88 | chengdu  |     1 |
|  3 | wangwu   |    75 | shanghai |     2 |
|  4 | zhaoliu  |    60 | beijing  |     2 |
|  5 | tianqi   |    78 | hangzhou |     1 |
|  6 | heiba    |    78 | hangzhou |     1 |
|  7 | heiba1   |    78 | hangzhou |     1 |
|  8 | heiba2   |    78 | hangzhou |     1 |
+----+----------+-------+----------+-------+
8 rows in set (0.00 sec)
mysql> insert into xuesheng(name,score,address,hobby) values('tom',85,'guangzhou',1);
Query OK, 1 row affected (0.00 sec)

mysql> insert into xuesheng(name,score,address,hobby) values('jerry',62,'chongqing',2);
Query OK, 1 row affected (0.01 sec)

mysql> delete  from xuesheng where id=2;  ## 模拟误操作
Query OK, 1 row affected (0.00 sec)
mysql> insert into xuesheng(name,score,address,hobby) values('lilei',73,'wuhan',1);
Query OK, 1 row affected (0.00 sec)

mysql> select * from xuesheng;
+----+----------+-------+-----------+-------+
| id | name     | score | address   | hobby |
+----+----------+-------+-----------+-------+
|  1 | zhangsan |    90 | nanjing   |     1 |
|  3 | wangwu   |    75 | shanghai  |     2 |
|  4 | zhaoliu  |    60 | beijing   |     2 |
|  5 | tianqi   |    78 | hangzhou  |     1 |
|  6 | heiba    |    78 | hangzhou  |     1 |
|  7 | heiba1   |    78 | hangzhou  |     1 |
|  8 | heiba2   |    78 | hangzhou  |     1 |
|  9 | tom      |    85 | guangzhou |     1 |
| 10 | jerry    |    62 | chongqing |     2 |
| 11 | lilei    |    73 | wuhan     |     1 |
+----+----------+-------+-----------+-------+
10 rows in set (0.00 sec)

5.2.3 Decoding binary files

[root@localhost data]#  mysqladmin -uroot -pabc123 flush-logs      ## 保存刚刚的增量备份二进制文件，生成新的备份文件
乱码看不懂  需解码
64位解码    -v  显示更详细信息  
mysqlbinlog --no-defaults --base64-output=decode-rows -v  mysql-bin.000002 >/mnt/bk02.txt
[root@localhost mnt]# vim /mnt/bk02.txt 
# at 891    ## 误操作时间与位置
#200824 10:21:46 server id 1  end_log_pos 954 CRC32 0x0d15a891  Table_map: `students`.`xuesheng` mapped to number 124
# at 954
#200824 10:21:46 server id 1  end_log_pos 1015 CRC32 0xc2261844         Delete_rows: table id 124 flags: STMT_END_F
### DELETE FROM `students`.`xuesheng`
### WHERE
###   @1=2
###   @2='lisi'
###   @3=88
###   @4='chengdu'
###   @5=1


# at 1187                ## 紧接着插入lilei的时间与位置
#200824 10:22:23 server id 1  end_log_pos 1250 CRC32 0xfa311240         Table_map: `students`.`xuesheng` mapped to number 124
# at 1250
#200824 10:22:23 server id 1  end_log_pos 1310 CRC32 0x9bce74dd         Write_rows: table id 124 flags: STMT_END_F
### INSERT INTO `students`.`xuesheng`
### SET
###   @1=11
###   @2='lilei'
###   @3=73
###   @4='wuhan'
###   @5=1

5.2.4 Restore files based on location

• Location based recovery

mysql> drop table xuesheng;    ## 删除操作错误后存在的表
Query OK, 0 rows affected (0.01 sec)
mysql> source /mnt/students.sql  ## 用完整备份的文件恢复
mysql> select * from xuesheng;
+----+----------+-------+----------+-------+
| id | name     | score | address  | hobby |
+----+----------+-------+----------+-------+
|  1 | zhangsan |    90 | nanjing  |     1 |
|  2 | lisi     |    88 | chengdu  |     1 |
|  3 | wangwu   |    75 | shanghai |     2 |
|  4 | zhaoliu  |    60 | beijing  |     2 |
|  5 | tianqi   |    78 | hangzhou |     1 |
|  6 | heiba    |    78 | hangzhou |     1 |
|  7 | heiba1   |    78 | hangzhou |     1 |
|  8 | heiba2   |    78 | hangzhou |     1 |
+----+----------+-------+----------+-------+
8 rows in set (0.00 sec)

[root@localhost data]#  mysqlbinlog --no-defaults --stop-position='891' /usr/local/mysql/data/mysql-bin.000002 | mysql -u root -p
mysql> select * from xuesheng;   ## 此时已经恢复了tom  和jerry
+----+----------+-------+-----------+-------+
| id | name     | score | address   | hobby |
+----+----------+-------+-----------+-------+
|  1 | zhangsan |    90 | nanjing   |     1 |
|  2 | lisi     |    88 | chengdu   |     1 |
|  3 | wangwu   |    75 | shanghai  |     2 |
|  4 | zhaoliu  |    60 | beijing   |     2 |
|  5 | tianqi   |    78 | hangzhou  |     1 |
|  6 | heiba    |    78 | hangzhou  |     1 |
|  7 | heiba1   |    78 | hangzhou  |     1 |
|  8 | heiba2   |    78 | hangzhou  |     1 |
|  9 | tom      |    85 | guangzhou |     1 |
| 10 | jerry    |    62 | chongqing |     2 |
+----+----------+-------+-----------+-------+
[root@localhost data]# mysqlbinlog --no-defaults --start-position='1187' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p
mysql> select * from xuesheng;      ## 恢复lilei  未删除lisi                                                                  
+----+----------+-------+-----------+-------+
| id | name     | score | address   | hobby |
+----+----------+-------+-----------+-------+
|  1 | zhangsan |    90 | nanjing   |     1 |
|  2 | lisi     |    88 | chengdu   |     1 |
|  3 | wangwu   |    75 | shanghai  |     2 |
|  4 | zhaoliu  |    60 | beijing   |     2 |
|  5 | tianqi   |    78 | hangzhou  |     1 |
|  6 | heiba    |    78 | hangzhou  |     1 |
|  7 | heiba1   |    78 | hangzhou  |     1 |
|  8 | heiba2   |    78 | hangzhou  |     1 |
|  9 | tom      |    85 | guangzhou |     1 |
| 10 | jerry    |    62 | chongqing |     2 |
| 11 | lilei    |    73 | wuhan     |     1 |
+----+----------+-------+-----------+-------+

5.2.5 Restore files based on point in time

• The previous operations are the same, but the restoring commands are different

[root@localhost data]# mysqlbinlog --no-defaults --stop-datetime='2020-08-24 10:21:46' /usr/local/mysql/data/mysql-bin.000002 |mysql -uroot -p

[root@localhost data]# mysqlbinlog --no-defaults --start-datetime='2020-08-24 10:22:23' /usr/local/mysql/data/mysql-bin.000002 | mysql -uroot -p