table of Contents
- One, MySQL log management
- Two, MySQL full backup and recovery
- 3. Basic commands for cold database backup and recovery and full backup and recovery
- Four, MySQL incremental backup and recovery method
-
- 1. MySQL incremental backup
-
- Turn on the binary log function
- Full backup of database or table can be performed weekly
- Incremental backup operations can be performed every day to generate new binary log files (for example, mysql-bin.000002)
- Insert new data to simulate the increase or change of data
- Generate a new binary log file again (e.g. mysql-bin.000003)
- View the contents of the binary log file
- 2. MySQL incremental recovery
Prepare table class
mysql -u root -p
create database school;
use school;
create table class(
id int(10) not null,
name varchar(20) not null,
sex char(2) not null,
cardid varchar(20) not null,
phone varchar(11),
address varchar(50));
desc class;
insert into class values ('1','zhangsan','男','1','111111','南京');
insert into class values ('2','lisi','女','2','222222','苏州');
insert into class values ('3','wangchao','男','3','333333','扬州');
insert into class values ('4','zhanglong','男','4','444444','杭州');
insert into class values ('5','zhaohu','男','5','555555','泰州');
select * from class;
One, MySQL log management
The default storage location of MySQL logs is /usr/local/mysql/data
1. MySQL log classification
- Error log
- General query log
- Binary log
- Slow query log
2. MySQL log is turned on
vim /etc/my.cnf
[mysqld]
##错误日志,用来记录当MySQL启动、停止或运行时发生的错误信息,默认已开启
#指定错误日志的保存位置和文件名
log-error=/usr/local/mysql/data/mysql_error.log
##通用查询日志,用来记录MySQL的所有连接和语句,默认是关闭的
general_log=ON
general_log_file=/usr/local/mysql/data/mysql_general.log
##二进制日志(binlog),用来记录所有更新了数据或者已经潜在更新了数据的语句,记录了数据的更改,可用于数据恢复,默认已开启
log-bin=mysql-bin
#也可以 log_bin=mysql-bin
##慢查询日志,用来记录所有执行时间超过long_query_time秒的语句,可以找到哪些查询语句执行时间长,以便于优化,默认是关闭的
slow_query_log=ON
slow_query_log_file=/usr/local/mysql/data/mysql_slow_query.log
long_query_time=5 #设置超过5秒执行的语句被记录,缺省时为10秒
systemctl restart mysqld.service
3. Enter the database to check whether the corresponding log is open
Check whether the general query log is enabled
show variables like 'general%';
Check whether the binary log is turned on
show variables like 'log_bin%';
View slow query log related functions
(1) Check whether the slow query log function is enabled
show variables like '%slow%';
(2) View the slow query time setting
show variables like 'long_query_time';
(3) Set the method to start slow query in the database
set global slow_query_log=ON;
Two, MySQL full backup and recovery
1. The importance of data backup
The main purpose of backup is disaster recovery.
In a production environment, data security is of paramount importance.
Any loss of data may have serious consequences
2. Reasons for data loss
Program error
Human operation error
Operation error
Disk failure
Disaster (such as fire, earthquake) and theft
3. Classification of database backup
From the perspective of physics and logic:
Physical backup
- Backup of physical files (such as data files, log files, etc.) of the database operating system
- Methods of physical backup
- Cold backup (offline backup): It is performed when the database is closed
- Hot backup (online backup): The database is running and depends on the log file of the database
- Warm backup: the database is backed up when the table is locked (not writable but readable)
Logical backup
- Backup of database logic components (such as database objects such as tables)
From the perspective of database backup strategy:
Full backup
- Make a complete backup of the database every time
Differential backup
- Back up files that have been modified since the last full backup
Incremental backup
- Only files modified after the last full backup or incremental backup will be backed up
4. Common backup methods
Physical cold standby
- The database is closed during backup, and the database files are directly packaged
- The backup speed is fast, and the recovery is also the easiest
Dedicated backup tool mydump or mysqlhotcopy
- Commonly used logical backup tools for mysqldump
- mysqlhotcopy only has backup MyISAM and ARCHIVE tables
Enable binary log for incremental backup
- For incremental backup, you need to refresh the binary log
Third-party tool backup
- Free MySQL hot backup software Percona XtraBackup
5. MySQL full backup
concept
- It is a backup of the entire database, database structure and file structure
- What is saved is the database at the time the backup is completed
- It is the basis of differential backup and incremental backup
Pros and cons of full backup
advantage:
- The backup and restore operation is simple and convenient.
Disadvantages: - There is a lot of duplication in data
- Takes up a lot of backup space
- Long backup and restore time
Database full backup classification
Physical cold backup and recovery
- Shut down the MySQL database
- Use the tar command to directly package the database folder
- Simply replace the existing MySQL directory
mysqldump backup and recovery
- MySQL's own backup tool can easily realize the backup of MySQL
- Can export the specified library and table as SQL script
- Use the command mysql to import the backup data
3. Basic commands for cold database backup and recovery and full backup and recovery
1. Physical cold backup and recovery
systemctl stop mysqld
yum -y install xz #xz是一个压缩工具
#压缩备份
tar Jcvf /opt/mysql_all_$(date +%F).tar.xz /usr/local/mysql/data/
#解压恢复
tar Jxvf /opt/mysql_all_2021-02-05.tar.xz -C /usr/local/mysql/data
systemctl start mysqld
2. mysqldump backup and recovery
Full backup of one or more complete libraries (including all tables)
#导出的备份文件就是数据库脚本文件
mysqldump -u root -p[密码] --databases 库名1 [库名2] … > /备份路径/备份文件名.sql
例:
mysqldump -u root -p --databases school > /opt/school.sql
mysqldump -u root -p --databases mysql school > /opt/mysql-school.sql
Fully backup all the libraries in the MySQL server
mysqldump -u root -p[密码] --all-databases > /备份路径/备份文件名.sql
例:
mysqldump -u root -p --all-databases > /opt/all.sql
Full backup of some tables in the specified library
mysqldump -u root -p[密码] [-d] 库名 [表名1] [表名2] … > /备份路径/备份文件名.sql
#使用“ -d ”选项,说明只保存数据库的表结构
#不使用“ -d ”选项,说明表数据也进行备份
例:
mysqldump -u root -p school class > /opt/school_class.sql
View backup files
cat /opt/备份的文件 |grep -v "^--" | grep -v "^/" | grep -v "^$"
例:
cat /opt/school_class.sql |grep -v "^--" | grep -v "^/" | grep -v "^$"
3. MySQL is fully restored
Restore database
#“-e”选项,用于指定连接 MySQL 后执行的命令,命令执行完后自动退出
mysql -u root -p -e 'drop database school;'
mysql -u root -p -e 'show databases;'
mysql -u root -p < /opt/school.sql
mysql -u root -p -e 'show databases;'
Restore data table
When the backup file only contains the backup of the table and does not contain the statement of the created library, the library name must be specified when the import operation is performed, and the target library must exist.
mysql -u root -p -e 'drop table school.class;'
mysql -u root -p -e 'show tables from school;'
mysql -u root -p school < /opt/school_class.sql
mysql -u root -p -e 'show tables from school;'
Four, MySQL incremental backup and recovery method
1. MySQL incremental backup
Turn on the binary log function
vim /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id = 1
binlog_format = MIXED #指定二进制日志(binlog)的记录格式为 MIXED
#二进制日志(binlog)有3种不同的记录格式:STATEMENT(基于SQL语句)、ROW(基于行)、MIXED(混合模式),默认格式是STATEMENT
systemctl restart mysqld.service
ls -l /usr/local/mysql/data/mysql-bin.*
Full backup of database or table can be performed weekly
This kind of timing task can be executed in combination with crontab -e scheduled task
#手动执行备份
mysqldump -u root -p school class > /opt/school_class_$(date +%F).sql
mysqldump -u root -p --all-databases > /opt/allmysql_$(date +%F).sql
#使用crontab -e 计划性任务来执行;每周1凌晨2点对表class和所有的库进行备份
0 2 * * 1 mysqldump -u root -p school class > /opt/school_class_$(date +%F).sql
0 2 * * 1 mysqldump -u root -p --all-databases > /opt/allmysql_$(date +%F).sql
Incremental backup operations can be performed every day to generate new binary log files (for example, mysql-bin.000002)
mysqladmin -u root -p flush-logs
Insert new data to simulate the increase or change of data
mysql -u root -p
use school;
insert into class values ('6','zzz','男','897656','666666','南京');
insert into class values ('7','aaa','女','098765','777777','苏州');
Generate a new binary log file again (e.g. mysql-bin.000003)
mysqladmin -u root -p flush-logs
#之前的步骤4的数据库操作会保存到mysql-bin.000002文件中,之后数据库数据再发生变化则保存在mysql-bin.000003文件中
View the contents of the binary log file
cp /usr/local/mysql/data/mysql-bin.000002 /opt/
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
#--base64-output=decode-rows:使用64位编码机制去解码并按行读取
#-v:显示详细内容
2. MySQL incremental recovery
General recovery
Simulate recovery steps for lost and changed data
mysql -u root -p
use school;
delete from class where id=6;
delete from class where id=7;
select * from class;
quit
mysqlbinlog --no-defaults /opt/mysql-bin.000002 | mysql -u root -p
mysql -u root -p -e "select * from school.class;"
Simulate recovery steps for all lost data
mysql -u root -p
use school;
drop table class;
show tables;
quit
mysql -uroot -p school < /opt/school_class_2021-02-06.sql
mysqlbinlog --no-defaults /opt/mysql-bin.000002 | mysql -uroot -p
mysql -u root -p -e "select * from school.class;"
Breakpoint recovery
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
#部分二进制文件的内容
......
BEGIN
/*!*/;
##-------------解释:at xxx 表示位置点------------------------------------------------
# at 302
##--------------解释:开头210206 15:45:53表示时间,其他的现在用不到-----------------------------------
#210206 15:45:53 server id 1 end_log_pos 449 CRC32 0xe972def7 Query thread_id=6 exec_time=0 error_code=0
##--------------解释:这里是执行的操作语句---------------------
use `school`/*!*/; <-------------use school;使用数据库
SET TIMESTAMP=1612597553/*!*/; <------------建立时间戳
insert into class values ('6','zzz','男','897656','666666','南京') <-------向表中插入数据
/*!*/;
##---------------------------------------------------------------
# at 449
#210206 15:45:53 server id 1 end_log_pos 480 CRC32 0x5efde826 Xid = 446
COMMIT/*!*/;
# at 480
#210206 15:45:54 server id 1 end_log_pos 545 CRC32 0x11768895 Anonymous_GTID last_committed=1 sequence_number=2 rbr_only=no
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 545
#210206 15:45:54 server id 1 end_log_pos 628 CRC32 0x778ea5fa Query thread_id=6 exec_time=0 error_code=0
SET TIMESTAMP=1612597554/*!*/;
##-------------------------------插入第二个数据--------------------------
BEGIN
/*!*/;
# at 628
#210206 15:45:54 server id 1 end_log_pos 775 CRC32 0x66e3bb53 Query thread_id=6 exec_time=0 error_code=0
SET TIMESTAMP=1612597554/*!*/;
insert into class values ('7','aaa','女','098765','777777','苏州')
/*!*/;
# at 775
#210206 15:45:54 server id 1 end_log_pos 806 CRC32 0x7b972395 Xid = 447
COMMIT/*!*/;
# at 806
#210206 15:48:52 server id 1 end_log_pos 853 CRC32 0x0d77c456 Rotate to mysql-bin.000003 pos: 4
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
.......
Location based recovery
Only restore to the data before the position "628", that is, do not restore the data of "id=7"
#模拟数据丢失
mysql -uroot -p123456 school < /opt/school_class_2021-02-06.sql
mysql -uroot -p123456 -e "select * from school.class;"
#到位置点628停止恢复数据
mysqlbinlog --no-defaults --stop-position='628' /opt/mysql-bin.000002 | mysql -uroot -p123456
#查看class表的数据
mysql -uroot -p123456 -e "select * from school.class;"
Restore only the data of "id=7", skip the data of "id=6"
#模拟数据丢失
mysql -uroot -p123456 school < /opt/school_class_2021-02-06.sql
mysql -uroot -p123456 -e "select * from school.class;"
#从位置点628开始恢复数据
mysqlbinlog --no-defaults --start-position='628' /opt/mysql-bin.000002 | mysql -uroot -p123456
#查看class表的数据
mysql -uroot -p123456 -e "select * from school.class;"
Point-in-time recovery
Only restore the data before 210206 15:45:54, that is, do not restore the data with "id=7"
#模拟数据丢失
mysql -uroot -p123456 school < /opt/school_class_2021-02-06.sql
mysql -uroot -p123456 -e "select * from school.class;"
#到2021-02-06 15:45:54截止恢复数据
mysqlbinlog --no-defaults --stop-datetime='2021-02-06 15:45:54' /opt/mysql-bin.000002 | mysql -uroot -p123456
#查看class表的数据
mysql -uroot -p123456 -e "select * from school.class;"
Only restore the data of "id=7", skip the data restoration of "id=6"
#模拟数据丢失
mysql -uroot -p123456 school < /opt/school_class_2021-02-06.sql
mysql -uroot -p123456 -e "select * from school.class;"
#从2021-02-06 15:45:54开始恢复数据
mysqlbinlog --no-defaults--start-datetime='2021-02-06 15:45:54' /opt/mysql-bin.000002 | mysql -uroot -p123456
#查看class表的数据
mysql -uroot -p123456 -e "select * from school.class;"