MySQL backup, recovery and log management, simplified management of MySQL

One, MySQL log management basic commands

MySQL 的日志默认保存位置为 /usr/local/mysql/data

vim /etc/my.cnf
[mysqld]
##错误日志，用来记录当MySQL启动、停止或运行时发生的错误信息，默认已开启
log-error=/usr/local/mysql/data/mysql_error.log     #指定日志的保存位置和文件名

##通用查询日志，用来记录MySQL的所有连接和语句，默认是关闭的
general_log=ON
general_log_file=/usr/local/mysql/data/mysql_general.log

##二进制日志(binlog)，用来记录所有更新了数据或者已经潜在更新了数据的语句，记录了数据的更改，可用于数据恢复，默认已开启
log-bin=mysql-bin      #也可以log_bin=mysql-bin

##慢查询日志，用来记录所有执行时间超过long_query_time秒的语句，可以找到哪些查询语句执行时间长，以便于优化，默认是关闭的
s1ow_query_log=ON
slow_query_log_file=/usr/local/mysql/data/mysql_slow_query.log
long_query_time=5        #设置超过5秒执行的语句被记录，缺省时为10秒

---------------------------------------------------------------
log-error=/usr/local/mysql/data/mysql_error.log
general_log=ON
general_log_file=/usr/local/mysql/data/mysql_general.log
log-bin=mysql-bin
slow_query_log=ON
slow_query_log_file=/usr/local/mysql/data/mysql_slow_query.log
long_query_time=5
----------------------------------------------------------------

systemctl restart mysqld
mysql -u root -P
show variables like 'general%';       #查看通用查询日志是否开启

show variables like 'log_bin%';       #查看二进制日志是否开启

show variables like '%slow%';         #查看慢查询日功能是否开启
show variables like 'long_query_time';    #查看慢查询时间设置

set global slow_query_log=ON;         #在数据库中设置开启慢查询的方法

Second, the importance of data backup

1. The main purpose of backup is disaster recovery.
2. In the production environment, the safety of data is very important.
3. Any loss of data may have serious consequences.
4. Reasons for data loss.

• Program error
• Human error
• Operation error
• Disk failure
• Disasters (e.g. fires, earthquakes) and theft

Three, the classification of database backup

(1) From a physical and logical perspective, backup can be divided into

1. Physical backup: backup of physical files (such as data files, log files, etc.) of the database operating system

Physical backup method

• Cold backup (offline backup): It is performed when the database is closed
• Hot backup (online backup): the database is running and depends on the log file of the database
• Warm backup: the database is backed up when the table is locked (not writable but readable)

2. Logical backup: backup of database logical components (such as tables and other database objects)

(2) From the perspective of database backup strategy, backup can be divided into

1. Full backup: make a complete backup of the database each time.
2. Differential backup: back up the files that have been modified since the last full backup.
3. Incremental backup: only the files that have been modified since the last full or incremental backup. Files will be backed up

(3) Common backup methods

1. Physical cold standby

• The database is closed during backup, and the database files are directly packaged
• Fast backup speed and easiest to restore

2. The dedicated backup tool mydump or mysqlhotcopy

• mysqldump Commonly used logical backup tool
• mysqlhotcopy only has backup MyISAM and ARCHIVE tables

3. Enable binary log for incremental backup

• For incremental backup, you need to refresh the binary log

4. Third-party tool backup

• Free MySQL hot backup software Percona XtraBackup

Four, MySQL full backup

• It is a backup of the entire database, database structure and file structure
• Save the database at the time the backup is completed
• It is the basis of differential backup and incremental backup

1. Advantages:

• Simple and convenient operation of backup and recovery

2. Disadvantages:

• There is a lot of duplication in data
• Takes up a lot of backup space
• Long backup and restore time

(1) Classification of complete database backups

1. Physical cold backup and recovery

• Shut down the MySQL database
• Use the tar command to directly package the database folder
• Simply replace the existing MySQL directory

2. mysqldump backup and recovery

• MySQL's own backup tool can easily realize the backup of MySQL
• Can export specified libraries and tables as SQL scripts
• Use the command mysq| to import the backup data

Five, MySQL backup and restore related commands

环境准备：
use gcc;
create table if not exists info1 (
id int(4) not null auto_increment,
name varchar(10) not null,
sex char(10) not null,
hobby varchar(50),
primary key (id));

insert into info1 values(1,'user1','male','running');
insert into info1 values(2,'user2','female','singing');

----------------------------MySQL完全备份与恢复-----------------------------------

----------------MySQL完全备份----------------------
InnoDB 存储引擎的数据库在磁盘上存储成三个文件: db.opt(表属性文件)、表名.frm(表结构文件)、表名.ibd(表数据文件)。

1.物理冷备份与恢复
systemctl stop mysqld
yum -y install XZ
#压缩备份
tar Jcvf /opt/mysql_all_$(date +%F).tar.XZ /usr/local/mysql/data/
#解压恢复
tar Jxvf /opt/mysql_all_2020-11-22.tar.xz -C /usr/local/mysql/data/

2. mysqldump 备份与恢复
(1)、完全备份一个或多个完整的库 (包括其中所有的表)
mysqldump -u root -p[密码] --databases 库名1 [库名2] ... > /备份路径/备份文件名.sql    #导出的就是数据库脚本文件
例:
mysqldump -u root -p --databases gcc > /opt/gcc.sql       #备份一个gcc库
mysqldump -u root -p --databases mysql gcc > /opt/mysql-gcc.sql    #备份mysql与 gcc两个库

(2)、完全备份 MySQL 服务器中所有的库
mysqldump -u root -p[密码] --all-databases > /备份路径/备份文件名.sql
例:
mysqldump -u root -p --all-databases > /opt/all.sql

(3)、完全备份指定库中的部分表
mysqldump -u root -p[密码] 库名 [表名1] [表名2] ... > /备份路径/备份文件名.sql
例:
mysqldump -u root -p [-d] gcc info1 info2 > /opt/gcc_info1.sql
#使用“-d”选项，说明只保存数据库的表结构
#不使用“-d"选项，说明表数据也进行备份

(4)查看备份文件
grep -v "^--" /opt/gcc_info1.sql | grep -v "^/" | grep -v "^$"


-----------------MySQL完全恢复---------------------
systemctl start mysqld

(1)恢复数据库
mysql -u root -P -e 'drop database gcc;'
#“-e"选项，用于指定连接MySQL后执行的命令，命令执行完后自动退出
mysql -u root -p -e 'SHOW DATABASES;'

mysql -u root -p < /opt/gcc.sql         #将之前备份的gcc数据库进行恢复
mysql -u root -p -e 'SHOW DATABASES;'

(2)、恢复数据表
当备份文件中只包含表的备份，而不包含创建的库的语句时，执行导入操作时必须指定库名，且目标库必须存在。
mysqldump -u root -p gcc info1 > /opt/gcc_info1.sql    #将表info1进行备份

mysql -u root -p -e 'drop table gcc.info1;'      #模拟删除gcc库中的info1表
mysql -u root -p -e 'show tables from gcc;'

mysql -u root -p gcc < /opt/gcc_info1.sql        #将gcc库中的info1表进行恢复
mysql -u root -p -e 'show tables from gcc;'



---------------------MySQL 增量备份与恢复---- ------------------------

----------MySQL 增量备份------------
1.开启二进制日志功能
vim /etc/my.cnf
[mysqld]
log-bin=mysql-bin
binlog_format = MIXED      #可选，指定二进制日志(binlog)的记录格式为MIXED
server-id = 1              #可加可不加该命令

#二进制日志(binlog)有3种不同的记录格式: STATEMENT (基于SQL语句)、ROW(基于行)、MIXED(混合模式)，默认格式是STATEMENT

systemctl start mysqld 
ls -l /usr/local/mysql/data/mysql-bin. *

2.可每周对数据库或表进行完全备份
mysqldump -u root -p gcc info1 > /opt/gcc_info1_$(date +%F).sql
mysqldump -u root -P --all-databases gcc > /opt/gcc_$(date +%F).sql

3.可每天进行增量备份操作，生成新的二进制日志文件(例如:mysql-bin.000002)
mysqladmin -u root -p flush-logs

4.插入新数据，以模拟数据的增加或变更
use gcc;
insert into info1 values(3,'user3','male','game');
insert into info1 values(4,'user4','female','reading');

5.再次生成新的二进制日志文件(例如:mysql-bin.000003)
mysqladmin -u root -p flush-logs
#之前的步骤4的数据库操作会保存到mysql-bin.000002文件中，之后数据库数据再发生变化则保存在mysql-bin.000003文件中

6.查看二进制日志文件的内密
cp /usr/local/mysql/data/mysql-bin.000002 /opt/
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
#--base64-output=decode-rows：使用64位编码机制去解码并按行读取
#-v: 显示详细内容


--------MySQL增量恢复--------
1.一般恢复
(1)、模拟丢失更改的数据的恢复步骤
use gcc;
delete from info1 where id=3;
delete from info1 where id=4;

mysqlbinlog --no-defaults /opt/mysql-bin.000002 | mysql -u root -p

(2)、模拟丢失所有数据的恢复步骤
use gcc; .
drop table info1;

mysql -u root -p gcc < /opt/gcc_info1_2020-11-22.sql
mysqlbinlog --no-defaults / opt/mysql-bin.000002 | mysql -u root -p

2.断点恢复
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
例:
# at 302
#201122 16:41:16
插入了"user3"的用户数据

# at 623
#201122 16:41:24 
插入了"user4"的用户数据

(1)、基于位置恢复
#仅恢复到操作 ID 为“623"之前的数据，即不恢复"user4"的数据
mysqlbinlog --no-defaults --stop-position='623' /opt/mysql-bin.000002 | mysql -uroot -p

#仅恢复"user4"的数据，跳过"user3"的数据恢复
mysqlbinlog --no-defaults --start-position='623' /opt/mysql-bin.000002 | mysql -uroot -p

mysqlbinlog --no-defaults --start-position='400' --stop-position='623' /opt/mysql-bin.000002 | mysql -uroot -p      #恢复从位置为400开始到位置为623为止


(2)、基于时间点恢复
mysqlbinlog [--no-defaults] --start-datetime='年-月-日 小时:分钟:秒' --stop-datetime='年-月-日小时:分钟:秒' 二进制日志 | mysql -u 用户名 -p 密码

#仅恢复到16:41:24 之前的数据，即不恢复"user4"的数据
mysqlbinlog --no-defaults --stop-datetime='2020-11-22 16:41:24' /opt/mysql-bin.000002 | mysql -uroot -p 

#仅恢复"user4"的数据，跳过"user3"的数据恢复
mysqlbinlog --no-defaults --start-datetime='2020-11-2216:41:24' /opt/mysql-bin.000002 | mysql -uroot -p



如果恢复某条SQL语之前的所有数据，就stop在这个语句的位置节点或者时间点
如果恢复某条SQL语句以及之后的所有数据，就从这个语句的位置节点或者时间点start