MySQL log management, backup and recovery

One, MySQL log management basic commands

MySQL 的日志默认保存位置为 /usr/local/mysql/data

vim /etc/my.cnf
[mysqld]
##错误日志，用来记录当MySQL启动、停止或运行时发生的错误信息，默认已开启
log-error=/usr/local/mysql/data/mysql_error.log     #指定日志的保存位置和文件名

##通用查询日志，用来记录MySQL的所有连接和语句，默认是关闭的
general_log=ON
general_log_file=/usr/local/mysql/data/mysql_general.log

##二进制日志(binlog)，用来记录所有更新了数据或者已经潜在更新了数据的语句，记录了数据的更改，可用于数据恢复，默认已开启
log-bin=mysql-bin      #也可以log_bin=mysql-bin

##慢查询日志，用来记录所有执行时间超过long_query_time秒的语句，可以找到哪些查询语句执行时间长，以便于优化，默认是关闭的
s1ow_query_log=ON
slow_query_log_file=/usr/local/mysql/data/mysql_slow_query.log
long_query_time=5        #设置超过5秒执行的语句被记录，缺省时为10秒

---------------------------------------------------------------
log-error=/usr/local/mysql/data/mysql_error.log
general_log=ON
general_log_file=/usr/local/mysql/data/mysql_general.log
log-bin=mysql-bin
slow_query_log=ON
slow_query_log_file=/usr/local/mysql/data/mysql_slow_query.log
long_query_time=5
----------------------------------------------------------------

systemctl restart mysqld
mysql -u root -P
show variables like 'general%';       #查看通用查询日志是否开启

show variables like 'log_bin%';       #查看二进制日志是否开启

show variables like '%slow%';         #查看慢查询日功能是否开启
show variables like 'long_query_time';    #查看慢查询时间设置

set global slow_query_log=ON;         #在数据库中设置开启慢查询的方法

Second, the importance of data backup

1. The main purpose of backup is disaster recovery.
2. In the production environment, the safety of data is very important.
3. Any loss of data may have serious consequences.
4. Reasons for data loss
(1) program error
(2) Human operation error
(3) Operation error
(4) Disk failure
(5) Disaster (such as fire, earthquake) and theft

Three, the classification of database backup

1. Classification from a physical and logical perspective

(1) Physical backup: backup of the physical files (such as data files, log files, etc.) of the database operating system

Cold backup (offline backup): is performed when the database is closed.
Hot backup (online backup): the database is running and depends on the log file of the database.
Warm backup: the state of the database lock table (not writable but readable) Backup operation

(2) Logical backup: backup of database logical components (such as tables and other database objects)

2. Classification from the perspective of database backup strategy

(1) Full backup: make a complete backup of the database each time
(2) Differential backup: Back up the files that have been modified since the last full backup
(3) Incremental backup: Only after the last full or incremental backup Only the modified files will be backed up

3. Common backup methods

(1) Physical cold standby

The database is closed during the backup, and the database files are directly packaged. The
backup speed is fast and the recovery is also the easiest

(2) Special backup tool mydump or mysqlhotcopy

mysqldump, a commonly used logical backup tool,
mysqlhotcopy, only has backup MyISAM and ARCHIVE tables

(3) Enable binary log for incremental backup

For incremental backups, the binary log needs to be refreshed

(4) Third-party tool backup

Free MySQL hot backup software Percona XtraBackup

Four, MySQL full backup

1. Features

(1) It is the backup of the entire database, database structure and file structure
(2) It saves the database at the time the backup is completed
(3) It is the basis of differential backup and incremental backup

2. Advantages: simple and convenient backup and recovery operations

3. Disadvantages:

(1) Data has a lot of duplication
(2) Takes up a lot of backup space
(3) Backup and recovery time is long

4. Database complete backup classification

(1) Physical cold backup and recovery

Close the MySQL database
Use the tar command to directly package the database folder and
directly replace the existing MySQL directory.

(2) mysqldump backup and recovery

MySQL comes with a backup tool, which can facilitate the backup of MySQL.
You can export the specified library and table as a SQL script.
Use the command mysq| to import the backup data

Five, MySQL backup and restore related commands

环境准备：
use edg;
create table if not exists info1 (
id int(4) not null auto_increment,
name varchar(10) not null,
sex char(10) not null,
hobby varchar(50),
primary key (id));

insert into info1 values(1,'user1','male','running');
insert into info1 values(2,'user2','female','singing');

----------------------------MySQL完全备份与恢复-----------------------------------

----------------MySQL完全备份----------------------
InnoDB 存储引擎的数据库在磁盘上存储成三个文件: db.opt(表属性文件)、表名.frm(表结构文件)、表名.ibd(表数据文件)。

1.物理冷备份与恢复
systemctl stop mysqld
yum -y install XZ
#压缩备份
tar Jcvf /opt/mysql_all_$(date +%F).tar.XZ /usr/local/mysql/data/
#解压恢复
tar Jxvf /opt/mysql_all_2020-11-22.tar.xz -C /usr/local/mysql/data/

2. mysqldump 备份与恢复
(1)、完全备份一个或多个完整的库 (包括其中所有的表)
mysqldump -u root -p[密码] --databases 库名1 [库名2] ... > /备份路径/备份文件名.sql    #导出的就是数据库脚本文件
例:
mysqldump -u root -p --databases edg > /opt/edg.sql       #备份一个edg库
mysqldump -u root -p --databases mysql edg > /opt/mysql-edg.sql    #备份mysql与 edg两个库

(2)、完全备份 MySQL 服务器中所有的库
mysqldump -u root -p[密码] --all-databases > /备份路径/备份文件名.sql
例:
mysqldump -u root -p --all-databases > /opt/all.sql

(3)、完全备份指定库中的部分表
mysqldump -u root -p[密码] 库名 [表名1] [表名2] ... > /备份路径/备份文件名.sql
例:
mysqldump -u root -p [-d] edg info1 info2 > /opt/edg_info1.sql
#使用“-d”选项，说明只保存数据库的表结构
#不使用“-d"选项，说明表数据也进行备份

(4)查看备份文件
grep -v "^--" /opt/edg_info1.sql | grep -v "^/" | grep -v "^$"


-----------------MySQL完全恢复---------------------
systemctl start mysqld

(1)恢复数据库
mysql -u root -P -e 'drop database edg;'
#“-e"选项，用于指定连接MySQL后执行的命令，命令执行完后自动退出
mysql -u root -p -e 'SHOW DATABASES;'

mysql -u root -p < /opt/gcc.sql         #将之前备份的edg数据库进行恢复
mysql -u root -p -e 'SHOW DATABASES;'

(2)、恢复数据表
当备份文件中只包含表的备份，而不包含创建的库的语句时，执行导入操作时必须指定库名，且目标库必须存在。
mysqldump -u root -p edg info1 > /opt/edg_info1.sql    #将表info1进行备份

mysql -u root -p -e 'drop table edg.info1;'      #模拟删除edg库中的info1表
mysql -u root -p -e 'show tables from edg;'

mysql -u root -p edg < /opt/edg_info1.sql        #将edg库中的info1表进行恢复
mysql -u root -p -e 'show tables from edg;'



---------------------MySQL 增量备份与恢复---- ------------------------

----------MySQL 增量备份------------
1.开启二进制日志功能
vim /etc/my.cnf
[mysqld]
log-bin=mysql-bin
binlog_format = MIXED      #可选，指定二进制日志(binlog)的记录格式为MIXED
server-id = 1

#二进制日志(binlog)有3种不同的记录格式: STATEMENT (基于SQL语句)、ROW(基于行)、MIXED(混合模式)，默认格式是STATEMENT

systemctl start mysqld 
ls -l /usr/local/mysql/data/mysql-bin. *

2.可每周对数据库或表进行完全备份
mysqldump -u root -p edg info1 > /opt/edg_info1_$(date +%F).sql
mysqldump -u root -P --all-databases edg > /opt/edg_$(date +%F).sql

3.可每天进行增量备份操作，生成新的二进制日志文件(例如:mysql-bin.000002)
mysqladmin -u root -p flush-logs

4.插入新数据，以模拟数据的增加或变更
use edg;
insert into info1 values(3,'user3','male','game');
insert into info1 values(4,'user4','female','reading');

5.再次生成新的二进制日志文件(例如:mysql-bin.000003)
mysqladmin -u root -p flush-logs
#之前的步骤4的数据库操作会保存到mysql-bin.000002文件中，之后数据库数据再发生变化则保存在mysql-bin.000003文件中

6.查看二进制日志文件的内密
cp /usr/local/mysql/data/mysql-bin.000002 /opt/
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
#--base64-output=decode-rows：使用64位编码机制去解码并按行读取
#-v: 显示详细内容


--------MySQL增量恢复--------
1.一般恢复
(1)、模拟丢失更改的数据的恢复步骤
use edg;
delete from info1 where id=3;
delete from info1 where id=4;

mysqlbinlog --no-defaults /opt/mysql-bin.000002 | mysql -u root -p

(2)、模拟丢失所有数据的恢复步骤
use edg; .
drop table info1;

mysql -u root -p edg < /opt/edg_info1_2020-11-22.sql
mysqlbinlog --no-defaults / opt/mysql-bin.000002 | mysql -u root -p

2.断点恢复
mysqlbinlog --no-defaults --base64-output=decode-rows -v /opt/mysql-bin.000002
例:
# at 302
#201122 16:41:16
插入了"user3"的用户数据

# at 623
#201122 16:41:24 
插入了"user4"的用户数据

(1)、基于位置恢复
#仅恢复到操作 ID 为“623"之前的数据，即不恢复"user4"的数据
mysqlbinlog --no-defaults --stop-position='623' /opt/mysql-bin.000002 | mysql -uroot -p

#仅恢复"user4"的数据，跳过"user3"的数据恢复
mysqlbinlog --no-defaults --start-position='623' /opt/mysql-bin.000002 | mysql -uroot -p

mysqlbinlog --no-defaults --start-position='400' --stop-position='623' /opt/mysql-bin.000002 | mysql -uroot -p      #恢复从位置为400开始到位置为623为止


(2)、基于时间点恢复
mysqlbinlog [--no-defaults] --start-datetime='年-月-日 小时:分钟:秒' --stop-datetime='年-月-日小时:分钟:秒' 二进制日志 | mysql -u 用户名 -p 密码

#仅恢复到16:41:24 之前的数据，即不恢复"user4"的数据
mysqlbinlog --no-defaults --stop-datetime='2020-11-22 16:41:24' /opt/mysql-bin.000002 | mysql -uroot -p 

#仅恢复"user4"的数据，跳过"user3"的数据恢复
mysqlbinlog --no-defaults --start-datetime='2020-11-2216:41:24' /opt/mysql-bin.000002 | mysql -uroot -p



如果恢复某条SQL语之前的所有数据，就stop在这个语句的位置节点或者时间点
如果恢复某条SQL语句以及之后的所有数据，就从这个语句的位置节点或者时间点start