1. The importance of data backup
2. Classification and backup strategy of database backup
2.1 Classification of database backup
1) Physical backup
2) Logical backup
2.2 Database backup strategy
2.3 Common backup methods
3. MySQL full backup
3.1 What is a full backup
3.2 Advantages and disadvantages of full backup
3.3 Method of full backup
1) Physical cold backup and recovery
2) mysqldump backup and recovery
4. Full backup and restore operations
4.1 Physical cold backup
(1) Full backup
(2) Data recovery
(3) Data Migration
4.2 Common methods for logical full backup and recovery
1) Use the xtrabackup tool
2)LVM
3)cp + takes
4)mysqlhotcopy
5) Use mysql master-slave replication
4.3 Use of the logical backup mysqldump tool
(1) Full backup of one or more complete libraries
(2) Fully back up all libraries in the MySQL server (including all tables in the library)
(3) Fully back up some tables in the specified library
(4) View the backup sql file
(5) Restore the entire library of the database
(6) Restore the tables in the database
5. Incremental backup and recovery
5.1 Incremental backup
(1) Reasons for incremental backup
(2) The process of incremental backup
(3) The method of incremental backup
(4) Operation of incremental backup
(5) Incremental backup for data recovery operations
5.2 How to recover data from incremental backup
(1) Restoration based on location points
(2) Point-in-time recovery method
6. Database backup script for full recovery plus incremental recovery
(1) Incremental backup script writing
(2) Full backup script
(3) Add the two written scripts to the scheduled tasks
1. The importance of data backup
The main purpose of backup is disaster recovery.
In a production environment, data security is paramount.
Any loss of data can have serious consequences.
Reasons for data loss:
Program errors, human errors, operation errors, disk failure disasters (such as fires, earthquakes) and theft.
2. Classification and backup strategy of database backup
2.1 Classification of database backup
1) Physical backup
Physical backup: backup of physical files (such as data files, log files, etc.) of the database operating system.
Physical backup method:
Cold backup (offline backup): It is performed when the database is closed
Hot backup (online backup): The database is running, depending on the log file of the database
Warm backup: The status of the database locked table (not writable but readable)
2) Logical backup Logical
backup: Backup of logical components of the database (such as database objects such as tables).
That is, the library, table structure, and table data are saved in the form of SQL statements.
2.2 Database backup strategy
Full backup (full backup): complete backup of the database each time.
Differential Backup: Backs up files that have been modified since the last full backup.
Incremental Backup: Only files that have been modified since the last full or incremental backup will be backed up.
2.3 Common backup methods
Physical cold backup: (full backup)
The database is closed during backup, and the database files are packaged directly.
The backup speed is fast and the recovery is also the simplest.
Special backup tools mydump or mysqlhotcopy (full backup, logical backup)
Mysqldump is a commonly used logical backup tool (exported as sql script)
mysqlhotcopy only has backup MyISAM and ARCHIVE tables
Enable binary logs for incremental backup (incremental backup)
For incremental backup, you need to refresh the binary log
Third-party tool backup
Free MySQL hot backup software Percona XtraBackup
(Aliyun's tool: dts, supports hot migration)
3. MySQL full backup
3.1 What is a full backup? A full
backup is a backup of the entire database, database structure, and file structure. It
saves the database at the moment the backup is completed.
It is the basis for differential backup and incremental backup.
3.2 Advantages and disadvantages of full backup
Advantages:
Backup and restore operations are simple and convenient
Disadvantages:
There is a lot of duplication of data ,
taking up a lot of backup space, and the backup and recovery
time is
long And restore the built-in backup tool of MySQL, which can conveniently realize the backup of MySQL. You can export the specified library and table as SQL script and use the command mysq| to import the backed up data. 4. Complete backup and recovery operation 4.1 Physical cold backup (1) Full backup systemctl stop mysqld #Turn off the service first mkdir /backup/ #Create a backup directory rpm -q xz #Use the xz tool to compress, check whether the xz tool is installed yum install xz -y #If not installed, you can first yum install cd /usr/local/mysql/ tar Jcf /backup/mysql_all_$(date +%F).tar.xz data #Package database files. /usr/local/mysql/data is the database file storage directory
tar tf mysql_all_2022-10-25.tar.xz
(2) Data recovery
and database deletion:
Operations for full recovery:
cd /backup/
tar Jxf mysql_all_2022-10-25.tar.xz -C /opt/
mv /opt/data/ /usr/local/mysql/
Open the database service and log in:
(3) Data Migration
#Host A, use the scp command to transfer the tar package to another host B
scp /backup/mysql_all_2022-10-25.tar.xz [email protected]:/opt
##
Operation of host B ##
systemctl stop mysqld #Close mysql
cd /opt/
mkdir /opt/bak/ #Create a backup directory
tar Jxf mysql_all_2022-10-25.tar.xz -C /opt/bak/ #Decompress the tar package to the backup directory
cd / opt/bak/ #Switch to the decompression directory of the tar package
\cp -af usr/local/mysql/data/ /usr/local/mysql #Copy the data directory to the /usr/local/mysql/ directory and overwrite the original file
systemctl
start mysqld #Start mysql
mysql -u root -p #Log in to the database to view
show databases;
4.2 Common methods of logical full backup and recovery
mysqldump is a commonly used logical backup tool.
mysqldump can export specified libraries and tables as SQL scripts.
1) Use the xtrabackup tool,
which is an open source tool for backing up MySQL databases.
main feature:
Online hot backup. Innodb and myisam can be backed up. Innodb mainly applies the recovery principle. myisam copies files directly.
Streaming backup is supported. It can be backed up to disk, tape and reomot host. --stream=tar ./ | ssh user@remotehost cat ">" /backup/dir/ supports incremental backup. Incremental backups can be performed using lsn and the base backup directory.
Support recording master log and master position information on slave.
It supports simultaneous hot backup of multiple processes, and the stability of xtrabackup is quite good.
2) LVM
features: hot standby, support for all local disk-based storage engines, fast backup, low overhead, easy to maintain integrity, fast recovery, etc.
3) cp + tar
uses the method of directly copying the database file for package backup. It should be noted that the execution steps are: lock the table, backup, and untie the table.
Restoration is also very simple, just copy directly to the storage directory of the previous database file.
Note: For Innodb engine tables, you also need to back up log files, ie ib_logfile* files. Because when the Innodb table is damaged, you can rely on these log files to recover.
4) mysqlhotcopy
mysqlhotcopy is a perl program that uses lock tables, flush tables and cp or scp to quickly back up the database.
It is the fastest way to back up a database or a single table, but it can only run on the machine where the database files (including data table files, data files, and index files) are located.
mysqlhotcopy can only be used to back up MyISAM.
5) Using mysql master-slave replication
Mysql replication refers to transferring the DDL and DML operations of the master database to the slave server through binary files (bin-log), and then re-executing these logs on the slave server, so that Data synchronization is maintained between the slave server and the master server.
4.3 Use of the logical backup mysqldump tool
(1) Fully backup one or more complete databases
mysqldump -uroot -p[password] --databases database name 1 [database name 2].. >/backup path/backup file name.sql #Export is the database script file
(2) Fully back up all libraries in the MySQL server (including all tables in the library)
mysqldump -u root -p[password] --all-databases > / backup path/backup file name.sql
(3) Fully back up some tables in the specified library
mysqldump -u root -p [password] [-d] library name table name 1 [table name 2] ... > /backup path/backup file name.sql #use "
- d" option, indicating that only the table structure of the database is saved
#Do not use the "-d" option, indicating that the table data is also backed up
(4) View the backup sql file
cd /opt/sql_bak
cat file name | grep -v "^--" | grep -v "^/" |grep -v "^$"
or
cat file name | egrep -v " ^--|^/|^$"
(5) Restore the entire library of the database
Delete the library:
mysql -u root -p < /opt/sql_bak/test.sql #Redirect input library file
mysql
-u root -p -e 'SHOW DATABASES;' #Check which databases are currently available
(6) Restore the table in the database
Delete the table:
#Redirect to import the backup file, the library name must be specified, and the target library must exist, if there is no library, the library must be built
mysql -u root -p test < /opt/sql_bak/info_info2.sql
5. Incremental backup and recovery
5.1 Incremental backup
(1) The reason for incremental backup
is the problem of using mysqldump for full backup
There is duplicate data in the backup data
The backup time and recovery time are too long
What is incremental backup:
It is the file or content added/changed since the last backup.
The characteristics of incremental backup
There is no duplicate data, the amount of backup is not large, and the time is short.
Restoration requires the last full backup and all incremental backups after the full backup to restore, and all incremental backups must be reversed one by one.
(2) The process of incremental backup
MySQL does not provide a direct incremental backup method
Incremental backup can be realized indirectly through the binary log provided by MySQL
Significance of MySQL binary log for backup
The binary log saves all operations that update or may update the database.
The binary log starts recording after starting the MySQL server, and recreates a new log file after the file reaches the size set by max_binlog_size or receives the flush logs command. It only needs to execute flush logs
regularly The method recreates a new log, generates a sequence of binary files, and saves these logs in a safe place in time to complete an incremental backup for a period of time
(3) Incremental backup method
General recovery
Restoring all backed up binary log content
based on location
The database may have both wrong operations and correct operations at a certain point in time.
The wrong operation can be skipped based on the precise location
and restored based on the point in time
Skip a certain point in time when an error occurs to achieve data recovery
(4) The operation of incremental backup
needs to modify the main configuration file before incremental backup:
vim /etc/my.cnf
[mysqld]
log-bin=mysql-bin #Enable binary log. If you use a relative path, save it in the /usr/local/mysql/data/ directory
binlog_format = MIXED #Optional, specify the record format of the binary log (binlog) as MIXED
server-id = 1
systemctl
restart mysqld
ls -l / usr/local/mysql/data/mysql-bin.*
The binary log (binlog) has 3 different record formats:
STATEMENT (based on SQL statement), ROW (based on row), MIXED (mixed mode), the default format STATEMENT
STATEMENT (based on SQL statement): record the modified sql statement. In the case of high concurrency, there may be errors in the order of the SQL statements for recording operations, resulting in data loss or errors when restoring data. The efficiency is high, but the data may have errors.
ROW (row-based): Record each row of data, accurate, but inefficient when restoring.
MIXED (mixed mode): Use STATEMENT under normal circumstances, and intelligently switch to ROW under high concurrency. insert
into info values(8,'Xiaolan','female',22,'Wuhan','I like to listen to "Chicken You Are Too Beautiful"'); insert into info values(9,'
Xiaokong','female', 22,'Sichuan','I like to listen to "Hey Hey Hey"');
mysqladmin -u root -p flush-logs #Refresh a new binary file
mysqlbinlog --no-defaults --base64-output=decode-rows -v mysql-bin.000001
#
--base64-output=decode-rows: Use 64-bit encoding mechanism to decode and read by row
#-v: show details
#Create a new folder as the directory where the binary backup file is saved
mkdir -p /opt/sql_bak
#Move the newly generated binary backup file to a new directory (safer way: copy), and modify it by time naming its name
mv /usr/local/mysql/data/mysql-bin.000001 /opt/sql_bak/info_sql_`date +%Y%m%d`
(5) Incremental backup for data recovery operations
delete from info where id=8 ;
delete from info where id=9 ;
Perform an incremental restore:
mysqlbinlog --no-defaults /opt/sql_bak/info_sql_20221027 | mysql -u root -p
5.2 How to recover data from incremental backup
(1) Restoration based on location points
#Start recovery from a certain point until the end of the log
mysqlbinlog --no-defaults --start-position='position point' file name | mysql -u root -p
#Recover
from the beginning of the log to a certain position End mysqlbinlog before the point
--no-defaults --stop-position='position point' file name | mysql -u root -p #Start
from the specified point to the end of the specified point
mysqlbinlog --no-defaults --start- position='xxx'--stop-position='position point' file name | mysql -u root -p
1) Resume from start point to end
mysqlbinlog --no-defaults --start-position=4 mysql-bin.000002 |mysql -u root -p
2) From the specified point to the specified end
mysqlbinlog --no-defaults --start-position=1513 --stop-position=1800 mysql-bin.000003 |mysql -u root -p
(2) Point-in-time recovery method
#Recover from a certain point in time until the end of the log
mysqlbinlog --no-defaults --start-datetime='point in time' file name | mysql -u root -p
#Recover
from the beginning of the log to a certain time End mysqlbinlog before point
--no-defaults --stop-datetime='point in time' file name | mysql -u root -p
#Recover
from a certain point in time, and end
mysqlbinlog before a certain point in time --no -defaults --start-datetime='start time point' --stop-datetime'end time point' file name | mysql -u root -p
6. The advantages of the database backup script combination of complete recovery and incremental recovery
: full backup and incomplete backup have their own advantages, and most companies will use full backup + incremental backup for database backup Full backup: disk It takes up a lot, but the operation is easy and error-free. Incremental backup: The disk occupies a small amount and can be used for short-term backups. However, once there are all incremental backups, the operation difficulty will be greatly increased and error-prone.
Requirements Design: A company has launched a business. The info table is the core data table. For this reason, the table will be backed up during the online time of the business. A full backup is required every Tuesday at 2:30 am (maintenance time), and an incremental backup is performed every day at 1:00 am.
(1) Incremental backup script writing
#!/bin/bash
logs_path="/opt/sql_bak"
[ -d $logs_path ] || mkdir -p $logs_path
/usr/local/mysql/bin/mysqladmin -u root -pabc123 flush-logs
lastlogs=`sed -n '$p' /usr/local/mysql/data/mysql-bin.index | awk -F '/' '{print $2}'`
mv /usr/local/mysql/data/$lastlogs /opt/sql_bak/binlog_`date +%Y%m%d`
(2) Full backup script
#!/bin/bash
home_path="/opt/sql_bak/backup_sql"
[ -d $home_path ] || mkdir -p /opt/sql_bak/backup_sql
/usr/local/mysql/bin/mysqldump -u root -pabc123 test info > /$home_path/infos_$(date +%Y%m%d).sql
/usr/local/mysql/bin/mysqldump -u root -pabc123 --databases test > /$home_path/tests_$(date +%Y%m%d).sql
(3) Add the two written scripts to the scheduled tasks
crontab -e
00 1 * * * /opt/addbak.sh
30 2 * * 2 /opt/backup.sh