MySQL backup tools mysqldump and --xtrabackup

MySQL backup using mysqldump ---- xtrabackup and tools

A, Xtrabackup8.0:

A physical backup tool MySQL database hot backup for support MySQL, Percona server and MariaDB, a free open source, is the more popular mainstream backup tool. xtrabackup only two data backup innoDB and xtraDB engine tables, but can not back up MyISAM tables.

Backup principle:

Works Percona XtraBackup to remember that the log sequence number (LSN) at startup, and then copy the data files. Meanwhile, Percona XtraBackup run a background detection process that monitors mysq redo the transaction log file, and copy changes from which credited the log file xtrabackup_log in the background. Innodb data files after copying a file system table space ibdatax, after the end of the copy will be executed flush tables with readlock, and then copy the .frm MYI MYD and other documents, the final implementation unlock tables, the final stop xtrabackup_log

Xtrabackup (centos / redhat) installation:

Method One (official recommended):
1. Download the appropriate rpm installation package
wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-8.0.4/binary/redhat/7/x86_64/percona-xtrabackup- 80-8.0.4-1.el7.x86_6 4.rpm

2. Install the downloaded rpm package
yum localinstall percona-xtrabackup-80-8.0.4-1.el7.x86_64.rpm

3. Check whether the success of the installed
rpm -qa | grep -i xtrabackup
relevant documents queries
rpm -ql percona-xtrabackup-80

4. Create a backup directory

5. Modify profile mysql
open bin-log log, add or modify a database directory

Method Two (the official recommended):

1. Installation Percona yum repository
   yum install https://repo.percona.com/yum/percona-release-latest.noarch.rpm

2. 安装Percona XtraBackup
   yum install percona-xtrabackup-80

Method three:

1. Download the binary package
   wget
   https://www.percona.com/downloads/Percona-XtraBackup-LATEST/Percona-XtraBackup-8.0-7/binary/tarball/percona-xtrabackup-8.0.7-Linux-x86_64.libgcrypt20.tar. gz
   

2. 解压
   tar zxvf percona-xtrabackup-8.0.7-Linux-x86_64.libgcrypt20.tar.gz
   mv percona-xtrabackup-8.0.7-Linux-x86_64 xtrabackup
   cp -r xtrabackup /usr/local/xtrabackup

3. Add the environment variable
   echo "Export the PATH = $ the PATH: / usr / local / xtrabackup / bin">
   /etc/profile.d/xtrabackup.sh

4. Let environment variables to take effect
   source /etc/profile.d/xtrabackup.sh

Xtrabackup advantages:

1. Backup speed, reliable physical backup
   
2. The backup process will not interrupt the transaction (no need to lock table) is being executed
3. Can save disk space and traffic-based compression capabilities
4. Automatic backup verification
5. Restore speed (do not need to perform any of the mysql statement, without indexing, innoDB table without having to completely cached in memory)
6. Backup transmission can be circulated to another machine
7. Back up data without increasing the load on the server

Xtrabackup Disadvantages:

1. Large file
2. Can not always cross-platform, operating system and MySQL version

Note that the new version 8.0:

(1) removing the innobackupex command;
(2) Percona XtraBackup MySQL8.0 version 8.0 support;
prior to (3) mysql8.0 version requires xtrabackup2.4 backup and recovery.

Xtrabackup permissions:

Xtrabackup create a database user with minimal privileges required for full backups:
MySQL> the CREATE the USER 'bkpuser' @ 'localhost' IDENTIFIED BY 's3cr% T';
MySQL> GRANT BACKUP_ADMIN, the PROCESS, RELOAD, LOCK TABLES, REPLICATION CLIENT ON . The TO 'bkpuser' @ 'localhost';
MySQL> the GRANT the SELECT the ON performance_schema.log_status the TO 'bkpuser' @ 'localhost';
MySQL> the FLUSH PRIVILEGES;

note:

Once connected to the server, you perform a backup of the file system level permissions datadir need to READ and EXECUTE on the server.
If there are multiple server instances, you must specify the correct connection parameters (port, socket, host), for communicating with the correct server xtrabackup.

Xtrabackup full backup:

xtrabackup --defaults-file=/etc/my.cnf --host=127.0.0.1 --user=root --password=xxxyyy --port=3306 --backup --target-dir=/backup/full/

Optimization options:

--rsync
use rsync utility to optimize local file transfers. After you specify this option, xtrabackup will use rsync to copy all non-InnoDB file, instead of generating a separate cp for each file, which can be faster for a server with a large database or table. This option can not be used with --stream.
--use-memory
will affect how much memory assigned to this option to use for backup
--parallel = #
This option specifies the number of threads created to simultaneously copy multiple data files during backup. The default value is 1 (i.e., without concurrent transmissions).

Xtrabackup full amount of recovery:

1. Prior to restore a backup, datadir must be empty. It is also important before performing recovery, MySQL server needs to close. You can not be restored to the running instance of mysqld DATADIR (except when backup lead-in portion).

2. Prepare a full backup, a major role is the use of data files have data files in a consistent state by rolling back uncommitted transactions and transaction synchronization has been submitted to.
   xtrabackup --defaults-file = / etc / my.cnf --user = root --password = xxxyyy --prepare (--apply-log-only) - target-dir = / backup / full /

3. When the data is restored to the data directory later, you should check whether the recovered file has the correct ownership and permissions. Since the file attributes are preserved, in most cases, before starting the database server, you need to change the ownership of files for mysql, because they will have to create a backup of the user:
   xtrabackup --defaults-File = / etc / My. --user = the root --password = CNF XXXYYY --copy-Back---target the dir = / Backup / Full /
   chown -R & lt MySQL: MySQL / var / lib / MySQL

4. Restart the mysql service

note:

If you plan to back up as the basis for further incremental backup, you should use -apply-log-only option when you are backing up, otherwise it will not apply the incremental backup

Xtrabackup parameters introduced:

--defaults-file
is specified mysql configuration file: The default option can only be read from the given file. And must be used as the first option on the command line; must be an authentic document, it can not be a symbolic link.

--target-dir
specifies the directory backup data

--prepare
data file is not consistent until you have a point in time, because they are replicated at different times running, and may have been changed in this process, this step makes exactly the same file at the same time

--backup
make a backup into --target-dir

--copy-back
copy all of the files previously backed up to their original path. But you can not have any files or directories under the original path, unless you specify --force-non-empty-directories option.

Specify the database backup:

xtrabackup --defaults-file=/etc/my.cnf --host=127.0.0.1 --user=root --password=xxxyyy --port=3306 --databases=”xxx” --backup --target-dir=/part/single

Specifies the database recovery:

Prepare partial backup process similar to the process of export table, --export option to use the process to restore parts of the backup process with the same import table. Of course, the reduction can also be achieved by direct copying backup data prepared state directly to the directory, the directory data is not required at this time in a consistent state.

1. xtrabackup --defaults-file=/etc/my.cnf --prepare --export --target-dir=/part/single
   
2. Created and corresponding database table structure
3. Discarding table space
   alter table dbname.tbname discard tablespace
   
4. Copy the file to a subdirectory of the exported table in the corresponding database directory of the target server
   cp /part/single/dbname/*.ibd / var / bin / mysql / dbname /
   
5. The document re-authorization
   chown -R mysql: mysql / var / lib / mysql
   
6. Loading table space
   alter table dbname.tbname import tablespace;

--export
files needed to create the export table

note:

To innodb_file_per_table open, if the innodb storage engine data, delete data if it is so, there will still be under the datadir .ibd, .frm file, if a drop of data, then our table to be restored manually perform the construction of the table statement. cfg file is used and the data dictionary to match.

Xtrabackup incremental backup:

The main change is incremental backup by copying the pages in innodb (LSN is greater than the mean number LSN in xtrabackup_checkpoints). Xtrabackup_checkpoints file will record a checkpoint when the backup is completed LSN, incremental backup is based on data bountiful, first incremental backup is a full backup based on, after each increase by backup equipment are based on time , ultimately increasing the consistency of preparation, prepared by the process, and all equipment is similar, except that the second step

The first incremental backup:

xtrabackup --defaults-file=/etc/my.cnf --host=127.0.0.1 --port=3306 --user=root --password=123456 --backup --target-dir=/incr/incr1
--incremental-basedir=/data/backup/

Second incremental backup:

xtrabackup --defaults-file=/etc/my.cnf --host=127.0.0.1 --port=3306 --user=root --password=123456 --backup --target-dir=/part/incr2
--incremental-basedir=/incr/incr1

note:

Note that, an incremental backup can only be applied InooDB XtraDB or table, for MyISAM tables, all equipment and the same increment.

Xtrabackup incremental recovery:

1. Preparing a full backup, full backup data to ensure data consistency directory
   xtrabackup --defaults-file = / etc / my.cnf --prepare --apply-log-only --target-dir = / data / backup

2. Incremental backups to restore full backups (incremental backups began to recover to add --apply-log-only parameters, to the last incremental backup to remove --apply-log-only), even if --apply-log- only in the last incremental backup, the backup will remain the same, but in this case, the server will perform rollback phase.
   File---defaults = xtrabackup / etc / --apply the my.cnf --prepare --target-log-only-the dir = / Data / Backup / --incremental-the dir = / incr / INCR1
   xtrabackup --defaults-File = / etc / my.cnf --prepare --target- dir = / data / backup / --incremental-dir = / incr / incr2

3. The final data is in / data / backup /, the application of a new backup incremental backup restore
   xtrabackup --user --password = XXXYYY --copy = the root-Back---target the dir = / data / backup /
   chown -R mysql: mysql / var / lib / mysql

note:

Step --prepare incremental backup and full backup of different steps. In a full backup, perform two types of operations in order to make the database consistent: the transaction has been submitted with respect to the data file playback from the log file, uncommitted transactions are rolled back. In preparation for an incremental backup, you must skip the rollback of uncommitted transactions because the transaction when the backup uncommitted may be in progress, and likely will be submitted to the next incremental backup. You should use this --apply-log-only option to prevent the rollback phase.

Xtrabackup parameters introduced:

--apply-log-only
option to redo only lead to the implementation stage in preparation for backup. This is important for an incremental backup.

--target-dir
backup: Incremental backup directory
Recovery: In preparation for incremental backups, which are incremental backups and full backups combined to create a new full backup of the directory.

--incremental-basedir
when creating incremental backups, this directory contains the complete backup that basic data sets incremental backup.

-Apply-log-only
prevent the transaction rollback phase

Each backup directory files Description:

(1) xtrabackup_checkpoints - backup type (e.g., full or incremental), backup status (e.g., whether as prepared state) and the LSN (Log Sequence Number) range information.
Each InnoDB page (typically 16k size) will contain a log sequence number, i.e. LSN. LSN is the system version of the database system, each page related to this page LSN able to show how the recent change of.
(2) xtrabackup_binlog_info - mysql server binary log files currently in use up until the moment the backup location binary log events.
(3) xtrabackup_info - records are some of the parameters when you backup script version, database version, backup time, binlog, whether compression, incremental backups or full amount and other information.
(4) backup-my.cnf - contains information needed my.cnf backup.
(5) xtrabackup_logfile-- recording the backup log, the log restoring the data needs to conform to the available data at the time of data prepare.
(6) Other information database files
Sometimes you can see to_lsn (the last checkpoint LSN) and last_lsn there is a difference between (LSN last copy), which means there are some traffic on the server during the backup process

xtrabackup whole backup process:

1. Real-time backup redo log

• xtrabackup open xtrabackup_log monitoring thread
• LSN records began, and real-time monitoring of changes redo log files
• The backup log is written to the new page in the transaction log file copied to xtrabackup_logfile;

1. Innodb file copy

• When you turn on redo log backup, open xtrabackup copy thread, innodb start copying files.
• Copy order:
• ibd file ibdata1, sys_config.ibd, mysql.ibd, as well as other datadir directory
• ibd in the database directory, test libraries, employees, sakila etc.
• undo file

1. After copying innodb file, LOCK TABLES FOR BACKUP

• Non-locking innodb table

1. Non-innodb file copy

• Copy mysql, .frm under p_s and other libraries, .MRG, .MYD, .MYI, .ARM, .ARZ, .CSM, .CSV, .sdi and .par and other types of files.

1. Get LSN and binary log position

• At this end of the file backup, and you need to get LSN binlog position
• LOCK BINLOG FOR BACKUP
• To prevent all possible to change the position of the operating binlog
• Then complete copy REDO log file and obtain the coordinates binlog
• Get information from the p_s.log_status
• binlog file recorded xtrabackup_binlog_info
• LSN recorded xtrabackup_checkpoints

1. FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS

• Close and reopen any refresh the log storage engine has been installed, not recorded binlog.
• This causes InnoDB to log flushed to disk.

1. Stop xtrabackup_log thread
2. Unlock tables, binlog
3. Additional information backup

• Copying ib_buffer_pool
• Writing backup-my.cnf
• Writing xtrabackup_info

1. binlog position will print to STDERR, if everything is normal, xtrabackup will exit returns 0.

二、mysqldump:

Backup principle:

Export SQLy statement file

mysqldump advantages:

1. Recovery simple, they can use the pipe input to mysql
2. Regardless of the storage engine, because the data are extracted from the generated MySQL server, thus eliminating the underlying data stored in different
3. It helps prevent data corruption. If the disk drive has failure to copy the original document, this time will be backed up in a corrupted

mysqldump Disadvantages:

1. There must be a logical database server to complete the work, you need to be more cpu cycles

2. Logical backup and restore speed is slow: MySQL required to load and interpret statements converted storage format, reconstruction engine

   mysqldump full backup:

   mysqldump must bring --flush-logs command option to generate a new binary log files
   mysqldump -h127.0.0.1 -P3306 -uroot -p --single-transaction --flush-logs --master-data = 2 - all-database> /backup/all.sql

   Parameter Description

   For MyISAM replace the -single-transaction-All-Tables -lock
   --flush to end the current log-logs, generates a new log file;
   after --master-data = 2 at the output of the SQL option will record the new full backup log file name; its starting position and pos
   --all-database all databases
   --master-Data = [0 | 1 | 2]
   0: not record
   1: CHANGE MASTER statement is recorded as
   2: CHANGE MASTER recorded as annotations statement
   --single-transaction the option to submit a BEGIN SQL statement before exporting the data, BEGIN does not block any applications and can ensure the consistency of the database status when exported. It applies only to multi-storage engine version, only InnoDB

mysqldump full amount of recovery:

mysql -h127.0.0.1 -P3306 -uroot -p < /backup/all.sql

mysqldump designated backup:

mysqldump -h127.0.0.1 -P3306 -uroot -p --single-transaction --flush-logs --master-data=2 dbname > /backup/part.sql

mysqldump specified database recovery:

mysql -h127.0.0.1 -P3306 -uroot -pxxxx < /backup/part.sql

mysqldump incremental backup:

Incremental backup data is updated since the last full backup data, mysql binlog log file is stored in the data incremental backup. binlog is a binary file format for recording user information database update SQL statement.

mysqladmin -uroot -pxxxx flush-logs

note:

Prerequisites perform incremental backups of MySQL is open binlog log function, add log-bin = mysql-bin in the my.cnf mysql configuration file

mysqldump incremental recovery:
mysqlbinlog MySQL-bin.0000xx | MySQL-uroot--p

data recovery mysqldump the specified time or location:
--start = datetime-mm-dd-YYYY HH: the MM: the SS
--stop = datetime-mm-dd-YYYY HH: the MM: the SS
upper specifies a set of parameters for the recovery start time point and end time point may be given alone or together

XXXXX-position = --start
--stop XXXXX-position =
given a set of parameters used to specify the upper restoration start position and end position, together or separately

Two sets of parameters may also be used in combination

Three, mysqldump contrast with xtrabackup

Fourth, the full backup and restore performance test comparison:

tool	mysqldump	xtrabackup
The amount of data	1G around	1G around
Full backup time	Average 1m15s	Average 20s
Restore full backup time	The average 8m	Average 18s
take up space	617MB	930MB

tool	mysqldump	xtrabackup
The amount of data	About 5G	About 5G
Full backup time	Average 3m50s	Average 1m10s
Restore full backup time	Average 45m	Average 1m5s
take up space	3.3GB	4G

Fifth, the backup period:

Done once a week full backup, incremental backup or a month rest of the time to do a full backup, incremental backup once a week, relying on system timing tasks, the business pressure is relatively small point in time in the morning (0 - 5 points), time to perform backups script

Sixth, the backup file maintenance

If the full backup, you can delete the unit of days
if the full amount + incremental backup, regularly delete (recommended period of one month) is obsolete backup files