Installation and use of xtrabackup-2.4.8
Xtrabackup is a mysql database backup tool provided by percona. According to the official introduction, it is also the only open source tool in the world that can perform hot backup of innodb and xtradb databases. Features:
- (1) The backup process is fast and the physical backup is reliable;
- (2) The backup process will not interrupt the ongoing transaction (no need to lock the table);
- (3) It can save disk space and traffic based on functions such as compression;
- (4) Automatically realize backup inspection;
- (5) The restoration speed is fast;
- (6) The backup can be streamed to another machine;
xtrabackup contains two main tools, namely xtrabackup and innobackupex, the difference between the two is as follows:
Xtrabackup: Only the tables of the innodb and xtradb engines can be backed up, but the tables of the myisam engine cannot be backed up;
Innobackup: It is a Perl script that encapsulates xtrabackup. It supports backing up innodb and myisam at the same time, but a global read lock needs to be added when backing up myisam. Also, myisam does not support incremental backups;
--- Yum installation
Official website address: https://www.percona.com/doc/percona-xtrabackup/LATEST/installation/yum_repo.html
[root@001 ~]# yum install http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm
[root@001 ~]# yum list | grep percona
[root@001 ~]# yum install percona-xtrabackup-24 -y # Uninstall command: yum remove percona-xtrabackup
[root@001 ~]# xtrabackup -version
xtrabackup version 2.4.8 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 97330f7)
--- Use the binary compiled general installation package
Download address: https://www.percona.com/downloads/XtraBackup/LATEST/binary/redhat/6/x86_64/ #Choose your own version
Download address: https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.8/binary/tarball/percona-xtrabackup-2.4.8-Linux-x86_64.tar.gz #This version
[root@001 ~]# wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.8/binary/tarball/percona-xtrabackup-2.4.8-Linux-x86_64.tar.gz
[root@001 ~]# tar xf percona-xtrabackup-2.4.8-Linux-x86_64.tar.gz -C /usr/local/
[root@001 ~]# mv /usr/local/percona-xtrabackup-2.4.8-Linux-x86_64/ /usr/local/xtrabackup
[root@001 local]# echo "export PATH=$PATH:/usr/local/xtrabackup/bin" >> /etc/profile
[root@001 local]# source /etc/profile
[root@001 ~]# yum install numactl -y #Install dependencies
----Full backup
Backup data is stored under /data/backup/, innobackupex will automatically create a folder + timestamp of the current system
Create test database, table
mysql> create database test;
mysql> use test;
mysql> create table t1;
mysql> create table t1(id int auto_increment primary key);
mysql> insert into t1 values(null);
mysql> insert into t1 values(null);
mysql> insert into t1 values(null);
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --socket=/data/mysqldata/mysqld.sock /data/backup/full
#This test directly uses root privileges to attach the command to create users and privileges
mysql> create user 'backup'@'localhost' identified by '123456';
mysql> grant reload,lock tables,replication client,show view,event,process on *.* to 'backup'@localhost;
mysql> plush privileges;
#Backup files: During the backup, the backup data will create a directory with the current date and time as the name under the backup directory to store the backup files.
Description of each file:
(1) backup-my.cnf - configuration option information used by the backup command;
(2) ibdata1 - backed up tablespace file;
(3) xtrabackup_binary - the executable file of xtrabackup used in backup;
(4) xtrabackup_binlog_info - the binary log file currently being used by the mysql server and the location of the binary log event up to the moment of backup;
(5) xtrabackup_checkpoints - backup type (such as full or incremental), backup status (such as whether it has been prepared) and LSN (log sequence number) range information;
6) xtrabackup_logfile - backup redo log file.
You can see that the related files record the LSN, log offset, and you can also see that this time is a full backup
Delete the database, and then restore the full backup (don't do this online)
mysql> drop database test;
full recovery
To restore the data file directory backed up to mysql, this process must first close the mysql database, rename or delete the original data file directory, then create a new data file directory, and copy the backup data to the new data file directory, Authorize, modify permissions, start database
[root@001 ~]# service mysqld stop
[root@001 ~]# mv /data/mysqldata/ /tmp/mysqldata/
[root@001 ~]# mkdir /data/mysqldata
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --use-memory=4G --apply-log /data/2017-08-05_11 -44-06/ #The first step is apply-log. In order to speed up the speed, it is generally recommended to set --use-memory. After this step is completed, the directory is ready
170805 16:49:32 completed OK! #success
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --copy-back /data/2017-08-05_11-33-06/ #Section The second step is copy-back, that is, copy the backup file to the original data directory. Note: innobackupex incremental backup is only for engines that support transactions such as InnoDB. For engines such as MyISAM, it is still full backup
170805 16:49:32 completed OK! #success
You can see that it has been successfully restored, modify the permissions of the data directory, start mysql, and verify whether the data is normal. Check the data in the t1 table under the yayun library.
[root@001 ~]# chown -R mysql.mysql /data/
[root@001 ~]# service mysqld start
Data has been successfully restored
Incremental backup
When performing an incremental backup, a full backup should be performed first. The first incremental backup is based on the full backup, the subsequent incremental backups are based on the last incremental backup, and so on.
Full backups are placed in /data/backup/full, incremental backups are placed in /data/backup/incremental
Full backup first
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --socket=/data/mysqldata/mysqld.sock /data/backup/full
To test the effect, we insert data into the t1 table
mysql> insert into t1 values(null);
Incremental backup again
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --socket=/data/mysqldata/mysqld.sock --incremental /data/backup/incremental/ --incremental-basedir=/data/backup/full/2017-08-05_11-59-39/ --parallel=2
Let's look at the size of the incremental backup and the file contents
Create t2 in test, insert data and then create incremental backup 2
mysql> use test;
mysql> create table t2(name varchar(20));
mysql> insert into t2 values('will');
mysql> insert into t2 values('tom');
mysql> insert into t2 values('jim');
Create incremental backup 2 (this time based on the last incremental backup)
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --user=root --password=123456 --socket=/data/mysqldata/mysqld.sock --incremental /data/backup/incremental/ --incremental-basedir=/data/backup/incremental/ 2017-08-05_12-13-27/ --parallel=2
Incremental backup recovery
The recovery of incremental backup is roughly three steps
* Restore full backup
*Restore the incremental backup to the full backup (add the --redo-only parameter to the incremental backup that starts to restore, and remove the --redo-only parameter to the last incremental backup)
* Restore the entire full backup, roll back those uncommitted data
Restore a full backup (note that the --redo-only parameter must be added here, which means that only the committed transaction data in the xtrabackup log will be applied, and the uncommitted data will not be rolled back)
[root@001 ~]# innobackupex --apply-log --redo-only /data/backup/full/2017-08-05_11-59-39
Apply Incremental Backup 1 to Full Backup
[root@001 ~]# innobackupex --apply-log --redo-only /data/backup/full/2017-08-05_11-59-39 --incremental-dir=/data/backup/incremental/2017-08-05_12-04-59
Apply incremental backup 2 to full backup (note that the --redo-only parameter needs to be removed when restoring the last incremental backup, and the uncommitted data in the xtrabackup log is rolled back)
[root@001 ~]# innobackupex --apply-log --redo-only /data/backup/full/2017-08-05_11-59-39 --incremental-dir=/data/backup/incremental/2017-08-05_12-13-27
Perform an apply operation on all the combined full backups to roll back uncommitted data:
[root@001 ~]# innobackupex --apply-log /data/backup/full/2014-04-07_23-37-20/
Copy the restored backup to the database directory file, empower, and then start the mysql database to check the correctness of the data
[root@001 ~]# service mysqld start
[root@001 ~]# mv /data/mysqldata/ /tmp/mysqldata/
[root@001 ~]# mkdir /data/mysqldata
[root@001 ~]# innobackupex --defaults-file=/etc/my.cnf --copy-back --rsync /data/backup/full/2017-08-05_11-59-39/
[root@001 ~]# chown -R mysql.mysql /data/
clone slave
In daily work, we sometimes need to add slave libraries online. For example, there is one master and one slave database online. However, due to business needs, the reading of one slave library cannot meet the current needs, so we need to add it online. From the library, due to security considerations, we usually need to clone the slave online from the library.
When cloning a slave, the commonly used parameters --slave-info and --safe-slave-backup.
--slave-info will save the master's binlog filename and offset location to the xtrabackup_slave_info file
--safe-slave-backup will suspend the slave SQL thread until the backup starts when there are no open temporary tables. The SQL thread is automatically started after the backup is completed, and the purpose of this operation is to ensure a consistent replication state.
The following example will introduce the online construction of a new slave library in the case of one master and one slave. The environment is as follows:
master 192.168.0.10 #main library
slave 192.168.0.20 #slave library
newslave 192.168.0.100 # New slave library
In the above example, newslave is the slave library to be newly built. Make a backup on the old slave library:
[root@MySQL-02 ~]# innobackupex --user=root --password=12345 --socket=/tmp/mysqld.sock --defaults-file=/etc/my.cnf --slave-info --safe-slave-backup --no-timestamp /data/cloneslave
innobackupex: Backup created in directory '/data/cloneslave'
innobackupex: MySQL binlog position: filename 'mysql-bin.000022', position 107
innobackupex: MySQL slave binlog position: master host '192.168.0.10', filename 'mysql-bin.000006', position 732
140413 23:25:13 innobackupex: completed OK!
The /data/cloneslave directory here does not exist, if it exists, an error will be reported.
View the generated files in the directory:
[root@MySQL-02 ~]# ll /data/cloneslave/
total 26668
-rw-r--r-- 1 root root 261 Apr 13 23:24 backup-my.cnf
-rw-r--r-- 1 root root 27262976 Apr 13 23:24 ibdata1
drwxr-xr-x 2 root root 4096 Apr 13 23:25 mysql
drwxr-xr-x 2 root root 4096 Apr 13 23:25 performance_schema
drwxr-xr-x 2 root root 4096 Apr 13 23:25 sakila
drwxr-xr-x 2 root root 4096 Apr 13 23:25 world_innodb
-rw-r--r-- 1 root root 13 Apr 13 23:25 xtrabackup_binary
-rw-r--r-- 1 root root 23 Apr 13 23:25 xtrabackup_binlog_info
-rw-r--r-- 1 root root 79 Apr 13 23:25 xtrabackup_checkpoints
-rw-r--r-- 1 root root 2560 Apr 13 23:25 xtrabackup_logfile
-rw-r--r-- 1 root root 72 Apr 13 23:25 xtrabackup_slave_info
drwxr-xr-x 2 root root 4096 Apr 13 23:25 yayun
[root@MySQL-02 ~]#
Check the content of the xtrabackup_slave_info file. This content is the parameter that needs to change the master to when building the slave library:
[root@MySQL-02 ~]# cat /data/cloneslave/xtrabackup_slave_info
CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000006', MASTER_LOG_POS=732
[root@MySQL-02 ~]#
Restore on the old slave server, ie 192.168.0.20
[root@MySQL-02 ~]# innobackupex --apply-log --redo-only /data/cloneslave/
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
140413 23:30:37 InnoDB: Starting shutdown...
140413 23:30:37 InnoDB: Shutdown completed; log sequence number 12981048
140413 23:30:37 innobackupex: completed OK!
[root@MySQL-02 ~]#
Copy the restored files to the new slave library newslave, ie 192.168.0.100
[root@MySQL-02 data]# rsync -avprP -e ssh /data/cloneslave/ 192.168.0.100:/data/mysql/
Add authorization to the new slave library newslave on the master library master:
mysql> grant replication slave on *.* to 'repl'@'192.168.0.100' identified by '123456';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.02 sec)
mysql>
Copy the configuration file of the old slave library to the new slave library newslave, and modify the server-id parameter. After the modification, start the new slave library;
[root@MySQL-02 data]# scp /etc/my.cnf 192.168.0.100:/etc/
[email protected]'s password:
my.cnf 100% 4881 4.8KB/s 00:00
[root@MySQL-02 data]#
[root@newslave mysql]# egrep 'log-slave|^server-id|skip_slave' /etc/my.cnf
server-id = 3
skip_slave_start
log-slave-updates=1
[root@newslave mysql]#
[root@newslave mysql]# chown -R mysql.mysql .
[root@newslave mysql]# /etc/init.d/mysqld restart
Shutting down MySQL. [ OK ]
Starting MySQL.. [ OK ]
[root@newslave mysql]#
Find the xtrabackup_slave_info file generated after the old slave library backup, extract the master_log_file and master_log_pos information in it, and then perform the change master to operation on the new slave library:
Sync on the new slave:
mysql> CHANGE MASTER TO MASTER_HOST='192.168.0.10',MASTER_USER='repl', MASTER_PASSWORD='123456',MASTER_LOG_FILE='mysql-bin.000006', MASTER_LOG_POS=732;
Query OK, 0 rows affected (0.09 sec)
mysql>
Start the io thread and sql thread, and observe whether the replication is normal:
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql>
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.0.10
Master_User: repl
Master_Port: 3306
Connect_Retry: 2
Master_Log_File: mysql-bin.000006
Read_Master_Log_Pos: 1309
Relay_Log_File: MySQL-02-relay-bin.000002
Relay_Log_Pos: 830
Relay_Master_Log_File: mysql-bin.000006
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table: yayun.%
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1309
Relay_Log_Space: 989
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
1 row in set (0.00 sec)
mysql>
Check the main library and find that there are already two threads (Binlog Dump)
mysql> show processlist\G
*************************** 1. row ***************************
Id: 8
User: slave
Host: 192.168.0.20:44251
db: NULL
Command: Binlog Dump
Time: 1088
State: Master has sent all binlog to slave; waiting for binlog to be updated
Info: NULL
*************************** 2. row ***************************
Id: 9
User: root
Host: localhost
db: yayun
Command: Query
Time: 0
State: NULL
Info: show processlist
*************************** 3. row ***************************
Id: 10
User: repl
Host: 192.168.0.100:45844
db: NULL
Command: Binlog Dump
Time: 124
State: Master has sent all binlog to slave; waiting for binlog to be updated
Info: NULL
3 rows in set (0.00 sec)
mysql>
It works normally, and the online clone slave is over.