Detailed explanation of the use of MySQL backup and recovery tool XtraBackup in Docker environment | Spring Cloud 62

1. Introduction to XtraBackup

Percona XtraBackupis an open source MySQLand MariaDBdatabase backup tool that creates high-performance, consistent backups with minimal impact on production environments. Backup and restore operations are implemented by copying the storage engine's data files and transaction logs Percona XtraBackupwithout stopping MySQLthe server .InnoDB

Percona XtraBackupThe main features are as follows:

• High-performance backup: Percona XtraBackupAbility to back up databases in parallel, providing fast backup speeds with less impact on production systems.

• Consistency backup: Percona XtraBackupUse InnoDBa specific algorithm to ensure the consistency of the backup, that is, the backup data file and transaction log are in the same state at the same time.

• Incremental backup: Percona XtraBackupIncremental backup is supported, and incremental backups can be created based on previous full backups, reducing backup time and storage space consumption.

• Support compression and encryption: Percona XtraBackupYou can use a compression algorithm to reduce the size of the backup file during the backup process, and you can also encrypt the backup file to improve data security.

• Flexible backup and recovery: Percona XtraBackupIt supports copying backup files back to the database directory for recovery, and also supports outputting backup data in stream mode for recovery.

• Compatibility: Percona XtraBackupCompatible MySQLwith MariaDBand can be used on different versions of the database.

In short, Percona XtraBackupis a powerful and flexible database backup tool, which can help database administrators to easily create reliable backups and perform efficient restore operations when needed. Whether it is data protection for production environments or data replication for development and test environments, Percona XtraBackupit is a reliable choice.

Official website address: https://docs.percona.com/percona-xtrabackup/8.0/

2. Environment and deployment

2.1 Introduction to the operating environment

All demonstration operations in this chapter are in Dockerthe environment.

• dockerVersion20.10.6
• docker-composeVersion1.28.5

2.2 Mysql and XtraBackup configuration

The directory structure is as follows:

MysqlConfiguration file my.cnf:

# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M

# Remove leading # to revert to previous value for default_authentication_plugin,
# this will increase compatibility with older clients. For background, see:
# https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_default_authentication_plugin
# default-authentication-plugin=mysql_native_password
default-time_zone='+8:00'
log-bin
log_bin_trust_function_creators=1
lower_case_table_names=1
skip-host-cache
skip-name-resolve
datadir=/var/lib/mysql
socket=/var/run/mysqld/mysqld.sock
secure-file-priv=/var/lib/mysql-files
user=mysql
innodb_undo_tablespaces=2
innodb_read_only=off
pid-file=/var/run/mysqld/mysqld.pid
[client]
socket=/var/run/mysqld/mysqld.sock

# !includedir /etc/mysql/conf.d/

docker-compose.yml：

version: "3.8"
# 通用日志设置
x-logging:
    &default-logging
    # 日志大小和数量
    options:
        max-size: "100m"
        max-file: "3"
    # 文件存储类型
    driver: json-file
services:
  mysql:
    container_name: mysql
    image: mysql:8.0.32
    user: 999:999
    environment:
      - MYSQL_ROOT_PASSWORD=1qaz@WSX
    volumes:
      - data_volume:/var/lib/mysql
      - backup_volume:/backup
      - /root/apps/mysql/my.cnf:/etc/my.cnf:ro
      - /etc/timezone:/etc/timezone:ro
    logging: *default-logging
    ports:
      - "3306:3306"
  xtrabackup-command:
    container_name: xtrabackup-command-line
    image: percona/percona-xtrabackup:8.0.32
    user: 999:999
    restart: always
    depends_on:
      - mysql
    volumes_from:
      - mysql
    command: tail -f /dev/null
volumes:
  data_volume:
    driver: local
  backup_volume:
    driver: local

Note 1:

• xtrabackup: Please keep XtraBackupversion and Mysqlversion consistent.
• volumes_from: Indicates that docker mysqlthe container uses the same volume information as the container
• user: Indicates that the container runs with the specified user, mysqland UIDfor GID, the user is the same as that of the user to avoid file permission problems.999:999xtrabackupmysql
• command: tail -f /dev/null: Use this command to xtrabackup-command-linemake the container resident as a service, which is convenient for the subsequent demonstration of the use process.

Note 2:

• data_volume: Use the local data volume binding, the default location: /var/lib/docker/volumes/mysql_data_volume/_data, because the above-mentioned container runs with the specified user, you need to change the permissions of this directory, execute outside the container:chmod 777 -R /var/lib/docker/volumes/mysql_data_volume/_data
• backup_volume: Use the local data volume binding, the default location: /var/lib/docker/volumes/mysql_backup_volume/_data, because the above-mentioned container runs with the specified user, you need to change the permissions of this directory, execute outside the container:chmod 777 -R /var/lib/docker/volumes/mysql_backup_volume/_data

2.3 Start the service

cd /root/apps/mysql
docker-compose up -d

2.4 Prepare test data

create DATABASE t1;
use t1;
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `name` varchar(20) DEFAULT NULL,
  `gender` int NULL DEFAULT 0,
  `birthday` varchar(10) DEFAULT NULL,
  `address` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1;
INSERT INTO `user` ( id, name, gender, birthday, address ) VALUES( 1, '张三', 1, '2023-08-06', '黑龙江');
select * from `user`;

3. Detailed explanation of XtraBackup use

3.1 Introduction to Common Parameters

Percona XtraBackupDescriptions of some commonly used parameters and their default values:

• --backup: Perform a backup operation, which is enabled by default.

• --target-dir=<directory path>: Specify the directory path where the backup file is stored.

• --incremental-basedir=<directory path>: specifies the base directory path of the incremental backup, indicating which full backup the incremental backup is based on.

• --prepare: Perform preparation operations on backup files, disabled by default.

• --copy-back：Copy backup files back to the database directory for restore operations, disabled by default.

• --apply-log: Apply transaction logs on backup files for restore operations, disabled by default.

• --datadir=<directory path>: Specify the data directory path of the database, the default is the data directory specified in the MySQL configuration file.

• --defaults-file=<file path>: Specify the configuration file path for backup and restore, the default is the default configuration file path of MySQL.

• --user=<username>: Specify the username used when connecting to the database, the default is the current system user.

• --password=<password>: Specifies the password used when connecting to the database, which is empty by default.

• --host=<host name>: Specify the host name used when connecting to the database, the default is localhost.

• --port=<port number>: Specify the port number used when connecting to the database, the default is 3306.

• --compress: Use compression algorithm during backup to reduce backup file size, default is disabled.

• --compress-threads=<number of threads>: specifies the number of threads used to compress the backup file, the default is 1.

• --encrypt: Encrypt backup files during backup, disabled by default.

• --encrypt-key=<key>: Specifies the key used for backup file encryption.

• --stream=<stream type>: output the backup data to standard output, file or network socket in the form of stream, the default is disabled.

3.2 Full backup and restore

3.2.1 Access to the container

docker exec -it xtrabackup-command-line /bin/bash

3.2.2 Create a full backup

xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/backup/full --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

Precautions:

• The target directory does not exist, XtraBackupit will be created.
• directory exists and is empty, XtraBackupwill succeed.
• The directory exists and is not empty, XtraBackupexisting files are not overwritten, and it fails at runtime with a system error 17, file exists.

3.2.3 View full backup

ls -lh /backup/full

Precautions:

• Backups can take a long time, depending on the size of the database. It is safe to cancel at any time as XtraBackupthe database will not be modified.

3.2.4 Simulate original database damage

• close container

docker stop mysql

• delete database file

cd /var/lib/docker/volumes/mysql_data_volume/_data
rm -rf *

3.2.5 Prepare a full backup

After creating a backup, you need to prepare it for restoration. Data files are not consistent at a point in time until they are prepared because they are copied at different times during the program run and may have been changed while doing so.

If you try to boot with these data files InnoDB, it will detect the corruption and stop working to avoid running on corrupted data. This step makes the files fully consistent at a single moment, so you can run on them InnoDB.

You can run the prepare operation on any machine; it does not need to be on the original server or the server you are restoring to. You can copy the backup to the utility server and prepare it there.

Note that Percona XtraBackup 8.0only prepare MySQL 8.0, Percona Server for MySQL 8.0and Percona XtraDB Cluster 8.0backups of the database are possible. 8.0Previous versions are not supported .

During the prepare operation, xtrabackupa modified embedded InnoDB( extrabackuplinked library) is started. InnoDBThese modifications are necessary to disable standard security checks, such as: log file size is not appropriate. This warning does not apply to processing backups. These modifications apply only to extrabackupbinaries; you don't need the modified ones InnoDBto use xtrabackupfor backup.

The preparation step uses this "embedded InnoDB" to perform crash recovery on the replicated data files using the replicated log files. This step is very simple to use: you just run with options xtrabackupand tell it which directory to prepare. The specific operation is as follows:

xtrabackup --prepare --target-dir=/backup/full

Precautions:

• Interrupting the process while preparing a backup is not recommended XtraBackupas this may result in data file corruption and the backup will become unusable. Backup validity is not guaranteed if the preparation process is interrupted.
• If you want the backup to be the basis for further incremental backups, you should use the option when preparing the backup --apply-log-only, otherwise incremental backups will not be applied to it. For more details below.

Once this is done, you should see a message like the following, where LSNthe value of will again depend on your system: InnoDB shutdown:

3.2.6 Full backup restore

For convenience, xtrabackupthe binary can optionally be copied to the server's datadir:

xtrabackup --copy-back --target-dir=/backup/full

Precautions:

• The data directory must be empty before restoring a backup. Also, it is important to note that the service needs to be shut down before performing the restore MySQL. You cannot restore to the data directory of a running mysqldinstance (except when importing a partial backup).
• You should check that the restored files have the correct ownership and permissions, and if necessary execute: chown -R mysql:mysql /var/lib/mysqlmake ownership changes.

3.2.7 Start the database and verify

docker restart mysql

use t1;
select * from `user`;

So far the complete backup and restore introduction is complete.

3.3 Incremental backup and recovery

3.3.1 Create a full backup

Precautions:

• /var/lib/docker/volumes/mysql_backup_volume/_data/fullNeed to clear the directory first

For details, see chapters 3.2.1 , 3.2.2 , and 3.2.3 above , the operation steps and commands are the same

3.3.2 Add incremental data 1️⃣

use t1;
INSERT INTO `user` ( id, name, gender, birthday, address ) VALUES(4, '七七', 1, '2023-08-06', '海南');
select * from `user`;

3.3.3 Create incremental backup 1️⃣

xtrabackup --backup --target-dir=/backup/inc1 --incremental-basedir=/backup/full --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

3.3.4 Add incremental data 2️⃣

use t1;
INSERT INTO `user` ( id, name, gender, birthday, address ) VALUES(5, '八八', 1, '2023-08-06', '合肥');
select * from `user`;

3.3.5 Create incremental backup 2️⃣

xtrabackup --backup --target-dir=/backup/inc2 --incremental-basedir=/backup/inc1 --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

3.3.6 Prepare backup

The steps for an incremental backup are different from those for a full backup. In a full backup, two types of operations are performed to keep the database consistent: committed transactions are replayed from the log files relative to the data files, and uncommitted transactions are rolled back.

When preparing incremental backups, you must skip the rollback of uncommitted transactions, because uncommitted transactions may be in progress at the time of backup, and will likely be committed in the next incremental backup. You should use --prepare --apply-log-onlyoptions to prevent the rollback phase.

3.3.6.1 Prepare a full backup

xtrabackup --prepare --apply-log-only --target-dir=/backup/full --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

Notes:
--prepare--apply-log-only: The rollback phase is prevented by configuration.

3.3.6.2 Prepare incremental backup 1️⃣

To apply the first incremental backup to a full backup, run the following command:

xtrabackup --prepare --apply-log-only --target-dir=/backup/full --incremental-dir=/backup/inc1 --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

Precautions:

• This backup can actually be safely restored as-is, even if the rollback phase has been skipped. If you restore it and boot MySQL, InnoDBit will detect that the rollback phase was not performed, and it will do so in the background, just like crash recovery normally does on boot. It will notify you that the database was not shut down properly.
• The final data is in /backup/fullthe directory.

3.3.6.3 Prepare incremental backup 2️⃣

The procedure for preparing the second incremental backup is similar to the first:

xtrabackup --prepare --target-dir=/backup/full --incremental-dir=/backup/inc2 --user=root --password="1qaz@WSX" --port=3306 --host=192.168.0.50 --no-server-version-check

Notes:
- --apply-log-only: should be used when merging incremental backups ( except the last one ). That's why the previous line doesn't include that option. Even if used in the last step, the backup will remain consistent, but in this case the server will perform a rollback phase.

3.3.7 Simulate original database damage

See section 3.2.4 above for details , the operation steps and commands are the same

3.2.8 Full backup restore

xtrabackup --copy-back --target-dir=/backup/full

3.2.7 Start the database and verify

docker restart mysql

use t1;
select * from `user`;

So far, the introduction of incremental backup and recovery is complete.

3.3 Others

About XtraBackupcompressed backup and partial backup will be introduced later.

Reference: https://blog.51cto.com/u_11750496/6753459