[Operation and maintenance practice] TIDB v6.1.1: analysis of full backup, full recovery and incremental backup methods

Author: Fly-bird Original source: https://tidb.net/blog/0b55fa52

Background : Due to company requirements, the data security of the database must be ensured. The database in our production environment adopts the method of full backup + incremental backup + real-time synchronization from the database to ensure high availability of the database. This article introduces the database backup method of our company's production environment.

Note: We use real-time synchronization of data to the slave database to ensure high availability (using pump+drainer). We also support the operation of recovering data at any time to prevent accidental deletion of data, which will be introduced in another article. Pump generates binlog logs, and drainer synchronizes to downstream data.

**Note: **This practical operation uses Tidb's BR tool and pump+drainer components to achieve full backup, recovery, and incremental backup of the database (the backup is saved as a file). The following is the actual operation:

1. Install Tidb . This service is used as the target machine for data recovery (this is used to retrieve data. To save resources, install Tidb single node). The installation steps are as follows:

1. Install tiup tool

curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh |sh

2. Set environment variables

source .bash_profile

3. Install components

tiup cluster

4. Write the cluster configuration file topology.yaml

global:
  user: "tidb"
  ssh_port: 22
  deploy_dir: "/tidb-deploy"
  data_dir: "/tidb-data"
pd_servers:
  - host: 192.168.195.139
tidb_servers:
  - host: 192.168.195.139
tikv_servers:
- host: 192.168.195.139
tiflash_servers:
  - host: 192.168.195.139
monitoring_servers:
  - host: 192.168.195.139
grafana_servers:
  - host: 192.168.195.139
alertmanager_servers:
  - host: 192.168.195.139

5. Check the risks


tiup cluster check ./topology.yaml --user root -p

6. Repair risks


tiup cluster check ./topology.yaml --apply  --user root -p

7. Deploy the cluster

tiup cluster deploy qzn v6.1.1 ./topology.yaml --user root -p

8. Initialize the cluster

tiup cluster start qzn --init      #记录下生成的密码The new password is: '*******'.

9. The deployment is completed. Check the cluster information as follows. There is no business data in the database.

2. Add pump nodes to TIDB original cluster

1. View the original cluster information as follows

2. Edit the extension file. /data/pump-qzn.yaml is the configuration file that needs to be extended. Pump generates a binary file. Drainer saves the binlog to file. The specific content is as follows:

global:
  user: "tsp-tidb"
  ssh_port: 22
  deploy_dir: "/tidb-deploy"
  data_dir: "/tidb-data"
pump_servers:
  - host: 192.168.195.140
    port: 8250
    deploy_dir: "/tidb-deploy/pump-8250"
    data_dir: "/tidb-data/pump-8250"
    # The following configs are used to overwrite the `server_configs.pump` values.
    config:
      gc: 7
drainer_servers:
  - host: 192.168.195.140
    port: 8249
    deploy_dir: "/tidb-deploy/drainer-8249"
    data_dir: "/tidb-data/drainer-8249"
    # If drainer doesn't have a checkpoint, use initial commitTS as the initial checkpoint.
    # Will get a latest timestamp from pd if commit_ts is set to -1 (the default value).
    commit_ts: -1
    # The following configs are used to overwrite the `server_configs.drainer` values.
    config:
      syncer.db-type: "file"

3. Expand the cluster

tiup cluster scale-out 集群名 /data/pump-qzn.yaml --user root -p

4. After adding pump nodes, check the cluster status

tiup cluster display cluster name, see that each node is Up

5. Enable TIDB’s binglog configuration

vi /tidb-deploy/tidb-4000/conf/tidb.toml  
加入下面内容
[binlog]
enable = true
ignore-error = true

然后重启tidb
tiup cluster restart qzn -N 192.168.195.140:4000

在mysql客户端中查看是否开启（显示为ON）
show variables like "log_bin";

6. Log in to the source database, create the database qzn_test and the table info, and insert a row of data into the table.

3. Tidb original cluster backup data (only backing up a single database is shown here, please refer to the official documentation for br backup and recovery) Download the BR backup tool. The backup folder requires 777 permissions. The download page is https://cn.pingcap.com/product -community/

mkdir -p /data/qzn/qzn_test   
chmod -R 777 /data/qzn/qzn_test
wget https://download.pingcap.org/tidb-community-toolkit-v6.1.1-linux-amd64.tar.gz
tar -zxvf tidb-community-toolkit-v6.1.1-linux-amd64.tar.gz -C /data/
cd /data/tidb-community-toolkit-v6.1.1-linux-amd64
tar -zxvf br-v6.1.1-linux-amd64.tar.gz -C /data/

cd /data/
./br backup db -s local:///data/qzn/qzn_test --pd "192.168.195.140:2379" --db qzn_test --ratelimit 120 --log-file backupqzn_test.log

4. Tidb target database recovery data br recovery backup (copy the backup file to the target database server)

Deploy the br tool on the 139 server

#140上发送文件到139
scp br [email protected]:/data/
scp -r qzn_test/ [email protected]:/data/qzn/

#在139服务器上操作

./br restore db -s local:///data/qzn/qzn_test --db qzn_test --pd "192.168.195.139:2379" --log-file restore.log

5. View the target database (139), data recovery has been completed

6. Check the database server 140. The binlog file has been generated. This file is an incremental backup file. In the actual backup process, the database is backed up by a daily Br backup + incremental backup.

The above completes the basic operations of database backup and recovery. Please note that if the production cluster has multiple tikv nodes, you need to hang on the shared disk during backup so that all tikv backup data can be written to the shared disk.

[Operation and maintenance practice] TIDB v6.1.1: analysis of full backup, full recovery and incremental backup methods

Guess you like