mysql logical backup mydumper

 

mydumper for MySQL is a multithreaded, high-performance backup and recovery tool that provides concurrent backup, backup efficiency has greatly improved.

Installation mydumper

• yum install

  # rpm -ivh https://github.com/maxbube/mydumper/releases/download/v0.9.5/mydumper-0.9.5-2.el7.x86_64.rpm

• Compile and install

  # yum install glib2-devel mysql-devel zlib-devel pcre-devel openssl-devel
  # wget https://github.com/maxbube/mydumper/archive/v0.9.5.tar.gz
  # tar xvf v0.9.5.tar.gz
  # cd mydumper-0.9.5
  # cmake .
  # make

Backup

When the backup is not specified mydumper like mysqldump as to a particular sql file, you need to specify a backup directory:

mydumper -h ${host} --user ${user} --password ${passwd}  -B ${DB_NAME} -o ${bakdir} -e -t 16

mydumper - Help 
Usage: 
mydumper [OPTION ...] Multi - threaded dumping MySQL 
Help Options:
 - ?, - Help Options Show Help 
the Application Options:
 -B, - Database Database to dump
 # Export specified library 
-T, - comma-List table DELIMITED -tables List to the dump (does Not the exclude REGEX Option)
 # derived specified table, between a plurality of tables separated by commas. db1.t1, DB1, T2, DB2, T1, DB3, T1 
-O, --omit- from - File File containing entries It database.table of List A to Skip, One per Line (skips before Applying REGEX Option)
 -o, - outputdir directory to the Output Files to
 # specify a backup directory
-s, --statement-size Attempted INSERT of Statement of size in bytes, default 1000000
 # specify the size of each file Export SQL insert statement, the default 1000000value each insert 
-r, --rows the Try to Split INTO of chunks are the Tables of the this This turns OFF the Option rows MANY --chunk-. filesize
 -F, --chunk-filesize Split INTO of chunks are the Tables of the this the Output File This size value. IS  in MB
 -c, - the compress compress the Output Files
 # compressed backup, export SQL the file ending sql.gz 
-e, the Build files --build-empty the dump files the even- IF NO Data Available from table
 # If empty table is also generated file names corresponding to 
the -X-, Regular expression the --regex for  'db.table ' matching 
the regular matching library and table
 -i, --ignore- Engines Comma DELIMITED List of Storage Engines to the ignore
 # ignore not need to export specified storage engine tables 
-N, --insert- the ignore Dump rows with INSERT IGNORE
 # export SQL file IGNORE plus INSERT 
-m, --no-schemas the Do not dump the table at the schemas with the data
 # does not turn table structure, data only guide 
-d, --no-the data the Do not dump the table the data
 # nonconducting data only guide table structure 
-G, - the triggers Dump the triggers
 # backup trigger 
-E, - events events Dump
 # backup events 
-R, --routines Dump the Stored Procedures and Functions
# Backup storage procedures and functions 
-W, --no views the Do- Not the dump VIEWS
 # not Guide FIG 
-k, --no Locks the Do- Not This Will the cause is Inconsistent Backups: Execute the WARNING The Temporary Shared Read Lock.
 # When the backup is not plus global read lock, 
--no-Backup-locks the Do not use Percona Backup locks
 # do not use Backup locks Percona 
--less- locking the Minimize Time ON InnoDB locking the Tables.
 the -l, --long-Query-Guard Long Query the Set the Timer in seconds the, default 60
 # set the execution will be how long SQL KILL 
-K, --kill-Long- Queries Kill long running Queries (INSTEAD of ABORTING)
 # the kill long-running SQL 
#-D, --daemon Enable daemon mode 
backup daemon to default once every 60 minutes to back up
 the -I, --snapshot-interval The Interval Snapshot dump the BETWEEN the each ( in minutes), The requires --daemon, default 60
 # each backup the time interval between (in minutes) required --daemon, the default value is 60 
-L, --logfile the log File name to use, by default stdout iS Used
 # logging designated backup process, the default standard output 
- UTC-the SET TIME_ZONE = -TZ ' +00: 00 ' AT Top of the allow dumping of the dump to TIMESTAMP When Data Server A has Data in Different Time Zones or Data IS being Moved BETWEEN Servers with Different Time Zones, Defaults to use ON - -TZ-Skip UTC to disable.
 #Backup Time Zone settings, the default start adding in the backup file SET TIME_ZONE = '+ 00:00' statement 
--skip-TZ- UTC
 # disables the output SET TIME_ZONE statement in the backup file 
--use- savepoints the Use savepoints to the reduce the Metadata locking AND DELINQUENCY , needs SUPER privilege
 # used to reduce the metadata savepoints locking problem and needs SUPER privilege 
--success-ON-1146 Not INCREMENT COUNT error and Warning INSTEAD of Critical in Case of Table doesn ' T exist 
--lock the lOCK-All-the use Tables TABLE for All, INSTEAD of FTWRL
 -U, --updated-Operating since the Use update_time to dump only the Tables Updated in at The Last Days U
 --trx-consistency- only of the Transactional Consistency only
 --complete-Complete the Use INSERT statements that INSERT the include column names
 # backup files exported with a full field name 
-h, - Host at The Host to Connect to
 -u, - the User with the Username Necessary privileges at The
 -p, - password the User password
 - A, --ask- password Prompt the for the User password
 # interactive input password 
-P, --port TCP / IP Port to Connect to
 -S, --socket UNIX Domain socket to use File for Connection
 -t, Number The --threads of to use Threads, default. 4 
-C, --compress- Protocol the use Connection the MySQL The compression ON
 # backup transmission using compressed 
-V, --version the Show The Program Version and Exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file
# 指定mysql my.cnf配置文件

After the backup is complete, you can view the metadata file backup directory, view the backup was successful.

# cat metadata
Started dump at: 2019-06-19 17:02:45
SHOW MASTER STATUS:
    Log: mysql-bin.002714
    Pos: 73794931
    GTID:xxxxxxxx

SHOW SLAVE STATUS:
    Host: 192.168.1.10
    Log: mysql-bin.002721
    Pos: 133250446
    GTID:xxxxx

Finished dump at: 2019-06-19 17:59:20

View Code

 

restore

Before the first sql_mode recovery is set to null, otherwise it will error

mysql> show global variables like '%sql_mode%';
+---------------+------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value                                                                                                                  |
+---------------+------------------------------------------------------------------------------------------------------------------------+
| sql_mode      | STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+ --------------- + --------------------------------- -------------------------------------------------- + ------------------------------------- 
MySQL > SET Global the sql_mode = '' ;
after performing myloader then restore settings 
MySQL > sET Global the sql_mode = ' the STRICT_TRANS_TABLES, the NO_ZERO_IN_DATE, the NO_ZERO_DATE, the ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, NO_ENGINE_SUBSTITUTION ' ;

 

myloader -u ${user} -p ${passwd} -h ${host} -o -t 8 -d  ${bakdir}  

myloader parameters introduced:
 -d, --directory # import the backup directory 
-q, --queries-per-Transaction # number of queries for each execution, the default 1000 
-o, --overwrite-the Tables # If the table there is a deleted table 
-B , --database   # need to restore the database 
-s, --source-db database to restore   # restore from a backup file -s specified library to the database 
-e, --enable-binlog # enable binary data recovery. Whether to open the recovery binlog, close binlog when imported by default (if -e Lord will then recover from the environment with attention from the library, but slower ) 
-h, - Host at The Host to Connect to -u, - the User at The Necessary privileges with username
 -p, - password the User password
 -a, --ask- password Prompt the For the User password
 -P, --port TCP /IP port to connect to
-S, --socket UNIX domain socket file to use for connection  #指定socket file
-t, --threads Number of threads to use, default 4  # 指定线程数，默认为4
-C, --compress-protocol Use compression on the MySQL connection
-V, --version Show the program version and exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file

 

Examples of backup and recovery

master： 172.16.2.11    slave: 172.16.2.22

The master database data backup, and restore to which slave and master-slave synchronization configuration.

1. Create a backup of users and permissions in the master.
. MySQL> the GRANT the SELECT, the RELOAD, the PROCESS, SHOW DATABASES, SUPER, the LOCK TABLES, Of REPLICATION SLAVE, the CLIENT Of REPLICATION, the VIEW SHOW, the EVENT, the ON * * the TO TRIGGER 'backup'@'172.16.2.22' IDENTIFIED by 'xxxxxx';
MySQL > flush privileges;

2. In the remote database to a local backup slave
# mkdir / Data / mydumper
# mydumper -G -E -H -R & lt 172.16.2.11 --user --password XXXXX -o Backup / Data / mydumper / -e -t. 8

3. Restore. Mydumper restore backed up data to the database.
MySQL> Show Global Variables like 'the sql_mode%%';
+ --------------- + --------------------- -------------------------------------------------- ------------------------------------------------- +
| variable_name | Value |
+ --------------- + ---------------------------- -------------------------------------------------- + ------------------------------------------
| sql_mode | STRICT_TRANS_TABLES, the NO_ZERO_IN_DATE, the NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, NO_ENGINE_SUBSTITUTION |
+ --------------- + ------------------------- -------------------------------------------------- + ---------------------------------------------
MySQL> the SET, Ltd. Free Join sql_mode = '';
recovery steps:
myloader -u root -p xxxxxx -o -t 16 -d /data/mydumper/ -S /tmp/mysql.sock

After the restore, restore sql_mode:
MySQL> SET = Global sql_mode 'the STRICT_TRANS_TABLES, the NO_ZERO_IN_DATE, the NO_ZERO_DATE, the ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, NO_ENGINE_SUBSTITUTION';

4. After the data restoration is complete, the master and slave from the master is configured to synchronize data.
MySQL> = the CHANGE MASTER_HOST the MASTER the TO '172.16.2.11', MASTER_USER = 'the repl', MASTER_PASSWORD = 'xxxxxx', MASTER_AUTO_POSITION = 0, of MASTER_LOG_FILE = 'MySQL-bin.002714', MASTER_LOG_POS = 73,794,931;
# File and position in the binlog mydumper metadata file directory where to find.
mysql> start slave;