MySQL-Backup + Log: Media Failure and Database Recovery

Level 1: Backup and Recovery

mission details

The task of this level:
backup the database, and then restore it.

related information

In order to complete this task, you need to master:
1. MySQL recovery mechanism;
2. Backup and recovery tools provided by MySQL.

MySQL recovery mechanism

Like most DBMSs, MySQL utilizes backups and log files for recovery.
Specific theoretical knowledge will not be introduced in detail here.

MySQL backup and recovery tools

MySQL provides the following tools:

• Logical backup tool: mysqldump
• Physical backup tool: mysqlbackup (commercial version only)
• Log tool: mysqlbinlog
• Restore tool: mysql
• Management tool: mysqladmin

mysqldump
logical backup tool, it generates a series of SQL statements, executing these statements can rebuild all objects and data in the original database. The default output is the console, and the generated SQL statement collection can be stored in a file through the redirection symbol.
mysqldump can back up all databases on the server, or specify certain databases, or certain tables in a database.
mysqldump -h127.0.0.1 -uroot -p123123 [options] --databases db_name

The --databases parameter is used to specify the database name, which can be followed by the name of one or more databases, and multiple database names are separated by spaces.
The mysqldump command line tool can also take several parameters, and there are dozens of optional parameters, see the official reference manual for details. Only one is introduced here:
--flush-logs Flush the MySQL log, that is, restart a log file.
Starting over with a new log file can be helpful in determining which logs are more useful in the future. Usually, the log files before the mass backup are much less important, because there is a backup in hand, unless the backup file fails, you may no longer need to use the previous log files.

mysqlbackup
mysqlbackup is a physical backup tool for MySQL, only available in paid commercial enterprise editions.

mysql
mysql is the most important client management tool for MySQL. Usually, users log in to the MySQL server through mysql to perform various operations. In addition, it is possible to execute SQL scripts, restore or create new libraries directly through it.
mysql -h127.0.0.1 -uroot -p12313 < mydb.sql
This will directly execute the script of mydb.sql. The script file backed up by mysqldump can be directly used to restore the original database by this method.

mysqladmin
mysqladmin is a management tool for MySQL server. It is generally used to configure the server, and can also be used to create or delete databases:
mysqladmin [options] command [command-arg] [command [command-arg]]
common commands (execution commands) are:
create db_name create database
drop db_name delete database
flush-logs flush logs
flush-tables flush Table, all table data is written to the disk
kill id, id,... kill some processes
password new_password modify (login) login password
ping check whether the server is available
status display server status
variables display the value of each configuration parameter
...

Similarly, the tool also supports many optional parameters, see the official manual for details.

mysqlbinlog
mysqlbinlog is a log management tool for MySQL. This tool is essential for recovery from failures that require manual intervention. Of course, you can also use it to view logs in general.
mysqlbinlog mysql-bin.000983
The above example is used to view the log file mysql-bin.000983. MySQL's log files have the same prefix, and the following numbers are the order of the log files. This prefix is ​​configurable. For example, it may also be binlog.*, for example:

Executing the log file will cause the events recorded in the log to be done again, so that the data of a given time period can be recovered. The recovery method is as follows: the recovery of a media failure usually needs to list all the log
mysqlbinlog [option] binlog_files | mysql -u root -p
files after the latest backup.

mysqlbinlog also supports dozens of optional parameters, such as:
–disable-log-bin no longer writes logs during database recovery through logs
–no-defaults does not use MySQL default settings

For details on other parameters, please refer to the official documentation.

MySQL also has some other tools (such as mysqlimport, mysqlshow, etc.), which will not be introduced here.

programming requirements

There is a resident population registration database residents, please make a static (you own the server) massive logical backup for the database, and the backup file is named residents_bak.sql. Then use the logical backup file to restore the database.
The tester does not check the name of the backup file, you can use other file names, but make sure to use the same file when backing up and restoring. The backup and recovery commands are respectively written in the test1_1.sh and test1_2.sh files.

• test1_1.sh - for backup
• test1_2.sh - for recovery

Do not reverse the two files.

According to the prompt, fill in the statement in the code file editor on the right to complete the above requirements. By default you should see test1_1.sh. Move the mouse to the triangle symbol on the right side of "Code File", all code files will be listed, and you can switch between the two code files freely. When all code files are filled out, click the "Evaluate" button.

Note: mysql, mysqldump, and mysqladmin need to provide the login user name and password when running. Since these tools are not used in an interactive environment, you need to provide the information required for login on the command line, for example: mysql
-h127.0.0.1 -uroot -p123123
But exposing the password on the command line is extremely risky, MySQL will throw a warning message, so the evaluation program will provide the login password through other means, please omit the -p parameter on the command line. Take mysql as an example, you only need to write like this:
mysql -h127.0.0.1 -uroot
Test Description
The database residents already exist, the platform will first run test1_1.sh to make a backup, and then drop the residents. Then run test1_2.sh to restore the database. Finally, check whether the recovery is successful. If the evaluation results on each residents instance column are completely consistent with the expected results, you can pass the level.

Let's start your mission, I wish you success!

code reference

test1_1.sh file content

# 你写的命令将在linux的命令行运行
# 对数据库residents作海量备份,备份至文件residents_bak.sql:
mysqldump -h127.0.0.1 -uroot --flush-logs --databases residents > residents_bak.sql

test1_2.sh file content

# 你写的命令将在linux的命令行运行
# 利用备份文件residents_bak.sql还原数据库:
mysql -h127.0.0.1 -uroot < residents_bak.sql

Level 2 backup + log: the occurrence of media failure and the recovery of the database

mission details

The task of this level:

Simulate the occurrence of media failure, and how to use the backup and the log after the backup to restore the database.

related information

As long as there are backup files and log files since the last backup, even if the entire database is lost due to media failure, the database can be restored after replacing the media. The relevant knowledge has been introduced in the previous level.

In a production environment, database files, backup files, and log files are not stored on the same media, or even in the same geographic space. The purpose is to have the opportunity to recover data after a disaster occurs.

This level will simulate the occurrence of media failure and the recovery of the database. The tools you will use include:

• mysql
• mysqldump
• mysqlbinlog

The previous level has introduced their usage.

programming requirements

Simulate the occurrence of media failure and the recovery of the database. The following events occur in chronological order (the part marked in red will be completed by you, and the rest will be completed by the evaluation program):

Time point 1: The database train has business data generated;
Time point 2: Make a massive backup of the database train and open a new log at the same time;
Time point 3: database train has business data again;
time point 4: failure occurs;
Time point 5: Restore the database to ensure that the business data that occurred twice will not be lost.

Please complete the work at time point 2 and time point 5 (the part marked in red), and fill in the script to complete the task in the corresponding file:

• Make a massive logical backup of the database train and open a new log at the same time: test2_1.sh
• Restore the database using backup files and log files: test2_2.sh

Do not reverse the two files.

All other work, the evaluation program will do it for you:

• At the point in time when business data occurs, data will be written in batches;
• At the point in time when the failure occurs, the database train will be dropped.
• Before dropping the database, the evaluation program will save the newly opened log file after backup as log/binlog.000018

Here, the evaluation program always finds the log file you need for you, and saves it in a fixed log file for your recovery. But in practice, the log file is always changing. Even in such an environment where you exclusively use the server instance, if you click the "Evaluate" button twice, the log file will write data multiple times due to the evaluation program, adding Changes in your backup and recovery operations, when the log file reaches a certain size, MySQL will automatically open a new log file. When experimenting locally, learn to identify which log file(s) you need.

Please fill in the statements in the code file editor on the right according to the prompts to complete the above requirements. By default you should see test2_1.sh. Move the mouse to the triangle symbol on the right side of "Code File", all code files will be listed, and you can switch between the two code files freely. When all code files are filled out, click the "Evaluate" button.

Test instruction

The evaluation program will trigger the corresponding events above in chronological order, and run your script when it is the turn of time 2 and time 5. Check that the database has been successfully restored after all events are over. If on two different instances of the database, after the failure occurs, the database is fully restored, and the customs can be cleared.

Let's start your mission, I wish you success!

code reference

test2_1.sh file content

# 这是shell脚本，将在linux命令行上执行
# 命令行上可省略密码的指定
# 请写出对数据库train作逻辑备份并新开日志文件的命令，备份文件你可以自己命名(如train_bak.sql)：
mysqldump -h 127.0.0.1 -uroot --flush-logs --databases train > backup/train_bak.sql

test2_2.sh file content

# 这是shell脚本，将在linux命令行上执行
# 命令行上可省略密码的指定
# 请写出利用逻辑备份和日志恢复数据库的命令：
mysql -h127.0.0.1 -uroot < backup/train_bak.sql
mysqlbinlog --no-defaults log/binlog.000018 | mysql -h127.0.0.1 -uroot