Many people, I'm talking about operation and maintenance engineers, when it comes to writing a certain plan, it is a headache. It’s not a unified search at a certain degree, or the same sentence is sent to all N groups: "Is there any such-and-so plan that can be shared? Help, everyone." It is estimated that nine out of ten, all of them will fall to the sea. There is no news.
In fact, is it really difficult, or is it that you have not fully grasped the entire backup idea? The good or bad of a plan lies in whether it is possible for a layman to understand the meaning of it at a glance, and it does not require a lot of thinking.
A good backup plan simply includes the following:
Why do you need a backup?
What are the backup methods?
What is the difference between certain backup methods?
Overview of actual backup operations
Overview of recovery operations
Other remarks
So, the above article from several angles, knot together some practical hands-on experience, step by step set forth a complete backup solution in the end is how composed. If you need to learn more about Mysql database, you can reply "MySQL" in the background of the official account: Migrant Workers Technical Road to get the most comprehensive MySQL database learning guide.
Why do you need a database backup?
Many people, when they look at this title, they will definitely answer when they open their mouths. What should I do if the backup fails? Run away? What should I do if the data is deleted by sand sculpture development (not allowed to spray)? Back to the pot?
Of course, everyone knows the importance and necessity of backup.
1. Ensure data security and integrity
The data security of an enterprise should be the lifeblood of an enterprise. Once it is lost or damaged, it will lose customers and money in the slightest, and it will go bankrupt (there are precedents).
The purpose of backup: to ensure that data can be restored in a timely and effective manner after being deleted or damaged by human error, improper operation, deliberate, etc., and will not greatly affect business operations.
2. Provide uninterrupted service for business
The actual production environment requires the database to have the ability to provide 7×24×365 uninterrupted service, which is one of the reasons why the database must be backed up.
Database backup method
Commonly used backup methods include the following:
Logical backup
Physical backup
1. Logical backup
Logical backup is actually to use the mysqldump command that comes with the MySQL database, or use a third-party tool, and then export the data in the database into a file in the form of SQL statements. When you need to restore data, extract the SQL statements in the backup file by using related commands (such as source) and execute them again in the database to achieve the purpose of restoring data.
Examples are as follows:
mysqldump -A -B --single-transaction >/server/backup/mysql_$(date +%F).sql
Generally, compression is performed during backup to save disk space, as follows
mysqldump -A -B --single-transaction ｜gzip>/server/backup/mysql_$(date +%F).sql.gz
Recovery operation
cd /server/backup/gzip -o mysql_$(date +%F).sql.gz
mysql -uroot -pMyadmin -h mysqldb.mingongge.com
> source /server/backup/mysql_$(date +%F).sql
Advantages and usage scenarios of logical backup
Advantages: simple, easy to operate, convenient and reliable with its own tools.
Usage scenario: It can be used when the database data volume is not large. When the data volume is relatively large (more than 20G), the backup speed is relatively slow, which will affect the performance of the database itself to a certain extent.
2. Physical backup
Physical backup is to use commands (such as cp, tar, scp, etc.) to directly copy one or more copies of the stored data files of the database and store them in other directories to achieve the effect of backup.
This kind of backup method will cause the possibility of data loss to a certain extent due to the fact that data will be written into the database during backup. When performing data recovery, you need to pay attention to the directory path, version, configuration, etc. of the newly installed data to be highly consistent with the original data, otherwise there will also be problems.
Therefore, this physical backup method often needs to be performed in a shutdown state, which is generally not desirable for a database in actual production. Therefore, this method is more suitable for physical database migration, which is more efficient in this scenario.
Advantages and usage scenarios of physical backup
Advantages: fast speed and high efficiency.
Scenario: It can be used in shutdown maintenance and database physical migration scenarios.
In the actual production environment, which method to use depends on the requirements and application scenarios.
Overview of full and incremental backups
After introducing the backup method, let's introduce the two concepts of incremental and full backup.
What is a full backup?
Full backup: It means to back up all the data in the database or all the data in a specific library at once.
Back up all data in the database
mysqldump -A -B --single-transaction ｜gzip>/server/backup/All_data_$(date +%F).sql.gz
Backup data of a certain library
mysqldump -A -B --single-transaction testDB1｜gzip>/server/backup/testDB1_$(date +%F).sql.gz
What is an incremental backup?
Incremental backup: refers to the data that is updated or added in the database during the period from the last full backup to the next full backup, and it is backed up.
Note: The full backup is a file, and the incremental backup is the MySQL binlog log file. So the incremental backup that is often said is to back up the binlog log file.
What is the difference between the two?
Full backup: The backup time required is longer and the recovery time is shorter because the number of files is small and maintenance is convenient. However, full backup files are large and occupy a certain amount of disk space. During full backup, the performance of the database will be affected programmatically (this is why the backup is performed at 0:00). It is also inconvenient due to the large file size. The server stores too many files locally, and full backup files of important services may need to be manually downloaded or migrated to a storage space outside the server.
Incremental backup: The backup is simple, and the recovery is a bit more complicated. Because of the large number of files, all binlog files need to be parsed into SQL statements, as follows:
mysqlbinlog testDB1-bin.000001 testDB1-bin.000002 >./bin.sql
Then, recover through recovery
mysql -uroot -pMyadmin -h mysqldb.mingongge.com
> source /server/backup/bin.sqlOr as follows
cd /server/backup
mysql testDB1 <./bin.sqlBackup and recovery practices
For the backup of Mysql database, scripts + scheduled tasks are generally used for daily backups.
Commonly used execution strategies are:
Perform a full backup every day at 0:00
Perform incremental backups according to business needs
Share an example of a backup plan I made at the beginning of a startup
Alibaba Cloud database server backup solution
Option One:
At present, the database is synchronized between master and slave, and the binlog log function is turned on from the database to perform remote backup. As far as the current data volume is concerned, only regular full and incremental backups of the database are required on the basis of the slave database.
1、创建备份目录
mkdir /server/backup
2、备份数据库到指定目录
mysqldump --single-transaction -F -B phoenix_coupon_production|gzip >/server/backup/phoenix_$(date +%F).sql.gz
mysqldump --single-transaction -F -B ywotx|gzip >/server/backup/ywotx_$(date+%F).sql.gz
find /server/backup/ -type f –name “*.sql.gz”-mtime +7 |xargs rm-f
将脚本写入定时任务，分时段进行打包备份
3、定时备份二进制文件
通过参数刷新binlog产生新的文件，通过脚本判断文件新旧，然后备份旧的日志文件
mysqladmin -uroot -pywotx!123 flush-logs #刷新日志，产生新的日志文件Finally, the backup files are synchronized or manually downloaded to the remote backup server to store the backup files in different places to realize the double backup storage of the database backup files to prevent server hardware failure.
Option II
After the increase in data volume in the later period, the database needs to be separated from reading and writing to realize the architecture of master write, slave read, master-slave synchronization. The backup is still carried out according to the original backup plan, and the data can be backed up by sub-database and table to prevent the large amount of data The problem of recovery time caused by the problem, improve the recovery efficiency.
1、创建备份目录
Mkdir /server/backup
2、备份数据到指定目录( 分库分表)
#/bin/sh
#create by mingongge at 2017-06-01
BACKUPDIR=/server/backup
DATE=`date +%F`
USER=root
PASSWD=”123456”
CMD=”mysql –u$USER –p$PASSWD”
DUMPCMD=”mysqldump –u$USER –p$PASSWD --single-transaction -F”
for dbname in `${CMD} –e “show databases”|sed ‘1d’`
do
mkdir –p${BACKUPDIR}/${dbname}
for tablename in`${CMD} –D ${dbname} –e “show tables”|sed ‘1d’`
do
${DUMPCMD} --tables${dbname} ${tablename} |gzip > ${BACKUPDIR}/${dbname}/${tablename}_$(DATE).sql.gz
done
done
find /server/backup/${dbname} -type f –name “*.sql.gz”-mtime +7|xargs rm -f
3、定时备份二进制文件（增量）
备份方法同方案一
Backup frequency:
Full database backup at 0:00 every day
Every day 03:00 9:00 15:00 21:00 incremental backup once
The database backup is fully prepared once a day. The binlog log will be updated and a new log file will be regenerated during the full backup. Therefore, the binlog will be refreshed during the next incremental backup to generate a new log file again to realize the database after the full backup Incremental backup of the operation, once a data problem is found, immediately refresh the binlog to re-create a new log file, manually back up the original log file, and then find out the point of the data problem, so as to use the log file to restore the complete Generate the data between the data problem points, and then restore the data from the problem point to the time period when the problem is found.
Add a new backup server, the configuration is as follows:
Example configuration: 2 cores/4G/40G + 200G high-efficiency cloud disk classic network 1M 295 yuan/month
Program summary:
Basically, the local backup file of the database server only retains the data within a week. The backup server retains data for at least 30 days according to requirements. The data retained for 30 days includes full database files and incremental backup files. The actual production requirements are modified, and the retention time will only increase the corresponding server disk space and increase a certain cost. Other changes are not required, and the operation is more flexible and convenient.
If you need a complete backup plan, you can reply to "Backup" in the backstage of the official account of the Migrant Workers Tech Road to get the download address of the complete plan. Interested readers and friends can try more keywords: "project, MySQL, avoid pits".