pt-archiver configured automatically archived

Written in 2019-05-17

Mysql data archiving is generally used percona pt-archiver. By adding crontab shell script can cope with data under most scenarios automatic archiving.

Installation
Installation Percona Toolkit not repeat them, search their own references.

Archive script
mysql_archiver.sh code is as follows:

d1=`date -d "-1 month " +%Y-%m-%d`
d2=`date -d "-2 month " +%Y-%m-%d`
d3=`date -d "-3 month " +%Y-%m-%d`
d4=`date -d "-4 month " +%Y-%m-%d`
d5=`date -d "-5 month " +%Y-%m-%d`
d6=`date -d "-6 month " +%Y-%m-%d`
echo 'd1='$d1' d2='$d2' d3='$d3' d4='$d4' d5='$d5' d6='$d6

echo archiver table common_req
pt-archiver --source h=xxx.mysql.rds.aliyuncs.com,P=3306,u=test,p='你的密码',D=db1,t=common_req --dest h=xxx.mysql.rds.aliyuncs.com,P=3306,u=ucheer_cgb_user,p='你的密码',D=db1,t=common_req_archive --charset=UTF8 --where 'create_time<"'$d3'"' --progress 10000 --limit=10000 --txn-size 10000 --no-version-check --statistics --purge --ignore

pt-archiver function is very rich, relevant search syntax of your own, here a brief introduction of online and real use script parameters and values.
Variables d1 - d6 were 1 day ago - 6 months, most of the actual scene to clean up the data before long execution. The table must have a primary key, remember archive criteria to establish a suitable index, it is recommended to establish create_time each table and update_time.
Batch processing by 10 000, and displays statistics. purge means delete the original table archived data, ignore it refers to ignore the conflict when the data conflict.

Archive plan
crontab -e add a scheduled task:

00 04 * * * /bin/sh /home/xxx/mysql_archiver.sh>> /home/xxx/mysqlarchiver/output.txt

4:00 perform a daily archive script and plans to save the results to output.txt in.

Summary
pt-archiver feature-rich, archiving tasks can be achieved through regular shell scripts plus crontab. In the beginning of the design table, also we need to consider the conditions for data archiving and establish appropriate index. In addition to the timing of filing plans to share the project team, in order to prevent the programmer does not know the table is archived table to do the whole amount of the search business.