- In the daily operation of the website, there will be more and more business and log table data. When it reaches more than 10 million, complex queries such as join tables involved will become slower and slower, and it will also take up more and more disk space. . At this time, we can keep the data for a period of time, and we can delete or archive other data with a long time according to the situation.
- According to conventional knowledge, whether it is to directly delete invalid data or archive invalid data, empty the invalid number of the original table, you need to use the delete statement to delete. However, according to the test results given in the blogger Ao Bing ’s article Why MySQL does not recommend using delete to delete data , after directly using delete to delete 50w pieces of data in a table with 1 million data, not only the disk space occupied by the table is not reduced Small, and the efficiency of executing queries at the same time is not improved. This is becauseMySQL does not actually delete the space inside, and does mark deletion, that is, modify delflag:N to delflag:Y. After commit, it will be purged into the delete list. If a larger record is inserted next time, the space after delete will not be reused , If the inserted record is less than or equal to delete record empty will be reused
- At this time, in order to truly empty the space occupied by invalid data, we can use the table partitionPartition swapMethod to achieve this goal.
- First of all, make sure that the order of the table to be operated is a partitioned table, not that the performance of the partition is transformed into a partitioned table (it will take longer to transform directly into a partitioned table when there are more data in the table. You can create a partitioned table with the same structure first, and then use dataX Import the data into the partition table)
- Then, create a common intermediate table middle with the same structure as the partition table, and then create an archive partition table order_bak
- Finally, the partition exchange.
First import the data under an invalid partition into the intermediate table
alter table order exchange partition p201808 with table middle;
Import the data in the intermediate table into the corresponding partition in the archive table
alter table order_bak exchange partition p201808 with table middle;
Finally, delete the p201808 partition in the order table.
- In this way, the original table and the archive table are partitioned tables by month. You only need to create an intermediate ordinary table and do two partition swaps during the low peak period of the business. You can delete invalid data and reclaim space, and there is no space fragmentation. Will affect the index on the table and SQL execution plan.
- It is impossible for us to build all the partitions at once. We need to do a scheduled task. At the beginning or end of each month, add partitions to the original table and the archive table before performing the archive operation.