Reasons and Solutions for High Space Usage of MySQL Instances

When users use a MySQL instance, they may encounter space usage alarms or even be locked when the instance quota is exceeded. In the basic instance information of the RDS console, the following information will appear:

This article describes common causes of high space usage and how to fix them. For MySQL 5.6 instances, the instance can be unlocked after upgrading the instance specifications and storage space. For details on how to upgrade the instance configuration, see Changing the Configuration .

Common causes

There are four main reasons why MySQL instance space usage is too high:

• Binlog file usage is high.

• Data file usage is high.

• Temporary file usage is high.

• System file usage is high.

View space usage

You can view instance space usage through the diagnostic report in DMS.

1. Log in to the database on the DMS console .

2. Select Performance > Diagnostic Reports.

3. Click Start Diagnosis to create a report on the current instance operation, as shown in the following figure:

   

4. Click View Report to view the instance space usage in the diagnostic report, as shown in the following figure:

   

   Illustration description:

   • ins_size: The overall space of the instance.

   • other_size: The space used by system files and temporary files.

   • data_size: The space used by the data file.

   • binlog_size - Binlog: file footprint.

Solution

Upgrade instance specifications

Upgrading instance sizes is one of the effective ways to solve space problems. At present, RDS already supports instances with exclusive physical machine specifications, and the maximum storage space can reach 3T. It is recommended that you upgrade the instance specification to an exclusive physical machine. For details about the billing of an exclusive physical machine instance, see ApsaraDB for RDS Detailed Price Information . The steps to upgrade the instance specification are as follows.

1. Log in to the RDS management console .

2. Select the region where the target instance is located.

3. Click the ID of the target instance to enter the basic information page.

4. In the configuration information bar, click Change Configuration.

   Note: If it is an instance with an annual subscription and a monthly subscription, please click Upgrade Configuration Now or Renew to Downgrade/Renew to Upgrade.

5. On the Change Instance page, change the instance specification to Exclusive Physical Machine and select a larger storage space, as shown in the following figure.

   

6. Click Confirm Changes.

other methods

This section describes how to resolve space issues without upgrading instance sizes.

The solution to high binlog file usage

The Binlog file records the transaction information of the instance, which is the basis of the HA architecture of the MySQL instance, as well as high availability and recoverability, and cannot be closed.

The RDS instance automatically cleans up (uploaded to OSS and deleted from the instance space) binlog files older than the last 18 hours at certain intervals. If the instance DML operation generates a large amount of Binlog data in a short period of time, it may cause the instance disk space to be exceeded and be locked.

In this case, you can clean (upload Binlog files to OSS and delete from instance space) Binlog data through the RDS console.

Steps

1. Log  in to the RDS management console .

2. Select the region where the target instance is located.

3. Click the ID of the target instance to enter the basic information page.

4. Select Backup Restore in the left menu bar.

5. One click to upload Binlog.

   illustrate:

   • One-click upload of Binlog will submit cleanup tasks asynchronously in the background, so it will return quickly after a click. The cleanup task will first upload the completed binlog to the OSS of RDS (OSS not purchased by the user), and then delete the binlog file from the instance space. The binlog file currently being written is not completed because the writing is not completed. can be cleaned up. Therefore, there will be a certain delay in the cleaning process. It is recommended that you wait for a certain period of time after clicking, and do not click the button multiple times.

   • For operations such as DML (such as DML operations involving large fields) resulting in the rapid generation of Binlogs, there may be cases where the one-click upload Binlog button is clicked multiple times but the Binlog space occupancy still increases. This is because the Binlog files are uploaded to The processing speed of backing up space and deleting from instance space cannot keep up with the speed of binlog file generation by the instance. In this case, it is recommended to consider upgrading the disk space and investigate the reasons for the rapid generation of Binlog.

The solution to high data file usage

For data files with high space occupation, you can reduce the space occupation by cleaning up data, such as using  drop table and  truncate table commands to clean up data that is no longer needed.

In addition, the following are two methods that users often use to clear data files in the space, but in reality, in order to achieve the expected effect, the reasons and solutions are as follows:

• delete Delete data with  actions

  delete Operations cannot directly reclaim data file space occupied by deleted data, which is like emptying a swimming pool without changing its footprint.

  After  delete deleting data with an operation, you need  optimize table tab_name; to reclaim the space through an operation. For details, see  RDS for MySQL shows that the space does not decrease after deleting data .

• delete backup

  The RDS backup is placed on the background OSS and does not occupy the user's RDS instance space. Therefore, deleting the backup cannot solve the instance space problem. Also, deleting a backup will affect the recoverability of the instance, and it is strongly recommended not to consider deleting a backup under any circumstances.

How to query the space occupied by data files

method one:

The data file will have data holes after frequent DML. The space occupied by the data file obtained by the following query is closer to the calculation method of the actual data file space:

1. select sum(data_length + index_length + data_free) / 1024 / 1024 from information_schema.tables;

Method Two:

information_schema.tables provides some statistical information of tables obtained by sampling. Therefore, there will be discrepancies between the table and database data size obtained by the following query and the actual data file size (usually smaller than the actual data file size).

Note: Because the sampling statistics provided in information_schema.tables, the calculation method will be closer to the real space occupation when the statistics are closer to the actual situation.

1. select table_name, concat(round((data_length + index_length) / 1024 / 1024,2),’MB’)from information_schema.tableswhere table_schema = ‘rd_test’and table_name = ‘large_tab_01’;

The query result is shown in the following figure:

As can be seen from the above figure, there is a difference in the amount of table data fed back before and after the statistics of the table are collected. Even if the  analyze table statistics are re-collected through the command, the obtained value is usually smaller than the actual space occupied by the data file, for example, 16143 MB in this example is also smaller than the actual space occupied by the data file of the table.

The solution to the high occupancy of temporary files

Temporary files are automatically released with the end of the query or the termination of the session. Therefore, if the temporary file causes the instance space to be full and is locked, you can terminate the session to release the space.

For how to terminate a session, see  How RDS for MySQL Terminates a Session .

For common problems with temporary files, see  RDS for MySQL the table '/home/mysql/xxxx/xxxx/#tab_name' is full Causes and Handling .

https://help.aliyun.com/knowledge_detail/51682.html

Solutions to high system file usage

The system file involves the space file of the ibdata1 system table and the ib_logfile0, ib_logfile1 log files.

ibdata1 file

Because the InnoDB engine table supports multi-version concurrency control (MVCC), it will save the Undo information required by the query in the system file ibdata1.

If there is a query to an InnoDB table that does not end for a long time, and there are a lot of data changes in the table during the query process, a lot of Undo information will be generated, resulting in an increase in the size of the ibdata1 file. Due to the limitation of MySQL internal mechanism, the ibdata1 file currently does not support shrinking.

Therefore, if this happens, without upgrading the disk space, a better solution is to purchase an RDS instance with the same configuration in the same region and availability zone, and migrate the data to the new instance through the DTS tool.

It is recommended that you monitor and clean up sessions or transactions that take too long to execute. For details, see  Managing Long-Running Queries in RDS MySQL .

ib_logfile log file

The ib_logfile0 and ib_logfile1 log files store the transaction log information of the InnoDB engine table, and their file size is fixed and cannot be changed. A larger size is beneficial to reduce the number of transaction log file switching and improve instance performance in the scenario of high concurrent transactions.