Zabbix data cleaning

Zabbix data clean-up of a series of operations

Basic Information:

Zabbix version 4.0.9

MySQL version 5.5

First, the problem

We will Zabbixof RDS data stored in the test environment (Ali cloud), but when the RDS buy only 10G of storage, so there is no monitoring of several months, our database they reported a shortage of storage space warning.

First investigation, which is more storage space it occupied table, we find that mainly historyand history_uinttwo tables. Space is the biggest history_uinttable.

Then the two tables are stored in what it?

history_uint

This table is stored monitored item unsigned integer data.

When you save the data length, depending upon historical data monitoring long term retention settings.

CREATE TABLE `history_uint` (
  `itemid` bigint(20) unsigned NOT NULL,
  `clock` int(11) NOT NULL DEFAULT '0',
  `value` bigint(20) unsigned NOT NULL DEFAULT '0',
  `ns` int(11) NOT NULL DEFAULT '0',
  KEY `history_uint_1` (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

• itemid is monitored item ID

• timestamp clock time data (UNIX)

• value is the value of the monitored item data

• ns nanosecond time value (smaller than the value of 1s),

Precise real time monitoring of collection items, is composed by the clock + ns.

history

This table is stored in the floating-point type.

Like history_strand other character data is stored. These are the items we set up monitoring information corresponding to the type of decision.

When you save the data length, depending upon historical data monitoring long term retention settings.

Spread

Table trendsis stored in the trend data, and history are different, trends table holds only
the average value is small. Therefore trends there are many types of tables, the corresponding history.

Second, the solution

Temporary solution

To address this issue, we have a temporary solution is to delete history_uintand historysome historical data.

To delete the first time to obtain a time stamp, for example, we want to delete the data prior to September 2019 10 11:00. So this time corresponds to the time stamp is 1568084400.

We delete history_uintthe data in, a point to note here, if our data more than words, we propose to delete in several time periods, such as our data inside the database from October 2019 to October 2018, then we can inside the data into four stages delete, from October 2018 to January 2019, from January 2019 to April 2019, thus avoiding excessive load lead to a one-time delete data comparison database Big. (Or may be used limit 10000)

delete history_uint

delete  from history_uint  where  clock < 1568084400  LIMIT 10000;

delete history

delete  from history  where  clock < 1568084400  LIMIT 10000;

After completion of the implementation of the above delete operation, we found a very unusual thing happened.

So much that we delete the data, but the data storage space is not reduced. But increased (this is not due to the newly added data led, behind that Ali cloud appears to be incorrect).

The reason is :

For delete delete from table_name where xxx with the conditions, whether it is innodbor MyISAMwill not free up disk space.

Why delete where delete the space will not be reduced? For example, a company with 100 stations, 100 people sitting, when 50 people left the company, but their station is still there, only the station removed will not exist, they can be installed in the station is newly recruited staff.

So we need to OPTIMIZE TABLEoperate, be free up space.

Note: optimize table '表名'During operation, MySQLlocks the table.

OPTIMIZE TABLE history_uint;
OPTIMIZE TABLE history;

After completion of RDS operation, we probably need a few minutes to see our actual information stored in the console.

Draw on content: https://blog.csdn.net/weixin_40596063/article/details/82978736

1、drop table table_name 立刻释放磁盘空间 ，不管是 Innodb和MyISAM ，不保留表结构，;

2、truncate table table_name 立刻释放磁盘空间 ，不管是 Innodb和MyISAM 。truncate table保留表结构，删除方式类似drop table;

3、delete from table_name删除表的全部数据,对于MyISAM 会立刻释放磁盘空间 ,InnoDB 不会释放磁盘空间;

4、对于delete from table_name where xxx带条件的删除, 不管是innodb还是MyISAM都不会释放磁盘空间;

5、对于delete操作（或带条件的delete）需要释放空间，在delete以后使用optimize table table_name 会立刻释放磁盘空间。不管是innodb还是myisam ；所以要想达到释放磁盘空间的目的，delete以后执行optimize table 操作。

For space after delete unreleased. When insert without the need to open up new space. We will use the reserved space.

Permanent solution

• The amount of data is due to the time we save too much historical data is too long, we can set the length of time to retain historical data, set this value to be shorter, so as to reduce the amount of data it.
• Expand the storage space of the database.