Because the business wrote a large transaction, MySQL's binlog expanded. When parsing a large binlog, we often encounter this problem, which makes it impossible to parse. It is difficult to analyze the problem without other tools.
Author: Sun Xuzong, engineer of Sina Weibo DBA team, mainly responsible for the operation and maintenance of relational databases such as MySQL and PostgreSQL.
Produced by the Aikeson open source community. Original content may not be used without authorization. Please contact the editor and indicate the source for reprinting.
This article has a total of 3200 words and is expected to take 10 minutes to read.
Fault phenomenon
Because the business wrote a large transaction, MySQL's binlog expanded. When parsing a large binlog, we often encounter this problem, which makes it impossible to parse. It is difficult to analyze the problem without other tools.
Fault recurrence
[root@xuzong mysql]# ls -lh mysql-bin.003300
-rw-r----- 1 my3696 mysql 6.7G Oct 30 16:24 mysql-bin.003300
[root@xuzong mysql]# /usr/local/mysql-5.7.35/bin/mysqlbinlog -vv mysql-bin.003300 > 1.sql
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.334z3P' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
mysqlbinlog: Error writing file '/tmp/tmp.0Uirch' (Errcode: 28 - No space left on device)
guess
- It may be a problem with tmpdir in the configuration file, but to modify this you need to restart MySQL.
- Is it possible to modify this temporary space without restarting MySQL?
Verify guess
Guess one
Looking at the tmpdir set by my.cnf, I found that this parameter was not used. It seems that the guess was wrong.
[root@mysql mysql]# cat my.cnf | grep tmpdir
tmpdir = /data1/dbatemp
Guess 2
After searching online, most of them talk about how to solve the problem of full temporary table, which is the first guess. There is no clear way to modify the space occupied by the temporary handle used when mybinlog is parsed.
problem analysis
We can only look at the source code and see how mysqlbinlog obtains tmpdir.
mysqbinlog.cc
int main(int argc, char** argv)
{
........
MY_TMPDIR tmpdir;
tmpdir.list= 0;
if (!dirname_for_local_load)
{
if (init_tmpdir(&tmpdir, 0))
exit(1);
dirname_for_local_load= my_strdup(PSI_NOT_INSTRUMENTED,
my_tmpdir(&tmpdir), MY_WME);
}
........
}
mf_tempdir.cc
my_bool init_tmpdir(MY_TMPDIR *tmpdir, const char *pathlist)
{
char *end, *copy;
char buff[FN_REFLEN];
DBUG_ENTER("init_tmpdir");
DBUG_PRINT("enter", ("pathlist: %s", pathlist ? pathlist : "NULL"));
Prealloced_array<char*, 10, true> full_list(key_memory_MY_TMPDIR_full_list);
memset(tmpdir, 0, sizeof(*tmpdir));
if (!pathlist || !pathlist[0])
{
/* Get default temporary directory */
pathlist=getenv("TMPDIR"); /* Use this if possible */ //这里能看到是获取的机器环境变量
#if defined(_WIN32)
if (!pathlist)
pathlist=getenv("TEMP"); //windows是temp
if (!pathlist)
pathlist=getenv("TMP"); //linux是tmp
#endif
if (!pathlist || !pathlist[0])
pathlist= DEFAULT_TMPDIR;
}
........
}
Good guy, it turned out to be the machine environment variable obtained, so this problem is solved.
Problem solving
Just temporarily modify the machine's tmpdir variable.
[root@mysql mysql]# export TMPDIR="/data1"
[root@mysql mysql]# echo ${TMPDIR:-/tmp}
[root@xuzong mysql]# /usr/local/mysql-5.7.35/bin/mysqlbinlog -vv mysql-bin.003300 > 1.sql
Summarize
- If you have any questions, please check the source code.
- You can consider using binlog parsing tools such as bin2sql to solve the problem.
- You can check if there is any record in the slow log.
Replenish
It turns out that this problem is described in the MySQL official manual , so I will make a supplement here.
When running mysqlbinlog against a large binary log, be careful that the filesystem has enough space for the resulting files. To configure the directory that mysqlbinlog uses for temporary files, use the TMPDIR environment variable.
For more technical articles, please visit: https://opensource.actionsky.com/
About SQLE
SQLE is a comprehensive SQL quality management platform that covers SQL auditing and management from development to production environments. It supports mainstream open source, commercial, and domestic databases, provides process automation capabilities for development and operation and maintenance, improves online efficiency, and improves data quality.
SQLE get
| type | address |
|---|---|
| Repository | https://github.com/actiontech/sqle |
| document | https://actiontech.github.io/sqle-docs/ |
| release news | https://github.com/actiontech/sqle/releases |
| Data audit plug-in development documentation | https://actiontech.github.io/sqle-docs/docs/dev-manual/plugins/howtouse |