mysql日志体系大盘点

MySql日志文件主要包含:错误日志、慢查询日志、事务日志、二进制日志等

Mysql的日志配置可以通过命令 

1 show global variables like '%log%'; 

执行的结果如下

"back_log"    "80"
"binlog_cache_size"    "32768"
"binlog_checksum"    "CRC32"
"binlog_direct_non_transactional_updates"    "OFF"
"binlog_error_action"    "ABORT_SERVER"
"binlog_format"    "ROW"
"binlog_group_commit_sync_delay"    "0"
"binlog_group_commit_sync_no_delay_count"    "0"
"binlog_gtid_simple_recovery"    "ON"
"binlog_max_flush_queue_time"    "0"
"binlog_order_commits"    "ON"
"binlog_row_image"    "FULL"
"binlog_rows_query_log_events"    "OFF"
"binlog_stmt_cache_size"    "32768"
"expire_logs_days"    "0"
"general_log"    "OFF"
"general_log_file"    "DESKTOP-VH518HU.log"
"innodb_api_enable_binlog"    "OFF"
"innodb_flush_log_at_timeout"    "1"
"innodb_flush_log_at_trx_commit"    "1"
"innodb_locks_unsafe_for_binlog"    "OFF"
"innodb_log_buffer_size"    "1048576"
"innodb_log_checksums"    "ON"
"innodb_log_compressed_pages"    "ON"
"innodb_log_file_size"    "50331648"
"innodb_log_files_in_group"    "2"
"innodb_log_group_home_dir"    ".\"
"innodb_log_write_ahead_size"    "8192"
"innodb_max_undo_log_size"    "1073741824"
"innodb_online_alter_log_max_size"    "134217728"
"innodb_undo_log_truncate"    "OFF"
"innodb_undo_logs"    "128"
"log_bin"    "OFF"
"log_bin_basename"    
"log_bin_index"    
"log_bin_trust_function_creators"    "OFF"
"log_bin_use_v1_row_events"    "OFF"
"log_builtin_as_identified_by_password"    "OFF"
"log_error"    ".\DESKTOP-VH518HU.err"
"log_error_verbosity"    "3"
"log_output"    "FILE"
"log_queries_not_using_indexes"    "OFF"
"log_slave_updates"    "OFF"
"log_slow_admin_statements"    "OFF"
"log_slow_slave_statements"    "OFF"
"log_statements_unsafe_for_binlog"    "ON"
"log_syslog"    "ON"
"log_syslog_tag"    
"log_throttle_queries_not_using_indexes"    "0"
"log_timestamps"    "UTC"
"log_warnings"    "2"
"max_binlog_cache_size"    "18446744073709547520"
"max_binlog_size"    "1073741824"
"max_binlog_stmt_cache_size"    "18446744073709547520"
"max_relay_log_size"    "0"
"relay_log"    
"relay_log_basename"    "C:\ProgramData\MySQL\MySQL Server 5.7\Data\DESKTOP-VH518HU-relay-bin"
"relay_log_index"    "C:\ProgramData\MySQL\MySQL Server 5.7\Data\DESKTOP-VH518HU-relay-bin.index"
"relay_log_info_file"    "relay-log.info"
"relay_log_info_repository"    "FILE"
"relay_log_purge"    "ON"
"relay_log_recovery"    "OFF"
"relay_log_space_limit"    "0"
"slow_query_log"    "ON"
"slow_query_log_file"    "DESKTOP-VH518HU-slow.log"
"sql_log_off"    "OFF"
"sync_binlog"    "1"
"sync_relay_log"    "10000"
"sync_relay_log_info"    "10000"

一、错误日志

如上查询的配置文件中,log_error 就是配置的错误日志文件的存储位置,mysql的错误日志是强制开启的。主要记录mysql服务器每次启动和关闭时的详细信息,以及运行过程中所有的比较严重的警告信息和错误信息等,内容格式如下:

 1 2019-07-28T09:05:15.396749Z 9 [Note] Aborted connection 9 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error reading communication packets)
 2 2019-07-28T10:34:57.643413Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
 3 2019-07-28T10:34:57.646421Z 0 [Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
 4 2019-07-28T10:34:57.660459Z 0 [Note] C:\Program Files\MySQL\MySQL Server 5.7\bin\mysqld.exe (mysqld 5.7.17-log) starting as process 2548 ...
 5 2019-07-28T10:34:57.762731Z 0 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
 6 2019-07-28T10:34:57.764737Z 0 [Note] InnoDB: Uses event mutexes
 7 2019-07-28T10:34:57.774763Z 0 [Note] InnoDB: _mm_lfence() and _mm_sfence() are used for memory barrier
 8 2019-07-28T10:34:57.781781Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
 9 2019-07-28T10:34:57.784789Z 0 [Note] InnoDB: Adjusting innodb_buffer_pool_instances from 8 to 1 since innodb_buffer_pool_size is less than 1024 MiB
10 2019-07-28T10:34:57.810859Z 0 [Note] InnoDB: Number of pools: 1
11 2019-07-28T10:34:57.823893Z 0 [Note] InnoDB: Not using CPU crc32 instructions
12 2019-07-28T10:34:57.834922Z 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
13 2019-07-28T10:34:57.840938Z 0 [Note] InnoDB: Completed initialization of buffer pool
14 2019-07-28T10:34:57.902190Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
15 2019-07-28T10:34:57.910212Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 172728817
16 2019-07-28T10:34:57.910212Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 172728826
17 2019-07-28T10:34:57.912216Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 172728826
18 2019-07-28T10:34:57.912216Z 0 [Note] InnoDB: Database was not shutdown normally!
19 2019-07-28T10:34:57.913219Z 0 [Note] InnoDB: Starting crash recovery.
20 2019-07-28T10:34:58.941953Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
21 2019-07-28T10:34:58.941953Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
22 2019-07-28T10:34:58.942956Z 0 [Note] InnoDB: Setting file '.\ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
23 2019-07-28T10:34:59.177579Z 0 [Note] InnoDB: File '.\ibtmp1' size is now 12 MB.
24 2019-07-28T10:34:59.181591Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
25 2019-07-28T10:34:59.182593Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
26 2019-07-28T10:34:59.182593Z 0 [Note] InnoDB: Waiting for purge to start
27 2019-07-28T10:34:59.239188Z 0 [Note] InnoDB: 5.7.17 started; log sequence number 172728826
28 2019-07-28T10:34:59.245204Z 0 [Note] InnoDB: Loading buffer pool(s) from C:\ProgramData\MySQL\MySQL Server 5.7\Data\ib_buffer_pool
29 2019-07-28T10:34:59.253225Z 0 [Note] Plugin 'FEDERATED' is disabled.
30 2019-07-28T10:34:59.331405Z 0 [Note] InnoDB: Buffer pool(s) load completed at 190728 18:34:59
31 2019-07-28T10:34:59.331405Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key
32 2019-07-28T10:34:59.332409Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
33 2019-07-28T10:34:59.378531Z 0 [Note] IPv6 is available.
34 2019-07-28T10:34:59.378531Z 0 [Note]   - '::' resolves to '::';
35 2019-07-28T10:34:59.379534Z 0 [Note] Server socket created on IP: '::'.
36 2019-07-28T10:34:59.643220Z 0 [Note] Event Scheduler: Loaded 0 events
37 2019-07-28T10:34:59.644222Z 0 [Note] Executing 'SELECT * FROM INFORMATION_SCHEMA.TABLES;' to get a list of tables using the deprecated partition engine. You may use the startup option '--disable-partition-engine-check' to skip this check. 
38 2019-07-28T10:34:59.645225Z 0 [Note] Beginning of list of non-natively partitioned tables
39 2019-07-28T10:34:59.920058Z 0 [Note] End of list of non-natively partitioned tables
40 2019-07-28T10:34:59.921060Z 0 [Note] C:\Program Files\MySQL\MySQL Server 5.7\bin\mysqld.exe: ready for connections.
41 Version: '5.7.17-log'  socket: ''  port: 3306  MySQL Community Server (GPL)

二、慢查询日志

 慢查询日志主要记录mysql中执行的时间比较长的sql,默认的阈值是10秒,执行时间超过10秒的sql语句就会被慢查询日志所记录,慢查询日志的配置可以在mysql的配置文件中配置

slow-query-log=1  // 0表示关闭;1表示打开
slow_query_log_file="DESKTOP-VH518HU-slow.log" //慢查询日志文件的存储位置
long_query_time=10 //单位为秒,表示慢查询时间,默认超过10秒算一次慢查询

 慢查询是从sql语句开始执行到语句结束为止,包含了获取锁、检索数据、返回数据等一系列过程。如果一条很简单的sql语句在执行的过程中需要获取一个锁,而锁又被另一个事务所占用,也会可能被慢查询所记录

三、事务日志

InnoDB的事务日志包括 redo log 和 undo log 两种,

redo log 是指重做日志,提供前滚操作,通常是物理日志,记录的是数据页的物理修改,而不是某一行或某几行修改情况,用来恢复提交后的物理数据页

undo log 是值回滚日志,提供回滚操作,用来回滚记录到某个版本,一般是逻辑日志,根据每行记录进行回滚

3.1、redo log(重做日志)

 redo log 包含三个部分 :

redo log buffer:innodb的缓冲区,mysql的innodb是运行在操作系统的用户空间,当事务提交时,会先将redo log写入内存中,也就是用户空间的内存

os buffer:内核空间的缓冲区,redo log需要最终刷盘成日志文件,就需要通过内核空间将日志保存到磁盘,所以需要将用户空间的redo log buffer先写入内核空间的os buffer

redo log file:磁盘上的redo log文件,落盘之后就相当于redo log 持久化完成

所以redo log的持久化步骤就如下图所示:

 先将用户空间中的 redo log buffer 写入到内核空间的 os buffer,然后通过fsync方法将内核空间中的数据写入磁盘中的redo log file文件

redo log刷盘的方式有三种,可以提供配置innodb_flush_log_at_trx_commit 来进行配置,表示事务提交之后如何将log buffer刷新到磁盘,默认的值为1,

当值为0时:事务提交时,不会将redo log buffer 写入 os buffer,而是每秒将redo log buffer写入os buffer,并且调用 fsync()将os buffer 写入 redo log file

当值为1时:事务提交时,会将redo log buffer 写入os buffer,并且调用 fsync()将 os buffer 写入redo log file (默认配置,系统故障也不会丢失数据,但是IO性能较差)

当值为2时:事务提交时,会将redo log buffer 写入os buffer,但是不立即执行fsync(),而是每秒执行一次fsync()将os buffer 写入 redo log file

默认情况下会采用值为1的配置,因为这样才能保证数据不会丢失,而其他两种情况下性能差不多,都比1的情况要好很多,但是极端情况下会丢失1秒钟的数据,在高并发的情况下,1秒钟的数据量也是很大的。

redo log的格式

innodb存储数据的单位是页,所以redo log也是基于页的格式来记录,默认情况下,innodb 页大小为16K,而redo log由很多的redo block,每个redo block为512个字节,一页可以存放很多的log block,

而log block中记录的是数据页的变化,格式如下:

redo_log_type : 占用1个字节,表示redo log 的日志类型

space:表示表空间的ID,采用压缩的方式后,占用的空间可能小于4字节

page_no:表示页的偏移量,同样是压缩后的

redo_log_body:重做日志的数据部分

3.2、undo log (回滚日志)

undo log 可以提供回滚和多个行版本控制(MVCC)

当事务提交时,不仅需要保持redo log,还会记录undo log,如果事务失败或回滚了,就需要通过undo log进行回滚

undo log 记录的是逻辑日志,如当事务提交的时一个delete语句,那么undo log中记录的就是相对应的 insert 语句,相当于是事务sql的 相反操作。

当执行rollback时,就可以从undo log中的逻辑记录读取到相应的内容并进行回滚。有时候应用到行版本控制的时候,也是通过undo log来实现的:当读取的某一行被其他事务锁定时,它可以从undo log中分析出该行记录以前的数据是什么,从而提供该行版本信息,让用户实现非锁定一致性读取。

另外 undo log 也会产生 redo log

四、二进制日志(bin log )

 bin log 日志是用来记录数据库的更新操作(insert、update、delete),主要用来mysql数据库的恢复和主从同步,mysql的主从同步就是通过主节点将bin log 发送给从节点,从节点通过bin log 文件来更新数据达到数据一致性效果

 bin log的相关配置可以通过命令 show variables like '%log_bin%' 来查询,默认是关闭的

bin log 的格式主要通过参数 binlog_format 来控制,主要有三种 statement、row、mixed

当值为 statement 时:bin log中存储执行的具体sql语句,比如 insert、update、delete等,但是可能会出现数据不一致的情况,比如sql语句中使用了 NOW()、SLEEP()等实时性的函数时,每次执行都会结果不一样

当值为 row 时:bin log 中存储了更新的所有行的最新数据,这样就解决的数据不一致问题,但是日志文件会比较大,如第一种方案每个sql只会产生一条日志,而采用row的方式可能一条update操作会影响多条记录,就会产生多条的日志,

而如果更新的是整个表,就会将整个表的数据都会记录到日志中,显然这种方式也有很大的缺陷

当值为 mixed 时:bin log会根据具体的sql语句采用 第一种 或是 第二种方式,相当于是结合了两种方式的优点

bin log 的记录时间

bin log 日志是事务提交之后就会写入,也可以是异步的,可以通过配置sync_binlog 参数,当sync_binlog=n时,表示执行n条sql语句写入一次,如果设置值为1,则表示每一次sql事务提交都会同步写到磁盘中

实际也是写入os buffer,和redo log一样,都是通过innodb_flush_log_at_trx_commit的配置来决定何时刷盘

innodb通过记录 redo log 、undo log和bin log 可以保证数据的持久性、一致性和 原子性,而且当数据库启动恢复时也可以根据redo log来进行数据库的恢复操作

猜你喜欢

转载自www.cnblogs.com/jackion5/p/11259395.html