binlog in mysql

foreword

MySQL's binary log binlog can be said to be the most important log of MySQL. It records all DDL and DML statements (except the data query statement select) in the form of events, and also includes the time consumed by the statement execution. MySQL's binary log It is transaction-safe. (One thing to note is that even if the update operation does not cause data changes, it will be recorded in the binlog.)
Generally speaking, there will be about 1% performance loss when the binlog log is enabled.

DDL: The main commands are create, alter, drop, etc. ddl is mainly used to define or change the structure of the table (table), data types, connections and constraints between tables and other initial work, most of them are used when building tables .
DML: The main commands are slect, update, insert, delete, just like its name, these 4 commands are the language used to operate the data in the database

Binlog has two common usage scenarios

Master-slave replication: mysql replication opens binlog on the master side, and the master passes its binary log to slaves to achieve the purpose of master-slave data consistency.
Data recovery: recover data through the mysqlbinlog tool.

Commonly used binlog log operation commands

View all binlog log list [show master logs;]

View the master status, that is, the number name of the last (latest) binlog log, and the pos end point (Position) value of the last operation event. 【show master status;】

flush Refresh the log log, and generate a new numbered binlog log file from now on;【flush logs;】

Note: Whenever the mysqld service is restarted, this command will be automatically executed to refresh the binlog log; adding the -F option when mysqlddump backs up data will also refresh the binlog log;

Reset (clear) all binlog logs [reset master;]

binlog file

Prior to MySQL 5.1, the record format of binlog was based on the statement format of SQL statements. The binlog_format parameter was introduced after version 5.1, and the value that can be set for this parameter has also increased to three types: statement, row, mixed.

statement: SQL statement-based replication.

Advantages: The binlog log volume is small, saving IO and improving performance.
Disadvantage: Some system functions cannot be copied exactly or cannot be copied, such as now(), uuid(), etc.

row: row-based replication, which records changes to the actual data of each row.

Advantages: The copy process is accurate, and there will be no problem that the call and trigger of stored procedures, functions, and triggers cannot be copied correctly under certain circumstances.
Disadvantages: It greatly increases the disk usage of the binlog. When clearing a large amount of data in some large tables, many statements will be generated in the binlog, which may cause the delay of the slave library to increase.

mixed: In this mode, MySQL will flexibly select the record format according to each specific SQL statement executed. General statement modification uses the statment format to improve performance. When some statements cannot complete the master-slave replication operation, the row format is used to save the binlog.

Advantages: Combining the characteristics of the other two modes, the accuracy is strong and the file size is moderate.
Disadvantages: May lead to master-slave inconsistency.

The difference between binlog and redolog

binlog is implemented by the server layer of MySQL, all engines are available	redo log is unique to InnoDB engine
binlog is a logical log, such as "add 1 to the c field of the row with id = 2"	Redo log is a physical log, which records "XXX modification on XXX page"
binglog is a one-time write when the transaction is committed	The data will be\written to the redo log buffer first, and every time the transaction commits, the log in the redo log buffer will be written to the os buffer and fsync() will be called to flush it to the redo log file
Binlog can be appended to	The redo log has a fixed size, so its space will be used up. If it is used up, some operations must be written to the disk before it can continue

Consistency between binlog and redolog: 2PC

1. The executor first looks for the engine to fetch the line with ID=2. ID is the primary key, and the engine directly uses tree search to find this row. If the data page where the row ID=2 is located is already in the memory, it will be returned directly to the executor; otherwise, it needs to be read from the disk into the memory first, and then returned.
2. The executor gets the row data given by the engine, adds 1 to this value, for example, it used to be N, and now it is N+1 to get a new row of data, and then calls the engine interface to write the new row of data.
3. The engine updates this row of new data into the memory, and at the same time records the update operation in the redo log, and the redo log is in the prepare state at this time. Then inform the executor that the execution is complete and the transaction can be submitted at any time. [Write to redo log (in prepare stage)]
4. The executor generates a binlog of this operation and writes the binlog to disk. 【Write binlog】
5. The executor calls the commit transaction interface of the engine, and the engine changes the newly written redo log to the commit state, and the update is completed. 【Submit transaction (in commit state)】

Mysql's two-phase commit

The above process uses a two-phase commit, so why use a two-phase commit? It is to make the logic between binlog and redo log consistent.

Let's assume that MySQL crashes at every moment when the above update statement is executed, and see how the logic between the two logs is consistent.
Assuming that before step 3, MySQL crashes and restarts, the transaction commit fails and the data will not be affected. Although the memory is updated, after a crash, the memory is lost.
Assuming that it crashes after the completion of step 3, the redo log has been written at this time. After restarting, it is found that the redo log is in the prepare stage, and it will not be restored.
Assuming that it crashes after the completion of step 4, the binlog has been written at this time. After restarting, it is found that the binlog has been written, and the corresponding redo log is changed to the commit state.
In this way, the logical consistency of redo log and binlog can be guaranteed.

Two-phase commit is a commonly used scheme to maintain logical consistency of data across systems.

Commonly used binlog log operation commands: https://blog.csdn.net/sinat_32430939/article/details/121533785
Difference: https://blog.csdn.net/qq_45243783/article/details/125040675
Consistency problem: https:/ /blog.csdn.net/yzx3105/article/details/130685375