Before today's update this Q & A article, MySQL combat this column has been updated 14. In these articles, we left a lot of quality in the comments area. Now, each article has a comments section enthusiastic students to help summarize the article knowledge, there are many students made a lot of quality problems, and some students to help answer questions raised by other students.
In the process of browsing and reply to these messages, I encouraged, also best of my knowledge to help you solve problems, and to discuss with you. It can be said, your message enlivened the atmosphere of the entire column, to enhance the quality of the entire column, thank you.
Most of the comments section of my message directly reply, the need for expanded description of the problem, I have come up with a small book in mind down. These problems are down, this is our Q & A article today the material.
So far, I have collected 47 questions, it is difficult today through this article Expand All. So, I'll start with the very close links to find a few problems, strung up, I hope you can help solve some doubts about the logs and indexes. And other issues, we keep back slowly unfurl.
Log-related issues
In my first two articles "log system: how to update a SQL statement is executed? "In, and you talked about binlog (archive log) and redo log (redo log) with the time of crash recovery, with a reductio ad absurdum, shows that if there is no two-phase commit, can cause data inconsistency standby appear MySQL and other issues.
In this article below, many students asked to submit a different moment in two stages, MySQL if abnormal restart happens, it is how to ensure data integrity?
Now, we have this problem started.
Figure I came alive again in a two-phase commit, allowing you to learn the following content.
1 a schematic view of two-phase commit FIG.
Here, you and I have to explain a misunderstanding-ended question. Some students asked in the comments area, this figure is not an update execution flow statement for you, how could you call the commit statement?
The reason he produces this question, is the concept of two "commit" is confused:
- He said, "commit statement" refers to the MySQL syntax for submitting a transaction order. General with begin / start transaction paired.
- And this "commit step." We used the figure refers to the transaction commit process of a small step, and final step. When this step is executed, the transaction is submitted is complete.
- "Commit statement" when executed, it will include "commit step."
And our case, there's no explicitly open a transaction, so the update statement that he is a transaction, when the transaction is committed after the execution is complete, it will use this "commit step."
Next, we analyze together at different times of the two-phase commit, MySQL reboots what phenomenon.
If, after the place in time A figure, which is written redo log in the prepare phase, before writing binlog, a crash has occurred (crash), this time due to the binlog not write, redo log also did not commit, so crash recovery when the transaction is rolled back. At this time, binlog not written, so it will not spread to the standby database. Here, everyone can understand.
Where we have problems, mainly concentrated in the moment B, binlog is finished, redo log crash has not happened before commit, and that crash recovery time will be how to deal with MySQL?
We first look at the collapse of the rule of judgment recovery.
- If redo log inside the transaction is complete, which is already commit identity, then submitted directly;
- If only complete prepare redo log inside a transaction, it is determined whether or not there is a corresponding transaction binlog and complete:
A If so, then the transaction is committed;.
B. Otherwise, roll back the transaction.
Here, the time point B is where crash occurs corresponding 2 (a), the transactions are submitted crash recovery process.
Now, we continue to extend this issue.
Asked 1: MySQL binlog know how intact?
Answer: binlog a transaction is complete the form:
- binlog statement format, there will be the last COMMIT;
- binlog row format, there will be a final XID event.
Further, after the MySQL 5.6.2 version, also introduces binlog-checksum parameter to verify the correctness of the contents binlog. In the case of logs due to a disk binlog reasons, may be wrong in the middle of the log, MySQL can be found by checking the result of the checksum. So, MySQL still have a way to verify the integrity of the transaction binlog.
Asked 2: redo log and binlog is how associating?
Answer: They have a common data field, called XID. When crash recovery will sequentially scan redo log:
- If you hit both prepare, but also commit the redo log, it is submitted directly;
- If you hit only parepare, but not commit the redo log, took the XID to find the corresponding binlog affairs.
Asked 3: redo log in the prepare phase plus complete binlog, will be able to restart the recovery, MySQL Why such a design?
Answer: Actually, the problem is still with us when it comes to consistency in contradiction with the backup of the data related. At the time B, that is, after the crash MySQL binlog finish, this time binlog has been written, and then will be used from the library (or libraries recovered by this binlog).
So, in the main library should commit the transaction. With this strategy, the main library and the backup data library ensures consistency.
4 ask: If this is the case, why two-phase commit it? Simply to finish first redo log, write binlog. When crash recovery, you must have two logs are complete before they can. Is not the same logic?
Answer: In fact, two-phase commit is the classic problem of distributed systems, MySQL is not unique.
If you must give a scenario to explain the need to do so, then it is the persistent problem affairs.
For InnoDB engine, if the redo log submission is complete, the transaction can not be rolled back (if that's allowed to roll back, it is possible to overwrite update other matters). If redo log directly to, and then write the time binlog failure, InnoDB and can not be rolled back, and binlog log data and inconsistent.
Two-phase commit is to give everyone a chance, when everyone says "I'm ok" when submitting together.
Asked 5: Do not introduce two logs, there is no need of a two-phase commit. Only binlog to support crash recovery, but also support the archive, can not it?
Answer: The students mean, retaining only binlog, then you can put into this submission process: ... -> "update the data into memory" -> "Write binlog" -> "commit transaction", is not it also possible to provide crash the ability to recover?
The answer is no.
If historical reasons , then it is not the MySQL InnoDB storage engine native. MySQL's native engine is MyISAM, there is no beginning of the design support crash recovery.
Prior to joining MySQL InnoDB engine family as MySQL plug-ins, it is already providing a crash recovery and transaction support of the engine.
After InnoDB access the MySQL, I found that since binlog ability to recover without crashing, then use the existing InnoDB redo log better.
And if implementation reasons , then there a lot. According to say on the question, only to realize binlog crash recovery process, I drew a schematic, there is no redo log up.
Figure 2 crash recovery support with binlog
Under this process, binlog still can not support crash recovery. I said that does not support the point of it: binlog no ability to recover "data page."
If Plotted position, that is, binlog2 finished, but not yet commit the entire transaction time, MySQL has occurred crash.
After the restart, the internal affairs of the engine 2 is rolled back, and then apply binlog2 can make it up; but for transaction 1, the system has been considered submitted completed no longer applied once binlog1.
However, InnoDB engine uses technology WAL, when executing a transaction, memory and finished log, the transaction is complete. If after a crash, it depends on the log to recover data pages.
That is the position of the crash in the figure, then the transaction is 1 may be lost, but also the loss of data page level. At this time, the binlog update the details which do not record the data page is not back up.
If you want to say, I look binlog optimized content, change it to record the data page can it? However, this is in fact has made a redo log out.
So, at least for now the binlog capacity, can not support crash recovery.
Asked 6: That can, in turn, only redo log, do not binlog ?
Answer: If the only point of view in terms of recovering from the collapse is possible. You can turn off the binlog, so there is no two-phase commit, but the system is still a crash-safe.
But if you look at each company's industry usage scenario, they would find in the official production library, binlog are open. Because binlog has redo log function can not be replaced.
One is the archive. redo log cycle is written, is written to the end back to the beginning to continue to write. This can not retain the history log, redo log archiving will not achieve the role.
MySQL is a system dependent on binlog. As a function of the MySQL binlog there is a beginning, it is used in a lot of places. Among them, the basis of highly available MySQL system is binlog replication.
There are a lot of companies have heterogeneous systems (such as some data analysis system), these systems rely on MySQL's binlog consumer to update their data. Binlog then turned off, these systems would not be able downstream input.
In short, since now many systems, including high-availability mechanisms, including MySQL rely on binlog, so "dove over the magpie's nest" redo log, have failed. You see, the development of eco how important it is.
Asked 7: redo log is generally set much?
Answer: redo log is too small, will lead soon to be filled, then I had to force the brush redo log, so the ability to play WAL mechanism does not come out.
So, if it is now common to several TB of disk, then it is not stingy, directly to the redo log to four files, each 1GB of it.
Asked 8: Example of normal operation after the data is written to disk eventually fall, is coming updates from the redo log or updates from the buffer pool over it?
Answer: This question actually asked very well. Involved here to the, "redo log which in the end is what" issue.
In fact, there is no complete data and redo log record data pages, so it does not have the ability to update their own disk data page, and not the presence of "data off the disk eventually, redo log is updated by the Past".
- If you are running instances of the case, after the data page is modified, it is inconsistent with the disk data pages, known as dirty pages. Final data off the disk, is to write the data pages in memory disk. This process, even nothing to do with redo log.
- In crash recovery scenario, InnoDB to determine if a page of data may be lost during a crash recovery of the update, it will be read into memory, then let the redo log update memory contents. After the update is complete, the memory page becomes dirty pages, returned to the first case of state.
Asked 9: What redo log buffer that? It is to modify the memory, or the first to write redo log file?
Answer: These two questions can be answered together.
During the update process a transaction, the log is to write many times. For example, the following matters:
begin;
insert into t1 ...
insert into t2 ...
commit;
复制代码This transaction is to be inserted into the two records in the table, insert data in the process, the resulting log had first saved, but can not be directly written at the time did not commit the redo log file.
Therefore, redo log buffer is a memory for pre-existing redo logs. In other words, in the implementation of the first insert the memory data is modified, redo log buffer is also written to the log.
But the real writing to a redo log file (the file name is ib_logfile + digital), is in the implementation of commit statements made.
(Here say that during the transaction will not "take the initiative to brush the plate" to reduce unnecessary IO consumption. But the situation may appear "passive written to disk," such as not enough memory, other transaction commits and so on. This is something we It will be after the first 22 article "MySQL what" harm than good, "the way to improve performance?" and then expand in detail).
Perform a single update statement when, InnoDB will start its own transaction, the statement submitted at the time of completion of the execution. With the above process is the same, but is "compressed" into a sentence which is complete.
These problems, is to redo log and binlog questions about everyone mentioned string together, do a centralized answer. If you still have questions, you can continue to add comments in the comments area.
summary
Finally, although this is a Q & A article, but still have a question after class.
We created a simple table t, and insert a row, then this line to make changes.
mysql> CREATE TABLE `t` (
`id` int(11) NOT NULL primary key auto_increment,
`a` int(11) DEFAULT NULL
) ENGINE=InnoDB;
insert into t values(1,2);
复制代码At this time, the table t in a unique row of data (1, 2). Suppose I now want to perform:
mysql> update t set a=2 where id=1;
复制代码You will see this result:
The results show, matches (rows matched) a row, modified (the Changed) 0 line.
Only from the point of view phenomenon, MySQL interior in dealing with this command, you can have three options:
- Updates are read before writing, MySQL read data, found that the value of a 2 originally not updated, direct return, execution ends;
- MySQL called "modified to (1,2)" This interface provides InnoDB engine, but the engine was found the same value as the original, is not updated, direct return;
- InnoDB serious implementation "to change this value to (1,2)," this operation, the lock locked, updates the update.
Do you think the actual situation will be more than what it? Can you construct experiments with ways to prove your conclusion? Further, you can think about why MySQL should choose this strategy?
On the issue of time
The problem is that previous period, a service table record count table with a number of rows, when inserted into the data service table is required to count is incremented.
Logically implementation is to start a transaction, two statements:
- insert into the data table;
- update count table, the count is incremented.
From the perspective of system concurrency considerations, how to arrange the order of these two statements.
Knowledge in the "line lock merits and demerits: how to reduce the impact on the performance of the line lock? "Because the update count table row lock it comes to competition, first insert and then update can minimize lock wait between transactions, enhance the degree of concurrency.
Concurrent performance viewpoint, the recording operation should be inserted first, then update count table.
Comments area some students said, should be put behind the update count table, because the table might count saved counts in the plurality of business table. If the first statement to update the table into the affairs of the count, multiple business table while inserting data, the waiting time will be longer.
The answer to this argument is right, but understanding is not correct. Even if we count the number of rows with a plurality of business table table, table name field will certainly add a unique index. This structure is similar to the following table:
CREATE TABLE `rows_stat` (
`table_name` varchar(64) NOT NULL,
`row_count` int(10) unsigned NOT NULL,
PRIMARY KEY (`table_name`)
) ENGINE=InnoDB;
复制代码In the update count table, it must be passed where table_name = $ table_name, using the primary key index, plus update row lock locks only on one line.
And insert data in different business table, is to update different rows, no row locks.
Reproduced in: https: //juejin.im/post/5d061189e51d4577407b1d2f