How ⼀ the SQL query is to perform to?
It zoomed body, MySQL Server can be divided into two parts layer and the storage layer engine
Server layer
Server layer including connectors, query cache, analyzer, optimizer, YES, etc., covering most of MySQL's nuclear zoomed Center Weighted services, as well as all of the built-in functions (such as date, time, math and encryption functions, etc.), all across storage engine functions are achieved in that one level, ⽐ such as stored procedures, triggers, and other views.
Storage Engine layer
Storage and retrieval ⽽ layer is responsible for data storage engine. Its architecture is a plug-in model, ⽀ support InnoDB (MySQL 5.5.5 version after default), MyISAM, Memory, and other storage engines
Connector
After the handshake tcp connection, identity verification, checking permissions, etc.
Connection establishment time-consuming complexity, minimize connection with long connections.
Using the long connection MySQL account memory up much faster because MySQL in the process of temporary Perform Using the memory management screen for the connection object ⾥. These resources will be released only when disconnected.
solve:
1. Disconnect ⻓ periodically connected. It uses one time period, or the program is determined YES ⾥ ⾯ through ⼀ rear Use account query zoomed memory, disconnects, after reconnection to the query again.
2. If you are Using MySQL 5.7 or later, you can perform it after each one ⽐ a more zoomed operation to re-initialize by YES mysql_reset_connection connection resources. This process does not require rewiring and re-do the verification authority, but the connection will be restored to the state when just finished creating
Command show processlist: View the list of connections
Query Cache
The cache will be saved in sql query as a key, the result is the value of key data, the first cache query query.
Does not recommend the use of the cache, because all cached data will update the table as long as this table empty, the cache hit rate is low, save caching affect system performance, suitable only for static data tables do caching.
mysql query select whether to use dynamic caching mechanism, query_cache_type parameter is set to DEMAND, do not use the default general query sql query cache, use cache when the query specified by SQL_CACHE display:
select SQL_CACHE * from T where ID=10；
Note: mysql8.0 caching feature has been deleted
Analyzer
Lexical analysis, parsing, check the correctness of sql
Optimizer
Optimization field conditions such as sequential, index selection
Actuator
Table T, ID field is not indexed, then the YES YES filter process is such that:
1. Use InnoDB engine tune-connector ⾏ take the first frame of the table, the ID value is not determined 10, skip if not, then if this is present in the result set ⾏;
2. Use engine tune-connector taken "to record the next ⾏" logic repeats the same determination until the last frame ⾏ take this table.
3. Perform will above all during traversal ⾏ full EMPTY condition set being recorded as a result set returned to the client.
Log System: How to update a SQL statement is executed?
The query process is not ⼀ kind that update process also involves two important date blog module, the main ⻆ which is exactly what we want to talk about today: redo log (redo date Chi) and binlog (filing date Chi)
redo log
Ancient restaurant owner has a billing credit of this, a small blackboard temporary billing guests for a long time on the black board temporarily in mind, when fewer guests then moved on accounting books. Every time credited directly to the accounts of a large number of books need to query too time-consuming.
Database Similarly, each time data updates are writing disk efficiency is too low, the database serves as a small blackboard is the redo log, redo log and small blackboard function as limited in size, such as ⽐ can be configured to ⼀ group four files, each files ⼩ zoomed member is 1GB, then this "pink sheet" will be a total of 4GB of the recording operation. Written from scratch, written to the end on a separate warranty cycle back to the beginning
Write ring, as shown in FIG ⾯ this.
write pos is the current record position, ⼀ while writing after ⼀ side shift.
checkpoint is the current location to be erased, and also the push back cycle, it is necessary to update the data recorded before erasing the file record.
And between the checkpoint write pos is the "pink sheet" still empty portion, can use it to record a new operation. If writepos catch checkpoint, said that "pink sheet" full, this time not to perform it new updates have to stop to wipe ⼀ some records, to advance record the next checkpoint.
With the redo log, InnoDB database can ensure that even made ⽣ abnormal restart, before submitting records are not lost, this can care has called crash-safe.
As long as that is written to the redo log in, even if not committed to the database reboots automatically redo log data recording performed flushed to the database
binlog
Auto engine with MySQL MyISAM, no crash-safe capability, binlog only be used to archive, InnoDB is another company to introduce a plug-in, you need to realize crash-safe capability, and therefore the introduction of redo log.
Both Chi There are three different date.
1. redo log InnoDB engine is unique; binlog is the MySQL Server layer implementation, all engines can use the.
2. redo log Chi is a physical date, the record is "on a data ⻚ change has been made"; binlog Chi is a logical date, the record is the original logic of this statement, ⽐ such as "ID = 2 to this ⼀ ⾏ c-field plus one. "
3. redo log is written by circulation, will be fixed space and use completely; binlog can be added is writing in. "Additional write" refers to the binlog files written ⼀ set will switch after ⼩ zoomed to record the next one, and does not overwrite the previous date blog.
Redo log is a record of this page "to do what changes."
Binlog has two modes, statement formats, then is credited sql statement, the contents of row format will record row, two in mind, before the update and after the update has. Log row format will be relatively large
Perform internal processes and InnoDB engine if you perform this simple update statement
1. Perform first find an engine that takes ⼀ ⾏ ID = 2. ID is the primary key, the search engine directly Using the tree to find this ⼀ ⾏. If ID = 2 this ⼀ ⾏ ⻚ where data already in memory directly back to Perform device; otherwise, need to start reading the disk clicks into memory, and then back again.
2. Perform data engine is to get the ⾏, this value plus 1, such as the original ⽐ is N, is now N + 1, the new ⼀ ⾏ obtained data, and then transferred Using this engine is writing Connector Making new data.
3. Engine ⾏ these updated new data in memory, while the update operation to redo log record ⾥ ⾯, in this case redo log prepare state. Then inform Perform Perform is completed and ready to commit the transaction.
4. Perform an ⽣ into binlog this operation, and the binlog is writing to disk.
5. Perform Using an engine tune-connector commit the transaction, the engine just is writing the redo log into submission (commit) state, the update is complete.
FIG shallow box means is discolored inside InnoDB perform to dark-color block is represented in the vessel YES YES in.
the redo log is writing split into two steps: prepare and commit, this is the " two-phase commit ."
If two stages, the first to write redo log or to write binlog, system crashes in the gap between the two can cause database recovery and data memory read data inconsistencies.
Stage two logs are written after the submission of the two states to ensure consistent logic.
Two-phase commit necessity, for example:
Examples do with the update statement. Suppose the current row with ID = 2, the value field c is 0, the update statement is further assumed that after writing during the first log, the second log not finished during a crash occurs, without using two-phase commit What happens then?
After the first to write redo log write binlog
Assuming redo log finished, binlog not yet finished when, MySQL process reboots. As we said before, finished after the redo log, even if the system crashes, still be able to recover the data back, so the recovery value of the line c is 1. However, due binlog did not finish on the crash, and this time there would be no record binlog this statement. Therefore, the time after the backup log, kept up the binlog there is no this statement. Then you will find, if you need to use this binlog to restore the temporary library, then because binlog this statement is missing, the temporary library will be less this update, recovered the value of this line c is 0, the value of the original library different.
After-Write binlog write redo log
If the crash after binlog written, not written because the redo log, crash recovery after this transaction is not valid, so the value of this line c is 0. But binlog which has been recorded, "the c changed from 0 to 1," the log. So, in time to recover after using binlog on more than one transaction out, recovered the value of this line c is 1, and the value of the original library different.
You can see, if you do not use the "two-phase commit", then the state database and state recovery is likely to come out with its log database inconsistencies.
If a two-phase commit, this time redolog just finished prepare, and binlog they fail, then the transaction itself is rolled back, the value of the library which status is 0.
To commit the normal execution is considered finished, but the crash recovery process, you can accept the situation "redolog prepare and binlog complete", that is the last step to restart the recognition of the abnormal commit data in the database will be written to the disk again
innodb_flush_log_at_trx_commit this parameter is set to 1, each transaction represents the redo log directly persisted to disk
sync_binlog this parameter is set to 1, each transaction represents the binlog have persisted to disk.
Both values are recommended set to 1, ensure that data can not be lost when the database reboots.
sql perform detailed process
1. First, the client sends a sql statement by tcp / ip SQL interface to the server layer
2.SQL interface after receiving the request, the article first parse the statement, verify permissions match
3. After verification by the parser will analyze the statement, if there are grammatical errors
4. Next optimizer is generated corresponding execution plan, the execution plan to select the optimal
5. After the actuator would be the implementation of this statement is based on the implementation plan. In this step will go open table, if there is MDL on the table, waiting.
If not, the table is added to the added short MDL (S)
(If opend_table too big, too small to show open_table_cache. Need to stop to open frm file)
Layer 6 into the engine, will first go innodb_buffer_pool in the data dictionary (metadata) information table obtained
7. Adoption of the metadata information, to find out whether there will be related info in lock information lock, and put the update statements required
Lock information is written to lock info in (lock here remains to be supplemented)
8. is then related to the old data is stored in the undo page in innodb_buffer_pool through snapshots, and undo log records changes redo
(Data page, if there are loaded directly into Use the undo page, if not, it is necessary to remove the disk where the corresponding data page, the page loaded in the Use the undo)
9. In the data page do innodb_buffer_pool update operation. Page and the physical data recording operation changes in the redo log buffer
Since update this transaction will involve the modification of multiple pages, so redo log buffer modify the information will be recorded in multiple pages.
Because of the group commit, redo log buffer generated by the firm may follow other transactions with flush and sync to disk
10. The modified information simultaneously, the event will follow the format of the recorded binlog_cache. (Here is a note binlog_cache_size transaction level, not session-level parameters, once commit, dump thread will take the initiative to send the event to from binlog_cache in the slave I / O thread)
11. After this sql article, needs to be done in the secondary index modifier, and writes the change buffer page, wait until the next time needs to read the other sql secondary index, and again do merge secondary index
(Random I / O into a sequential I / O, but due to the current disk is SSD, so for the addressing, the random I / O and sequential I / O gap is not)
12. At this time, the update statement has been completed, need to commit or rollback. Commit the case discussed here, and bis 1
13.commit operation, since the engine between the storage layer and the inner layer is a server XA (two transactions to ensure consistency, and this is mainly to ensure the redo log binlog atomicity),
So prepare to submit divided into phases and phase commit
14.prepare stage, the transaction xid written in the binlog_cache be flush and sync operations (large transaction, then this step is very time-consuming)
15.commit stage, due before the transaction has generated redo log sync to the disk. So this step marks only commit redo log in
16. When the binlog and redo log have been off the disk, if the trigger operation flushing dirty pages, first copy the dirty pages to doublewrite buffer, the inside of the doublewrite buffer is flushed to the shared table space, and only then by page cleaner thread the dirty pages are written to disk
Note: In fact, in the implementation process is to call on 5 of the 6, so that is one thing. MySQL server layer and the layers are preserved InnoDB table structure, so there will be open, said when describing the book
detail:
binglog logic logs, redo log physical log
Logical log can give other database engine uses another, we all have been put through this "logic";
Physical log only "I" own use, others do not share my "physical format"
redo log is a sequential write, much faster speed
When the database update operations are based on the memory page, the update does not directly update the disk, if there is the presence of memory directly update the memory, if there is no memory is read from disk into memory, the memory update, and write redo log
Small surprise
1, add up, download Union high commission, to receive major platform hidden coupons, each purchase province 10 8 do not fragrant, registered under the Fanger Wei code by the user can add micro letter liershuang123 (Micro Signal) receive the value of thousand Yuan massive learning video.
We dedicate part of the information for the table:
Software eBook: link: https: //pan.baidu.com/s/1_cUtPtZZbtYTF7C_jwtxwQ extraction code: 8ayn
architect two: link: https: //pan.baidu.com/s/1yMhDFVeGpTO8KTuRRL4ZsA extraction code: ui5v
architect stage courses: link: https: //pan.baidu.com/s/16xf1qVhoxQJVT_jL73gc3A extraction code: 2k6j
2, I paid money buying front end of a source separation scaffolding, is 10 yuan sale, plus micro-channel acquisition source liershuang123
