Transaction submission process of mysql source code interpretation--Part 1

Keywords: transaction submission process of mysql source code interpretation - the first article


mysql is a relational database, an important feature of relational database is to support transactions, which is a core feature that is different from no-sql products. Of course, no-sql products support key-value queries, but not SQL statements, which is also a difference. Today, I mainly discuss the transaction submission process. Due to the MySQL plug-in storage architecture, after the binlog is turned on, the transaction submission is essentially a two-stage submission. The two-stage submission is used to ensure the consistency of the storage engine and the binary log. This article only discusses the submission process in the unchecked state of binlog, and the submission logic after turning on the binlog option will be discussed later. The source code debugging environment is as follows:

Test environment:
OS: windows
DB: mysql 5.6.12
engine: innodb

Test precondition:
set autocommit=0;
create table tt(col1 int, col2 varchar(100));

test statement:
insert into tt values(1, 'abcdef');
commit;

    no matter for dml statements [insert, update, delete, etc.] or dcl statements [commit, rollback], mysql provides a public interface mysql_execute_command, we first analyze the basic process of the mysql_execute_command interface:
mysql_execute_command
{
   switch (command)
   {
       case SQLCOM_INSERT:
                mysql_insert();
                break;
 
       case SQLCOM_UPDATE:
                mysql_update();
                break;
 
       case SQLCOM_DELETE:
                mysql_delete();
                break;
       ......
   }
   
   if thd->is_error() //Statement execution error
     trans_rollback_stmt(thd) ;
  else
    trans_commit_stmt(thd);
}
　　You can see that at the end of executing any statement, trans_rollback_stmt or trans_commit_stmt will be executed. These two calls are statement-level commit and statement-level rollback respectively. Statement-level commit, in the case of non-automatic mode commit, mainly do two things, one is to release the autoinc lock, this lock is mainly used to process multiple transactions mutually exclusive to obtain auto-increment column value, therefore, regardless of the final statement commit or It is a rollback, the resources are all needed and can be released immediately. The second is to identify the position of the statement in the transaction, which is convenient for statement-level rollback. After executing commit, you can enter the commit process. Now let's see how the specific transaction commit process is.

mysql_execute_command
    trans_commit
ha_commit_trans(thd, FALSE);
{
    TC_LOG_DUMMY:ha_commit_low
        ha_commit_low()   
            innobase_commit
            {
                //Get the transaction structure corresponding to the innodb layer
                trx = check_trx_exists(thd);

                if(single statement, not auto-commit)
                {
                     //release auto-increment The autoinc lock resource occupied by the column
                     lock_unlock_table_autoinc(trx);
                     //Identifies the position of the sql statement in the transaction, which is convenient for statement-level rollback
                     trx_mark_sql_stat_end(trx);
                }
                else transaction commit
                {
                     innobase_commit_low()
                     {  
                        trx_commit_for_mysql();
                            trx_commit(trx); 
                     }

　　　　　　　　　　　　//Determine whether the redo log corresponding to the transaction is placed on the disk [according to the flush_log_at_trx_commit parameter, determine the redo log placement method]
                     trx_commit_complete_for_mysql(trx);
　　　　　　　　　　　　　　trx_flush_log_if_needed_low(trx->commit_lsn);
　　　　　　　　　　　　　　　　log_write_up_to(lsn);
                }
            }
}
trx_commit
    trx_commit_low
        {
            trx_write_serialisation_history
            {
                trx_undo_update_cleanup //For the purge thread to process, clean up the rollback page
            }
            trx_commit_in_memory
            {
                lock_trx_release_locks // release lock resources
                trx_flush_log_if_needed(lsn) // flush log
                trx_roll_savepoints_free // release savepoints
            }
        }
  
　   MySQL uses the WAL method to ensure the consistency and durability of database transactions, that is, C (consistent) and D (durability) in the ACID feature. WAL (Write-Ahead Logging) is a standard method for implementing transaction logs. Specifically, it is necessary to write the log before modifying the record. During the transaction submission process, it is necessary to ensure that the log is placed on the disk before the transaction is submitted. Through the WAL method, the performance of the database can be submitted under the condition of ensuring the transaction characteristics. As can be seen from the above process, four main things are done during the submission process. The first is to clear the undo segment information. For the update operation of the innodb storage engine, the undo segment needs to be purged. The main function of purge here is to actually delete the physical Record. When the delete or update operation is performed, the actual old record is not actually deleted, but a mark is placed on the record, but after the transaction is committed, the purge thread is actually deleted and the physical page space is released. Therefore, during the submission process, the undo information will be added to the purge list for processing by the purge thread. Then is the release of lock resources. MySQL uses the lock mutual exclusion mechanism to ensure that different transactions do not operate a record at the same time. After the transaction is executed, all lock resources will be released and other transactions waiting for its lock resources will be awakened. Then, the redo log is brushed. As mentioned, mysql implements the mechanism of transaction consistency and durability. Through the redo log drop operation, it is ensured that even if the modified data page is not updated to the disk, as long as the log is completed, the integrity and consistency of the database can be guaranteed; the last is to clean up the savepoint list, each statement will actually have A savepoint (savepoint), the function of savepoint is to roll back to the state before any statement of the transaction is executed. Since the transaction has been committed, the list of savepoints can be cleaned up.

     Regarding the mysql lock mechanism, purge principle, redo log, undo segment, etc. mentioned in it, they are actually the core of the database, and there are a lot of contents in it. The following appendix is ​​about the key data structures of the transaction and its members.
struct trx_t{

        trx_rseg_t* rseg;       /*!< rollback segment assigned to the
                    transaction, or NULL if not assigned

        trx_undo_t* insert_undo;    /*!< pointer to the insert undo log, or
                    NULL if no inserts performed yet */
    trx_undo_t* update_undo;    /*!< pointer to the update undo log, or
                    NULL if no update performed yet */
    const char* mysql_log_file_name;
                    /*!< if MySQL binlog is used, this field
                    contains a pointer to the latest file
                    name; this is NULL if binlog is not
                    used */
    ib_int64_t  mysql_log_offset;
                    /*!< if MySQL binlog is used, this
                    field contains the end offset of the
                    binlog entry */
}


/* The rollback segment memory object */
struct trx_rseg_t{
    /* Fields for update undo logs */
    UT_LIST_BASE_NODE_T(trx_undo_t) update_undo_list;
                    /* List of update undo logs */
    UT_LIST_BASE_NODE_T(trx_undo_t) update_undo_cached;
                    /* List of update undo log segments
                    cached for fast reuse */
    /*--------------------------------------------------------*/
    /* Fields for insert undo logs */
    UT_LIST_BASE_NODE_T(trx_undo_t) insert_undo_list;
                    /* List of insert undo logs */
    UT_LIST_BASE_NODE_T(trx_undo_t) insert_undo_cached;
                    /* List of insert undo log segments
                    cached for fast reuse */
}