[Switch] MySQL innodb table auto-increment primary key problem

background:

Self-growth is a very common data attribute. In MySQL, everyone is willing to use the field of self-growth attribute as a primary key. Especially InnoDB, because of the characteristics of InnoDB's clustered index, it is better to use fields with self-increasing attributes as primary keys.

Problem 1: Table locks

Before MySQL 5.1.22, InnoDB self-increment was to obtain the value through its own self-increment counter, and the implementation was done through the table locking mechanism (AUTO-INC LOCKING). The lock is not released after each transaction is completed, but is released after the SQL statement inserted into the self-increasing value is completed, and subsequent operations can only be performed after it is released. For example, when there is an auto_increment field in the table, innoDB will save a counter in memory to record the value of auto_increment. When a new row data is inserted, a table lock will be used to lock the counter until the end of the insertion. . If there are a lot of concurrent inserts, table locks can cause SQL congestion.

After 5.1.22, InnoDB introduced the parameter innodb_autoinc_lock_mode in order to solve the problem of auto-incrementing the primary key lock table, which is implemented through the lightweight mutex growth mechanism. It is specially used to adjust the locking strategy in the case of using auto_increment, there are currently three options:

Insert Type Description:

INSERT-LIKE: refers to all insert statements, such as INSERT, REPLACE, INSERT...SELECT, REPLACE...SELECT, LOAD DATA, etc.
Simple inserts: Refers to the statement that can determine the number of inserted rows before inserting, including INSERT, REPLACE, excluding statements such as INSERT...ON DUPLICATE KEY UPDATE.
Bulk inserts: Refers to the statement that cannot be sure to get the inserted row before inserting. Such as INSERT...SELECT,REPLACE...SELECT,LOAD DATA.
Mixed-mode inserts: Some of them are self-increasing, and some are deterministic.

0: By means of table locking, that is, all types of inserts use AUTO-inc locking.

1: The default value. For the generation of the self-increasing value of the simple insert, the mutex is used to accumulate the counter in the memory. For the bulk insert, the table lock is used.

2: The mutex mechanism is used for the generation of all insert-like self-increment values, which has the highest performance. Concurrent insertion may lead to discontinuous self-increment, which may lead to inconsistencies in the Replication of Statement. To use this mode, you need to use Row Replication. model.

Before mysql5.1.22, mysql's INSERT-LIKE statement would use an AUTO-INC lock to lock the table during the execution of the entire statement until the end of the entire statement (rather than the end of the transaction). Therefore, when using INSERT...SELECT, INSERT...values(...), values(...), LOAD DATA and other time-consuming operations will lock the entire table and block other insert-like, update and other statements. It is recommended to use a program to divide these statements into multiple statements and insert them one by one to reduce the lock table time at a single time.

solve:

Solve by parameter innodb_autoinc_lock_mode = 1/2, and insert with simple inserts mode.

Problem 2: The auto-incrementing primary key is discontinuous

After 5.1.22, the default: innodb_autoinc_lock_mode = 1
directly analyzes the statement to obtain the number to be inserted, and then allocates enough auto_increment ids at one time, which will only lock the entire allocation process.

root@localhost : test 04:23:28>show variables like 'innodb_autoinc_lock_mode';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| innodb_autoinc_lock_mode | 1     |
+--------------------------+-------+
1 row in set (0.00 sec)

root@localhost : test 04:23:31>create table tmp_auto_inc(id int auto_increment primary key,talkid int)engine = innodb default charset gbk;
Query OK, 0 rows affected (0.16 sec)

root@localhost : test 04:23:35>insert into tmp_auto_inc(talkid) select talkId from talk_dialog limit 10;
Query OK, 10 rows affected (0.00 sec)
Records: 10  Duplicates: 0  Warnings: 0

root@localhost : test 04:23:39>show create table tmp_auto_inc\G;
*************************** 1. row ***************************
       Table: tmp_auto_inc
Create Table: CREATE TABLE `tmp_auto_inc` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `talkid` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=16 DEFAULT CHARSET=gbk
1 row in set (0.00 sec)

Insert 10 records, but the table's AUTO_INCREMENT=16, when inserting another one, the table's auto-increment id is discontinuous.

reason:

When the parameter innodb_autoinc_lock_mode = 1, the extra id (handler.cc:compute_next_insert_id) will be "pre-applied" each time , and after the insert execution is completed, these reserved ids will be vacated, that is, the current maximum value after pre-applying is deliberately set. The id is written back to the table (dict0dict.c:dict_table_autoinc_update_if_greater).

The strategy of this reservation is "apply for a few more when there is not enough", and the actual implementation is a step-by-step application. As for the number of applications, it is determined by "how many pieces of data N have been inserted" at that time. When auto_increment_offset=1, the number of pre-applications is N-1.

So you will find: when only 1 row is inserted, you cannot see this phenomenon and do not pre-apply. And when there are N>1 rows, it is required. The number of multiple applications is N-1, so the self-increment after execution is: 1+N+(N-1). There are 10 lines in the test, then: 1+10+9 = 20, which is inconsistent with 16? The reason is: when 8 rows are inserted, the AUTO_INCREMENT of the table is already 16, so when 10 rows are inserted, the id has been reserved in the 8th row, so it is used directly, and the auto increment is still 16. So when 8 rows are inserted, 7 more ids are applied, namely: 9, 10, 11, 12, 13, 14, 15. Insert 8~15 rows according to the method in the example, the AUTO_INCREMENT of the table is always 16

verify:

Insert 16 rows: guess the id of the pre-application: 1+16+(16-1)=32, ie: AUTO_INCREMENT=32

root@localhost : test 04:55:45>create table tmp_auto_inc(id int auto_increment primary key,talkid int)engine = innodb default charset gbk;
Query OK, 0 rows affected (0.17 sec)

root@localhost : test 04:55:48>insert into tmp_auto_inc(talkid) select talkId from sns_talk_dialog limit 16;
Query OK, 16 rows affected (0.00 sec)
Records: 16  Duplicates: 0  Warnings: 0

root@localhost : test 04:55:50>show create table tmp_auto_inc\G;
*************************** 1. row ***************************
       Table: tmp_auto_inc
Create Table: CREATE TABLE `tmp_auto_inc` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `talkid` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=32 DEFAULT CHARSET=gbk
1 row in set (0.00 sec)

As guessed, the auto-increment id is 32. So when 16 rows are inserted, 17, 18, 19..., 31 are applied for more.

So the reason for the discontinuous ID is because when innodb_autoinc_lock_mode = 1, more IDs will be applied for. The advantage is: allocating enough auto_increment id at one time will only lock the entire allocation process.

Default before 5.1.22: innodb_autoinc_lock_mode = 0

root@localhost : test 04:25:12>show variables like 'innodb_autoinc_lock_mode';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| innodb_autoinc_lock_mode | 0     |
+--------------------------+-------+
1 row in set (0.00 sec)

root@localhost : test 04:25:15>create table tmp_auto_inc(id int auto_increment primary key,talkid int)engine = innodb default charset gbk;
Query OK, 0 rows affected (0.17 sec)

root@localhost : test 04:25:17>insert into tmp_auto_inc(talkid) select talkId from talk_dialog limit 10;
Query OK, 10 rows affected (0.00 sec)
Records: 10  Duplicates: 0  Warnings: 0

root@localhost : test 04:25:21>show create table tmp_auto_inc\G;
*************************** 1. row ***************************
       Table: tmp_auto_inc
Create Table: CREATE TABLE `tmp_auto_inc` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `talkid` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=gbk
1 row in set (0.00 sec)

Insert 10 records, but the table's AUTO_INCREMENT=11, when inserting another one, the table's auto-increment id is still continuous.

innodb_autoinc_lock_mode = 2 and innodb_autoinc_lock_mode = 1 test the same. However, in this mode, one allocates one, and does not lock the table, but only locks the process of allocating IDs. The difference from 1 is that multiple pre-allocations are not performed, and this method has the highest concurrency. But there is a problem in replication when binlog_format is statement-based

solve:

Try to make the primary key ID have no business meaning, or use the simple inserts mode to insert.

in conclusion:

When innodb_autoinc_lock_mode is 0, the auto-increment id will be continuous, but there will be table locks. To solve this problem, you can set innodb_autoinc_lock_mode to 1 or even 2. It will improve the performance, but it will cause the self-increment id to be discontinuous under certain conditions.

Summarize:

According to the description of the above two problems, the self-incrementing primary key will generate table locks, which will cause problems; the self-incrementing primary key has business meaning, and the discontinuous primary key will lead to the inconsistency of the primary and secondary primary keys and cause problems. For the insert type of simple inserts, none of the above problems arise. For the insert type of Bulk inserts, the above problem occurs.

More information:

http://dinglin.iteye.com/blog/1279536

http://dev.mysql.com/doc/refman/5.5/en/innodb-auto-increment-handling.html

http://blog.chinaunix.net/uid-9950859-id-181376.html

[Switch] MySQL innodb table auto-increment primary key problem

Guess you like