MySQL limit query two ways to optimize the paging query of millions of data

Note: If you explain the wrong place, please point out that it will be revised in time, thank you java study notes mind map version code cloud warehouse address https://gitee.com/vx202158/vx202158.git

1. Generate test data

Reference article MySQL quickly generates a large amount of test data 10 million

Second test

1. Create an index

	建表时已经设置了 id 是自增主键 在innoDB存储引擎中 自增主键默认作为聚蔟索引

2. Paging query

2.1 The usual way

#测试 LIMIT  测试
select * from t limit 1900000,10

It can be seen that the execution time is 0.36 seconds
insert image description here
. Analyze the execution plan and

you can see that the type is ALL, which is the slowest full table scan, operating 2.1 million rows of records.

2.2 Optimization method one

ideas

Because there is a primary key index, we can use index coverage to first query the primary key id and then use eq_ref

2.2.1 The first step is to query the id

Because there is a primary key index, we can use index coverage to first query the primary key id and then use eq_ref

select id from t order by id limit 1900000,10

insert image description here

Analyze SQL execution plans

SQL execution plan analysis,
insert image description here
you can see that the type is index using the primary key index and covering the index

2.2.1 The second step eq_ref associated subquery

	select t.* from t , ( select id from t order by id limit 1900000,10 ) t1 where t.id = t1.id

insert image description here
It can be seen that the time-consuming is 0.245s, and the optimization is nearly 0.12s. The larger the amount of data, the more detailed the optimization time.

Analyze SQL execution plans

Then analyze the SQL execution plan,
insert image description here
you can see that there are two queries (caused by sub-queries), the first query t table, the full index scan, the use of the primary key, and the covering index, and
then the full table scan of the derived table to query 10 rows of records, Then perform an equivalent reference query on the associated table t

2.2 Optimization method 2

ideas

First query the starting primary key id for paging, then use the where condition range query to finally get the number of records we need to get

2.2.1 The first step is to query the primary key id

select id from t limit 1900000,1

insert image description here

Analyze SQL execution plans

SQL execution analysis
insert image description here

The operation t table performs a full index scan, uses the primary key index, and uses the covering index

2.2.1 The second step of the associated range subquery

select * from t,(select id from t limit 1900000,1) t1 where t.id > t1.id LIMIT 0,10

insert image description here
It can be seen that it takes only 0.19s to optimize 0.17s

Analyze SQL execution plans

Then we analyze the SQL execution plan
insert image description here

First perform a full index scan operation on the t table, use the primary key index, and cover the index,
then perform the system operation (the fastest system attribute operation) on the derived table , and
then perform a range query on the t table, using the primary key index and using where condition

Summarize

SQL operation speed

system -> const  -> eq_ref -> ref -> range -> index -> all

There are two ways to optimize the limit paging query.
The first one uses the primary key index to query the range primary key id of the range query, and then performs an associated subquery. The
query type is
eq_ref
index
all

The second is to first find out the minimum value of the id of the composite range query condition, and then perform an associated subquery in pagination . The
query type is
system
range
index