Remember a Mysql big data paging optimization problem

General paging uses limit

When the amount of data is relatively large, such as select * from u_user limit 10000000,10

In this way, the query will be extremely slow, because mysql will query the first one million + 10 pieces of data, then discard the first one million pieces of data and return the last 10 pieces of data;


How to optimize it? There are many ways to optimize, the general way is

select * from u_user where id >= (select id from u_user limit 10000000,1) limit 10

There is nothing wrong with this optimization, and the speed is also greatly improved;


1. But later found such a problem, the data after optimization and before optimization are inconsistent?

Later, it was found that the results of select id from u_user limit 10000000,1 were inconsistent each time;

Formally because of this inconsistency, the above optimized query is inconsistent. Why is it inconsistent?

Then Baidu, MySQL's default collation;

http://www.cnblogs.com/fnlingnzb-learner/p/6692680.html

I encountered a problem today. There is a Select statement without "Order By", and the returned data is uncertain.

This problem has been encountered more than a few times. To get to the bottom of it, if "Order By" is not added to the Select statement, how will MySQL sort it?

Searched the web and found this article on the MySQL forum.

http://forums.mysql.com/read.PHP?21,239471,239688#msg-239688

simply translate

* Cannot rely on MySQL's default sorting
* If you want to sort, always add Order By
* GROUP BY imposes Order By (this conflicts with standard syntax, if you want to avoid, use ORDER BY NULL) Here I have a doubt,
What kind of Order by is imposed

For MyISAM tables
MySQL Select default sort is displayed in physical storage order. (without additional sorting).
That is to say
SELECT * FROM tbl - results in a "table scan". If the table has no delete, replace, and update operations, the records will appear in the order they were inserted.

InnoDB tables
In the same case, it will be arranged in the order of the primary key. Again, this is just an unspoken rule (artifact of the underlying implementation: how to translate?),
Unreliable.

My understanding and speculation:

When "Select" does not add "Order by", MySQL will try to return the data in the fastest possible way (MySQL's actual method is not necessarily fast).
Since accessing the primary key and index will be faster in most cases (in the Cache), the returned data may be output in the order of the primary key and index.
There is no real sorting here, mainly because the primary key and index themselves are sorted and placed in memory, so the continuous output may be a certain sequence.
In some cases the data that consumes the shortest hard disk seek time will be returned first.
If only a single table is queried, it is regular in special cases.

final summary

"Order By is to be added"

If anyone wants to take a deeper look, they need to take a look at the MySQL source code.

As above, it is the key, because of the uncertainty of mysql's default sorting, it returns different each time



Solution: add order by

select * from u_user where id >= (select id from u_user order by id  limit 10000000,1) limit 10;


Then I found another problem

2. Why the order returned by select id from u_user order by id limit 10000000,1 cannot be guaranteed

   But it seems that select * from u_user order by id limit 10000000,1 is consistent every time it returns? ? ?

   Anyone tell?

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325894852&siteId=291194637