mysql pagination statement optimization

 Test data table structure

 

   Now there are two tables customer, and order_test, the primary key of customerId is the foreign key of order_test

   

CREATE TABLE `customer` (
  `customerId` int(11) NOT NULL AUTO_INCREMENT,
  `phone` varchar(11) NOT NULL COMMENT 'phone number',
  `password` varchar(20) NOT NULL COMMENT 'password',
  
  PRIMARY KEY (`customerId`)
 ) ENGINE=InnoDB  DEFAULT CHARSET=utf8
CREATE TABLE `order_test` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `orderNo` char(17) COLLATE utf8_bin NOT NULL,
  `customerId` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `111_idx` (`customerId`),
  CONSTRAINT `111` FOREIGN KEY (`customerId`) REFERENCES `customer` (`customerId`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=100880001 DEFAULT CHARSET=utf8 COLLATE=utf8_bin

 

   The customer table has more than 100 million pieces of data, and order_test has 9000w pieces of data.

 

   The following are some commonly used statements to optimize the above two tables

 

 limit paging optimization method

     Please refer to explain usage:  http://zcf9916.iteye.com/blog/2409731

    1. Suppose now you want to investigate and deal with 200 pieces of data from 500000 to 500200 in the customer table sorted by customerId

      This is the normal way of writing

      SELECT customerId FROM customer LIMIT 500000,200; execution time is 0.108 seconds

      This statement will find out the first 500,200 pieces of data, and then discard the first 500,000 pieces of data, so the speed is very slow

      Execute explain + sql, you can see type=all, a full table scan is performed 

1. id | select_type | table   | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra | 
2. +----+-------------+---------+------+---------------+------+---------+------+--------+----------+-------+ 
3. |  1 | SIMPLE      | customer | ALL  | NULL          | NULL | NULL    | NULL | 99808800 |   100.00 |       |

 

     If there is only one number after limit, limit 200, then the database will only scan these two hundred, so we can start from here.

 

 

    1. Assuming that the id is incremented and the data is not deleted,

     Create a primary key or unique index and use the index (assuming 10 entries per page)

    --- Statement style: In mysql, the following methods can be used: SELECT * FROM customer WHERE customerId > 500000 LIMIT 200, execution time 0.0032 seconds

    ---Adapted to the scene: Suitable for large amounts of data (tens of thousands of tuples)
    ---Reason: Index scan, the speed will be very fast. 

    Execute explain + sql, you can see that type=range, possible_keys=primary indicates that the primary key index is used

•     (ps: range: Retrieve only the rows of a given range, using an index to select rows.)

1. id | select_type | table   | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra | 
2. +----+-------------+---------+------+---------------+------+---------+------+--------+----------+-------+ 
3. |  1 | SIMPLE      | customer | range  | PRIMARY          | PRIMARY | 4    | NULL | 49808800 |   100.00 |       

 

   2. Assuming that the id is incremented, and some data has been deleted, and assuming that the 499800-499999 data has been deleted, then it cannot simply rely on customerId > 500000 to locate, because the actual id range is 50020-500400.

    ---Statement style: The following methods are available: SELECT * FROM customer WHERE customerId >= 

   (SELECT customerId FROM customer order by customerId  LIMIT 500000,1)  ORDER BY customerId  LIMIT 200

   

1. id | select_type | table   | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra | 
2. +----+-------------+---------+------+---------------+------+---------+------+--------+----------+-------+ 
3. |  1 | PRIMARY      | customer | range  | PRIMARY   | PRIMARY | 4    | NULL | 49808800 |   100.00 |   using where    
4. |  2 | SUBQUERY      | customer | index  |          | PRIMARY | 4    | NULL | 500001 |   100.00 |   using index

 

     

  PS: If limit offset, quantity. If the offset value is too large, the query time will still be very long, so when the offset exceeds a certain amount, it is best not to deal with it.   

  PS: found in experiments. SELECT customerId FROM customer order by customerId LIMIT 0,20; and SELECT customerId FROM customer LIMIT 0,20; Although the cutomer is incremented, the results of the two sentences may be different, because the order will be accessed according to the index pointer without order , not necessarily in order.