Index-06 order by

A sorting achieve
1. All fields are sorted
1.1 ordering process
1.2 the temporary file
2. rowid sort
3. Comparative summary of
two, a case of the optimization index
optimization summary

TOC

---

First, to achieve the sort

SELECT city,
       name,
       age
FROM t
WHERE city='杭州'
ORDER BY name LIMIT 1000 ;

---

1. All sort fields

1.1 ordering process

1. Initialization sort_buffer, determined into city, name, age field
2. Find the first city = 'Hangzhou' primary key
3. Found entire line, taking name, city, age three fields, primary key into sort_buffer from
4. Remove a record index from the city
5. Repeat 3-4 until the city does not satisfy the query value
6. sort_buffer sort data name field
7. Take the first row back to the client 1000

1.2 Temporary Files

To name sorting may be done in memory, it may require the use of external sorting, and the required memory depends on the sort sort_buffer_size
space is to sort sort_buffer_size opened,
1. sort data amount <sort_buffer_size, in-memory sort
2. Sort the data amount > sort_buffer_size, memory does not fit, had to use temporary disk files assist ranking

How to determine whether to use a sort statement to the temporary file

/* 打开 optimizer_trace，只对本线程有效 */
SET optimizer_trace='enabled=on'; 

/* @a 保存 Innodb_rows_read 的初始值 */
select VARIABLE_VALUE into @a from performance_schema.session_status where variable_name = 'Innodb_rows_read';

/* 执行语句 */
select city, name,age from t where city='杭州' order by name limit 1000; 

/* 查看 OPTIMIZER_TRACE 输出 */
SELECT * FROM `information_schema`.`OPTIMIZER_TRACE`\G

/* @b 保存 Innodb_rows_read 的当前值 */
select VARIABLE_VALUE into @b from performance_schema.session_status where variable_name = 'Innodb_rows_read';

/* 计算 Innodb_rows_read 差值 */
select @b-@a;

number_of_tmp_files indicates that the sorting process, the number of temporary files used.
the MySQL data sorted into 12 parts, the presence of these temporary files are stored in a separate after each, then these files 12 ordered recombined into an orderly large files

---

2. rowid sort

The above algorithm, the data read through the original table, the remaining operations are in sort_buffer and temporary files
there is a problem, a lot of time, sort_buffer put in more fields in the query field needs to be returned, so the memory at the same time relatively small number of rows down, to be divided into a number of temporary files.
so large single line, good efficiency, influence the algorithm parameters SET max_length_for_sort_data = 16;
control parameter data sorting line length, the length of a single line exceeds this value, the algorithm needs to change

city, name, age is the total length of the three fields 36, 16 is set to the max_length_for_sort_data placed sort_buffer beat sequence only the fields name and primary key id

Process:

1. Initialization sort_buffer, determined into two fields name, id
2. Find a first index from the city satisfies city = 'Hangzhou' primary key id
3. Accessed from the main key, take the name,, into the name id sort_buffer
4. Remove a record index from the city
5. Repeat 3-4
6. sort_buffer sort of name
7. Traversal sort results, taken before the 1000 lines, back to the table accessible city, name, age returned to the client

Twice during the back table access, examined_rows 4000, but select @ b- @ a = 5000, the cost of access + 1000 4000
sort_mode into sort_key, ROWID
number_of_tmp_file becomes 10, because the sort fields less

---

3. Comparison summary

MySQL enough memory, the more use of memory to minimize disk access.
High cost of sorting, and even disk sorting occurs because the original data is no need to
alter table t add index city_user (city , name);
if the establishment of such index, after the city = 'Hangzhou', name or orderly, query process:

1. From the city, name taken satisfy the first city = 'Hangzhou' primary key id
2. Taken from the master key name, city, age
3. Remove from a city Index
4. 2-3 is repeated, until the query to the second end of the line 1000, or does not meet city = 'Hangzhou'

This process does not require a temporary table, you do not need to sort

---

Second, the optimization of an index case

语句和表结构

## SQL
SELECT *  
FROM t_follow_timeline_099  force index (idx_uid_status)
WHERE UID = 1833901412800099
  AND status = 1
ORDER BY create_time DESC
LIMIT 10

CREATE TABLE `t_follow_timeline_099` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT ,
  `uid` bigint(20) NOT NULL DEFAULT '0' ,
  `timeline_id` varchar(32) NOT NULL ,
  `timeline_owner` bigint(20) NOT NULL DEFAULT '0' ,
  `status` tinyint(4) NOT NULL DEFAULT '1' ,
  `timeline_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ,
  `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP ,
  PRIMARY KEY (`id`),
  KEY `idx_uid` (`uid`),
  KEY `idx_create_time` (`create_time`),
  KEY `idx_uid_timeline_owner` (`uid`,`timeline_owner`),
  KEY `idx_uid_timeline_id` (`uid`,`timeline_id`),
  KEY `idx_uid_create_time` (`uid`,`create_time`),
  KEY `idx_uid_status` (`uid`,`status`)
) ENGINE=InnoDB AUTO_INCREMENT=1091861159388237825 DEFAULT CHARSET=utf8 

第一页优化

## 第一页查询
mysql> explain SELECT *
    -> FROM t_follow_timeline_099
    -> force index (idx_uid_status)
    -> WHERE UID = 1833901412800099
    -> AND status = 1
    -> ORDER BY create_time DESC
    -> LIMIT 10;
+----+-------------+-----------------------+------------+------+----------------+----------------+---------+-------------+------+----------+---------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-----------------------+------------+------+----------------+----------------+---------+-------------+------+----------+---------------------------------------+
| 1 | SIMPLE | t_follow_timeline_099 | NULL | ref | idx_uid_status | idx_uid_status | 9 | const,const | 2875 | 100.00 | Using index condition; Using filesort |
+----+-------------+-----------------------+------------+------+----------------+----------------+---------+-------------+------+----------+---------------------------------------+
1 row in set, 1 warning (0.00 sec)

# 执行时间
10 rows in set (0.02 sec)

修改排序为order by id, Because the secondary index with natural primary key, return directly sorted based on the id

#修改成 order by id 
SELECT *
FROM t_follow_timeline_099
force index (idx_uid_status)
WHERE UID = 1833901412800099
  AND status = 1
ORDER BY id DESC
LIMIT 10
#查询耗时: 0.001135 秒

Removing the force index, appears to merge with the primary key

调整索引, 二级索引带上主键

mysql> alter table t_follow_timeline_099 add index idx_uid_status_id(`uid`,`status`,`id`) , drop index idx_uid_status;
Query OK, 0 rows affected (1 min 7.67 sec)
Records: 0 Duplicates: 0 Warnings: 0

翻页查询 通过id实现

mysql> explain SELECT * FROM t_follow_timeline_099 WHERE UID = 1833901412800099 AND status = 1 and id < 1090943061166186580 ORDER BY id DESC LIMIT 10;
+----+-------------+-----------------------+------------+-------+--------------------------------------------------------------------------------------------------+-------------------+---------+------+------+----------+-----------------------+
----+------+----------+-----------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-----------------------+------------+-------+--------------------------------------------------------------------------------------------------+-------------------+---------+--

| 1 | SIMPLE | t_follow_timeline_099 | NULL | range | PRIMARY,idx_uid,idx_uid_timeline_owner,idx_uid_timeline_id,idx_uid_create_time,idx_uid_status_id | idx_uid_status_id | 17 | NULL | 2774 | 100.00 | Using index condition |
+----+-------------+-----------------------+------------+-------+--------------------------------------------------------------------------------------------------+-------------------+---------+------+------+----------+-----------------------+


查询耗时: 0.00155 秒

Optimization summary

mysql on order by statement optimizer

1. Indexed field where the latter condition, it is preferable order by id
2. In order to ensure order by id primary key index will not come, you need to last a composite index, plus id
3. Finally, a combination of the index plus the id, is to enable server layer identify, avoid reached the primary key of the implementation plan, at the level of the storage engine is exactly the same
4. The most elegant way to achieve this than the order by id + 0 lot better, because this is definitely not get to index the primary key, and the above method is also possible
5. order by id limit 10, the current row is not reflected in the implementation of the plan. eg: idx_uid_status_id, condition uid = 123 and status = 1 order by id limit 10, key_len uid_status length is still, but not sorted quick return