An article on the study with two Mysql index (an index of high-performance strategy) we mentioned a modified version of the Mysql5.7: "index condition pushdown" (index condition pushdown). Well, today let us uncover the mystery of it.
From the ICP (index condition pushdown) the literal meaning, we doubt that this point should be "pushdown" - a push down. What is push down, push down to where, what's the use? With questions, we start closing and opening some influence ICP sql statement and then answer further questions raised.
First, we can turn on or off The ICP Myslq by the following statement:
SET optimizer_switch = 'index_condition_pushdown=off'; //关闭
SET optimizer_switch = 'index_condition_pushdown=on'; //开启Mysql a new version of this feature is turned on by default, and then we prepare a table:
CREATE TABLE `demo` (
`id` bigint(11) unsigned NOT NULL AUTO_INCREMENT,
`pid` int(11) DEFAULT '0',
`nid` int(5) DEFAULT NULL,
`country` varchar(10) DEFAULT '',
`name` varchar(10) DEFAULT '',
`status` tinyint(1) DEFAULT '1',
`num` int(10) DEFAULT '0',
PRIMARY KEY (`id`) USING BTREE,
KEY `ix_pnnc` (`pid`,`nid`,`name`,`country`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 ROW_FORMAT=COMPACT;Wherein the amount of data in the table shown below:
Without the ICP
Close ICP:
SET optimizer_switch = 'index_condition_pushdown=off';Then call the following statement three queries:
select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%';View three statements by the show profiles query time are:
+----------+------------+-------------------------------------------------------------------------------------------------+
| Query_ID | Duration | Query |
+----------+------------+-------------------------------------------------------------------------------------------------+
| 740 | 0.07347475 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
| 741 | 0.07267875 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
| 742 | 0.072476 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
+----------+------------+-------------------------------------------------------------------------------------------------+The use of the ICP
Open ICP:
SET optimizer_switch = 'index_condition_pushdown=on';
Or call the above piece of sql statement three times, then show profiles View query time:
+----------+------------+-------------------------------------------------------------------------------------------------+
| Query_ID | Duration | Query |
+----------+------------+-------------------------------------------------------------------------------------------------+
| 723 | 0.02307625 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
| 724 | 0.0218665 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
| 725 | 0.02217525 | select * from demo where pid =14 and nid like '%2%' and country like '%a%' and name like '%a%' |
+----------+------------+-------------------------------------------------------------------------------------------------+We see that after opening ICP, as much as three times faster.
We look through explain the difference between the two:
Close ICP situation:
ICP open the case:
We can see that in the Extra field, closed ICP, use the Using where, when open ICP, using Using index condition.
We can see by the above comparison, after using the ICP, we improved our query performance, but have the effect of ICP is not for all sql statement it? We execute the following statement in view, the difference in time of three queries:
select * from demo where pid =14 and num > 0;Close ICP:
+----------+------------+-----------------------------------------------+
| Query_ID | Duration | Query |
+----------+------------+-----------------------------------------------+
| 100| 0.07347475 | select * from demo where pid =14 and num > 0 |
| 101| 0.07788525 | select * from demo where pid =14 and num > 0 |
| 102| 0.07912425 | select * from demo where pid =14 and num > 0 |
+----------+------------+-----------------------------------------------+Open ICP:
+----------+------------+-----------------------------------------------+
| Query_ID | Duration | Query |
+----------+------------+-----------------------------------------------+
| 117| 0.07916325 | select * from demo where pid =14 and num > 0 |
| 118| 0.08040075 | select * from demo where pid =14 and num > 0 |
| 119| 0.07897455 | select * from demo where pid =14 and num > 0 |
+----------+------------+-----------------------------------------------+We found that for this statement, ICP will not have to open or close any query performance.
Then the following strong brother and take a look at why it.
How to understand ICP
Index Condition Pushdown (ICP) is an index to a MySQL an optimized list of access to data. If the ICP is disabled, the engine will pass through the layers to find the index data rows in the base table, and then returns to the MySQL Server layer, filtered again after the WHERE condition data for these rows. ICP is enabled, if part of the WHERE condition in the field can use the index, MySQL Server will be pushed down to this part of the engine layer. Storage Engine evaluated by using the index entry, and then push the index conditions, using the index to satisfy this line is read from the table. The number of times the storage engine can reduce engine ICP layer access base tables and MySQL Server access.
With two pictures explain:
Close ICP:
At this point, the most left-prefix index in line with the principle of tweets before mentioned, when a column of multi-column index is the scope of the query, the fields will not go after the index.
Open ICP:
After opening ICP, the same query in line with the most left-prefix rules, but when multi-column index of a column is range queries, after the field will still be pushed down to the storage engine (Storage Engine) layer will be evaluated to filter out qualifying data and then return to the Server layer. And because the engine layer will be able to filter out a lot of data, this would no doubt be able to reduce the number of base table and mysql server access. So as to enhance performance.
Data were analyzed to enhance the cause
Close ICP is as follows:
1. Retrieve the next record pid = 14 inside the index and then use the primary key field to read the entire line.
2. Then the complete row using the remaining condition is judged to see whether the conditions for filtering and processing in the Server layer.
Open the ICP is this:
1. Remove the inside from the index the next record pid = 14, and other fields using this index to judge the condition, if the condition is satisfied, step 2. Filtered and processed on the engine layer.
2. The qualifying will be screened using the primary key index to find something this complete line, returned in the previous step.
也就是说： 在没有ICP前，由于优化器只能只能使用前缀索引来过滤满足条件的查询，那么mysql只能够利用索引的第一个字段pid，来扫描demo表满足pid=14条件的记录，而后面的nid和country由于使用了模糊查询，而不能在索引中继续过滤满足条件的记录，这样就导致了Server层对demo表的扫描增加了许多；
有了ICP，mysql在读取demo表前，继续检查满足nid和country条件的记录,这个行为在引擎层完成。直接把过滤好的返回给Server层，就减少了Server层的操作。总之是把之前在SERVER层的下推到引擎层去处理。
这里也就解释了ICP（索引条件下推）其实就是将索引条件下推到引擎层啦。
ICP的使用限制
ICP虽然挺好用的，但是并不是所有的sql都能够通过ICP得到性能提升，就如我们上面的第二条sql就无法通过ICP减少查询时间。因为如果where条件的字段不在索引列中,还是要读取整表的记录到server端做where过滤。
这里列出几点ICP的相关限制：
-
当sql需要全表访问时,ICP的优化策略可用于range, ref, eq_ref, ref_or_null 类型的访问数据方法 。
-
支持InnoDB和MyISAM表。
-
ICP只能用于二级索引，不能用于主索引。
-
并非全部where条件都可以用ICP筛选。如果where条件的字段不在索引列中,还是要读取整表的记录到server端做where过滤。
-
ICP的加速效果取决于在存储引擎内通过ICP筛选掉的数据的比例。
-
5.6 版本的不支持分表的ICP 功能，5.7 版本的开始支持。
-
当sql 使用覆盖索引时，不支持ICP 优化方法。
结论：
ICP的优化在引擎层就能够过滤掉大量的数据，这样无疑能够减少了对base table和mysql server的访问次数，提升了性能。
需要index condition pushdown 的query通常索引的字段出现where子句里面都是范围查询。比如：
select * from tb where tb.key_part1 < x and tb.key_part2 = y
select * from tb where tb.key_part1 = x andtb.key_part2 like '%yyyy%'
select * from tb where tb.key_part1 > x and tb.key_part1 < y and tb.key_part1 > xx and tb.key_part2 < yy但是需要注意的是：
如果索引的第一个字段的查询就是没有边界的比如 key_part1 like '%xxx%'，那么不要说ICP，就连索引都会没法利用。
如果select的字段全部在索引里面，那么就是直接的index scan了，没有必要什么ICP。
参考：
https://www.cnblogs.com/zhoujinyi/archive/2013/04/16/3016223.html
http://blog.codinglabs.org/articles/index-condition-pushdown.html
关注公众号获取更多内容，有问题也可在公众号提问哦：
强哥叨逼叨
叨逼叨编程、互联网的见解和新鲜事
