Maximize SQL Optimization - Subquery Optimization

Abstract: Editor's Notes: Subquery is an important syntax in SQL. Proper application will greatly improve the performance of SQL. If it is not used properly, it may cause many problems. Therefore, subqueries are also the difficult part of SQL to optimize. Today, let's learn the most common ways to optimize subqueries.
Editor's Notes: Subquery is an important syntax in SQL. Proper application will greatly improve the performance of SQL. If it is not used properly, it may cause many problems. Therefore, subqueries are also the difficult part of SQL to optimize. Today, let's learn the most common ways to optimize subqueries.
About the author:
Han
Feng640?wx_fmt=jpeg&wxfrom=5&wx_lazy=1 is
proficient in various relational databases including Oracle, MySQL, informix, etc., and has rich experience in database architecture design and development. Work at Yixin.
For subqueries, the optimizer supports a variety of optimization strategies. Oracle query transformation functions mainly include heuristic (rule-based) query transformation and Cost-based query transformation. For sub-queries, there are Subquery Unnest and Push Subquery. The purpose of query conversion is to convert it into Join (including Semi, Anti Join, etc.), and make full use of efficient access methods such as index and Join technology to improve efficiency. If the subquery cannot be unnest (heuristic), you can choose to convert the subquery to Inline View (based on Cost); if not, then the subquery will be executed last, and you may see an operation similar to Filter.
1. Subquery transformation
Let 's take a look at an example first.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
//The above code prepares the necessary data environment and collects statistics of related objects
640?wx_fmt=png&wxfrom=5&wx_lazy=1
//By default, the above operation is converted into an inter-table association method to execute
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*By prompting no_unnest, subquery de-nesting is prohibited. Once executed in the original way, the role of the subquery part is "FILTER"*/
2. Subquery
merging Subquery merging means that the optimizer no longer generates an execution plan for the subquery alone, but merges the subquery into the main query , and finally generate an optimal execution plan for the combined result. You can control whether to enable or disable subquery merging through the parameter _simple_view_merging or the prompt MERGE/NO_MERGE.
According to the complexity of sub-queries, sub-queries can be divided into simple sub-queries and complex sub-queries.
The so-called simple subquery refers to the case where the subquery fields can be simply projected to the outside. In this case, the optimizer adopts a heuristic strategy, that is, it merges when the conditions are met.
A complex subquery is when there is a grouped row count. In this case, the optimizer adopts a cost-based strategy, and whether the final conversion depends on the cost. Of course, there are some subqueries that cannot be merged.
Let's take a look at a few examples.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
//In this way, no subquery merge is performed.
Let's force a look at the effect
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/* Here you can see that the inline view is not regenerated, and the subqueries are merged. So why is subquery merge not performed by default? As can be seen from the cost, obviously the cost of not merging is lower */
3. Unnested subqueries
De-nesting subqueries means that when optimizing complex statements with nested subqueries, the query converter will try to expand the subqueries so that the tables in them can be associated with the tables in the main query, so as to obtain better execution plan. Part of subquery anti-nesting is a heuristic query transformation, and part is a cost-based transformation.
There is a parameter in the system to control the unnested subquery - _unnest_subquery. The default setting of parameter _unnest_subquery is false in 8i, and its default setting is true since 9i. However, 9i does not consider cost when non-nested. Only in 10g did you start to consider the cost of the two different options and choose the less expensive one. When upgrading from 8i to 9i, you may want to block non-nesting of certain queries. This can be done with the no_unnest hint in subqueries. In 8i and 9i, if star_transformation_enabled=true, it is disabled when non-nested (ie hints are used). Also controlled by the optimizer parameter _optimizer_unnest_all_subqueries in the 11g environment. In addition, the hint UNNEST/NO_UNNEST can control whether to unnested or not.
Let's look at unnested subqueries with a few examples.
1) IN/EXISTS is converted to SEMI JOIN:
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*The subquery in the example refers to the table DEPT, which is finally converted into a hash semi-join of the two tables. That is, the subquery in the exists clause is expanded, and the objects in it are directly semi-associative with the objects in the main query */
// The case of IN is similar, as follows:
640?wx_fmt=png&wxfrom=5&wx_lazy=1
2) IN/EXISTS to ANTI JOIN:
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*The optimizer unnested the subquery after NOT EXISTS, and then chose the anti-join of the hash. This transformation is a cost-based query transformation. */
//Let's look at the case of NOT IN
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*Similar to NOT EXISTS, hash join is also selected, but HASH JOIN ANTI NA. The NA here actually means Null-Aware. In 11g and later versions, Oracle has added support for null-sensitive anti-association */
3) De-
nesting of correlated sub-queries In the solution of correlated sub-queries During the nesting process, an inline view is constructed from the subquery, and the inline view is associated with the table of the main query. This operation can be controlled by the parameter _unnest_subquery. This transformation is a heuristic query transformation.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*The inline view is generated in the step of ID=2, and then the hash join with the external table. Let's try to modify the parameters to see how the optimizer handles */
640?wx_fmt=png&wxfrom=5&wx_lazy=1
//This is converted into a special FILTER of nested loops
4. Subquery
advancement Subquery advancement is a Complementary optimization techniques for merge or anti-nested subquery optimization. This technology was introduced in version 9.2. Usually, sub-plans of sub-queries that cannot be merged or de-nested are placed at the end of the entire query plan, and sub-query advancement enables sub-queries to be evaluated earlier, so that they can appear earlier in the overall execution plan steps to obtain a better execution plan. Can be controlled by PUSH_SUBQ/NO_PUSH_SUBQ.
640?wx_fmt=png&wxfrom=5&
//By default, the subquery advance technique is used.
For comparison, let's take a look at the case where it is not used forcibly.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*Compare the above, one step FILTER. Nested loops are used here, each EMP table record corresponds to a sub-query query, and the MAX value is obtained */
5. Sub-query decomposition The
so-called sub-query decomposition means that complex query statements created by WITH are stored in a temporary table , use the functionality of this temporary table in the same way as a normal table. Conceptually it is similar to nested views, but each has its advantages and disadvantages. The advantage is that if the subquery is referenced multiple times, the nested view needs to be executed multiple times, especially in the case of very few results that meet the conditions in the massive data, the difference between the two is obvious.
The advantage of using WITH subquery is that its complex query statement only needs to be executed once, but the result can be used multiple times in the same query statement.
The disadvantage is that the WITH subquery is used. Since the query statement is not allowed to be deformed, there are many invalid cases. In particular, the temporary table created by the query statement in WITH cannot have an index. When the data volume of the query result is relatively large, it is likely to affect the execution efficiency.
The following is an example to see.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*As can be seen from the above, there are two subquery statements in WITH, but only one temporary table is created, this is because the second subquery in WITH uses the first one The execution result of the subquery. In this case, only one temporary table is logically allowed to be created, there is no need to create it again. When processing WITH temporary tables, if the temporary table can be executed first and the amount of data before the connection can be reduced, a nested loop connection can be used, otherwise a hash connection must be used */
6. Subquery cache
For some subquery operations, the optimizer can cache the results of the subquery to avoid repeated reads. This property can be observed in either FILTER-type subqueries or scalar subqueries. See an example.
640?wx_fmt=png&wxfrom=5&wx_lazy=1
/*Note the Start=3 in the Id=3 step (the deptno in the emp table has 3 different values, so it is repeated 3 times here). This reflects the Cache technology, and there is a similar Cache technology in scalar sub-queries. */
Subquery related articles:
Oracle performance optimization query conversion class - self-query