With as usage/with-as performance tuning/with usage in sql

I. Overview

The with as statement is a common syntax in SQL 它可以为一个查询结果或子查询结果创建一个临时表, and this temporary table can be used in subsequent queries. The temporary table is cleared after the query ends. The use of this syntax can make complex queries simple and improve query efficiency.

The WITH AS phrase, also called subquery factoring, is used to define a SQL fragment that will be used by the entire SQL statement. This statement is considered a common table expression (CTE, Common Table Expression).

with-as meaning:
1. For multiple recurring subqueries, it can reduce the number of table scans and code rewriting, optimize performance and make coding more concise. It can also be used in different parts of UNION ALL as the part that provides data. .
2. For UNION ALL, use WITH AS to define a UNION ALL statement. When the fragment is called more than 2 times, the optimizer will automatically put the data obtained by the WITH AS phrase into a Temp table. The prompt "meterize" forces the data of the WITH AS phrase to be put into a global temporary table. Many queries can be speeded up this way.

The with as statement supports myql, oracle, db2, hive, sql server, MariaDB, PostgreSQL and other databases. The following are the supported versions of several databases.

• mysql版本：8以及8以上的
• sql server: sql server 2005 and later versions
• oracle: Oracle 9i version 2 database

2. Basic grammar

The with query statement does not start with select, but with the "WITH" keyword. It can be understood that a temporary table is pre-constructed before the query is performed, and it can be used multiple times for further analysis and processing.

The CTE is defined using the WITH clause and includes three parts: the CTE name cte_name, the query statement inner_query_definition that defines the CTE, and the external query statement outer_query_definition that references the CTE.
CTE可以在select , insert , update , delete , merge语句的执行范围定义。

Its format is as follows:

WITH cte_name1[(column_name_list)] AS (inner_query_definition_1)
   [,cte_name2[(column_name_list)] AS (inner_query_definition_2)]
[,...]
outer_query_definition

column_name_listSpecify inner_query_definitionthe column list name in. If you do not write this option, you need to ensure that the inner_query_definitioncolumns in have names and are unique. That is, there are two naming methods for column names: internal naming and external naming.

Note thatouter_quer_definition必须和CTE定义语句同时执行，因为CTE是临时虚拟表，只有立即引用它，它的定义才是有意义的。

示例：
-- 单个子查询
with tmp as(select username,userage from user)
select username from tmp

-- 多个子查询  多个CTE 之间加，分割
with tmp1 as (select * from father),
     tmp2 as (select * from child)
select * from temp1,temp2 on tmp1.id = tmp2.parentId

Note :

1. It must be used as a SQL query as a whole, that is, a semicolon cannot be added after the with as statement, otherwise an error will be reported.
2. The with clause must be defined before the referenced select statement. The sibling with keyword can only be used once, and multiples can only be separated by commas; there cannot be a comma between the last with clause and the following query, only through the right Bracket separation, the query with clause must be enclosed in brackets.
3. If the with clause is defined but is not followed by a SQL statement using CTE (such as select, insert, update, etc.), an error will be reported.
4. The query defined in the previous with clause can be used in the subsequent with clause. However, a with clause cannot be nested inside a with clause.
5. If a with clause is defined but not used in a query, an ora-32035 error will be reported: the query name defined in the with clause is not quoted. (The name of at least one with query is not referenced. The solution is to remove the unreferenced with query.) Note: As long as there is a reference later, it does not have to be referenced in the main query. For example, the subsequent with query also It's okay to quote it.
6. When a query block name is the same as a table name or other object, the parser searches from the inside out, giving priority to the subquery block name.
7. The result column of the with query has an alias, and you must use an alias or * when referencing it.

3. Usage scenarios

3.1. Define CTE and rename each column

Test the following sql in mysql 8.0.34 version

CREATE TABLE user(
    id INT NOT NULL PRIMARY KEY,
    sex CHAR(3),NAME CHAR(20)
);

INSERT INTO user VALUES 
(1,'nan','陈一'),
(2,'nv','珠二'),
(3,'nv','张三'),
(4,'nan','李四'),
(5,'nv','王五'),
(6,'nan','赵六');
  
# 定义CTE，顺便为每列重新命名，且使用ORDER BY子句
WITH nv_user(myid,mysex,myname) AS (
    SELECT * FROM user WHERE sex='nv' ORDER BY id DESC
)
# 使用CTE
SELECT * FROM nv_user;
+------+-------+-------------+
| myid | mysex | myname      |
+------+-------+-------------+
|    5 | nv    | 王五        |
|    3 | nv    | 张三        |
|    2 | nv    | 珠二        |
+------+-------+-------------+

3.2. Multiple references/multiple definitions

1. Multiple citations: avoid repeated writing.
2. Multiple definitions: avoid the nesting problem of derived tables.
3. You can use recursive CTE to implement recursive queries.

# 多次引用，避免重复书写
WITH nv_t(myid,mysex,myname) AS (
    SELECT * FROM user WHERE sex='nv'
)
SELECT t1.*,t2.*
FROM nv_t t1 JOIN nv_t t2
WHERE t1.myid = t2.myid+1;
  
# 多次定义，避免派生表嵌套
WITH
nv_t1 AS (          /* 第一个CTE */
    SELECT * FROM user WHERE sex='nv'
),
nv_t2 AS (          /* 第二个CTE */
    SELECT * FROM nv_t1 WHERE id>3
)
SELECT * FROM nv_t2;

If the above statement does not use CTE but uses a derived table, it is equivalent to:

SELECT * FROM
(SELECT * FROM
(SELECT * FROM user WHERE sex='nv') AS nv_t1) AS nv_t2;

You can see that this way of writing is not easy to view.

3.3. Use with and union all together

The query defined in the previous with clause can be used in the subsequent with clause.

with
sql1 as (select  s_name from test_tempa),  
sql2 as (select  s_name from test_tempb where not exists (select s_name from sql1 where rownum=1))  

select * from sql1
union all
select * from sql2
union all
select ‘no records’ from dual
where not exists (select s_name from sql1 where rownum=1)  
and not exists (select s_name from sql2 where rownum=1); 

3.4. with returns the values ​​of multiple results

In actual use, we may encounter scenarios where we need to return values ​​with multiple results.

-- 分类表
CREATE TABLE category ( cid VARCHAR ( 32 ) PRIMARY KEY, cname VARCHAR ( 50 ) );

-- 商品表
CREATE TABLE products (
	pid VARCHAR ( 32 ) PRIMARY KEY,
	pname VARCHAR ( 50 ),
	price INT,
	category_id VARCHAR ( 32 ),
	FOREIGN KEY ( category_id ) REFERENCES category ( cid ) 
);
-- 分类数据
INSERT INTO category(cid,cname) VALUES('c001','家电');
INSERT INTO category(cid,cname) VALUES('c002','鞋服');
INSERT INTO category(cid,cname) VALUES('c003','化妆品');
INSERT INTO category(cid,cname) VALUES('c004','汽车');

-- 商品数据
INSERT INTO products(pid, pname,price,category_id) VALUES('p001','小米电视机',5000,'c001');
INSERT INTO products(pid, pname,price,category_id) VALUES('p002','格力空调',3000,'c001');
INSERT INTO products(pid, pname,price,category_id) VALUES('p003','美的冰箱',4500,'c001');
INSERT INTO products (pid, pname,price,category_id) VALUES('p004','篮球鞋',800,'c002');
INSERT INTO products (pid, pname,price,category_id) VALUES('p005','运动裤',200,'c002');
INSERT INTO products (pid, pname,price,category_id) VALUES('p006','T恤',300,'c002');
INSERT INTO products (pid, pname,price,category_id) VALUES('p007','冲锋衣',2000,'c002');
INSERT INTO products (pid, pname,price,category_id) VALUES('p008','神仙水',800,'c003');
INSERT INTO products (pid, pname,price,category_id) VALUES('p009','大宝',200,'c003');

As shown in the picture above, if I want to query the information of "Gree Air Conditioner" and "Midea Refrigerator" in "Home Appliances", I don't need to write it with as as follows:

select * from category c
left join products p on c.cid = p.category_id
where c.cname = '家电' and p.pname in ('格力空调','美的冰箱');

Use with as to write as follows:

with c as (select * from category where cname = '家电'),
     p as (select * from products where pname in ('格力空调','美的冰箱'))
select * from c,p where c.cid = p.category_id;

②. Query the average price of "Home Appliances" and the minimum and maximum values ​​of all commodities

with tem as (select avg(price) as houseElecAvg from products p
			left join category c on c.cid = p.category_id
			where c.cname = '家电'),
	tem1 as (select max(p1.price),min(p1.price) from products p1)
select * from tem,tem1;

In fact, in addition to being used with SELECT, the WITH expression can also have the following combinations:
insert with, with update, with delete, with with, with recursive (can simulate sequences of numbers, dates, etc.), WITH can define multiple tables

3.5. Use with and insert

insert into table2
with
    s1 as (select rownum c1 from dual connect by rownum <= 10),
    s2 as (select rownum c2 from dual connect by rownum <= 10)
select a.c1, b.c2 from s1 a, s2 b where...;

4. Recursive query

In standard databases, such as hive, Oracle, DB2, SQL SERVER, and PostgreSQL, they all support WITH AS statements for recursive queries. Mysql8.0 and above support recursion.

Common table expressions (CTEs) have the important advantage of being able to reference themselves, thereby creating recursive CTEs. A recursive CTE is a common table expression that repeatedly executes an initial CTE to return a subset of data until a complete result set is obtained.

When a query refers to a recursive CTE, it is called a recursive query. Recursive queries are often used to return hierarchical data, for example: showing employees in an organization chart or a bill of materials scheme (where the parent product has one or more components, and those components may have sub-components, or other parents) data in the product's components).

4.1. Grammar

The recursive CTE contains one or more anchor members, one or more recursive members, and the last anchor member must use "union [all]" (the recursive CTE in mariadb only supports the union [all] set algorithm) to join the first recursive member.

For additional syntax considerations for CTE recursion, see Recursive Common Table Expressions

with recursive cte_name as (
    select_statement_1       /* 该cte_body称为定位点成员 */
  union [all]
    cte_usage_statement      /* 此处引用cte自身，称为递归成员 */
)
outer_definition_statement    /* 对递归CTE的查询，称为递归查询 */

in:

• select_statement_1: called the "anchor member", this is the first part of the recursive cte to be executed, and is also the data source for the recursive member when it starts to recurse.
• cte_usage_statement: called a "recursive member", cte itself must be referenced in this statement. It is the place where recursion really starts in recursive CTE. It first obtains the recursive data source from the anchor point member, and then combines it with other data sets to start the recursion. Each recursion passes the recursive result to the next recursive action, and the query is repeated repeatedly. Finally, the recursion ends when no data can be found.
• outer_definition_statement: It is a query for recursive cte. This query is called "recursive query".

4.2. Usage scenarios

4.2.1. Use with to recursively construct data from 1-10

# n迭代次数
with RECURSIVE c(n) as
 (select 1   union all select n + 1 from c where n < 10)
select n from c;

+------+
| 	 n |
+------+
|    1 |
|    2 |
|    3 |
|    4 |
|    5 |
|    6 |
|    7 |
|    8 |
|    9 |
|   10 |
+------+
10 rows in set (0.00 sec)

4.2.2, with and insert recursively create data

Using WITH expressions to create data is very simple, such as the following example: add 10 records to table y1, and the date field must be random.

-- 创建测试表
create table y1 (id serial primary key, r1 int,log_date date);

-- 插入数据
INSERT y1 (r1,log_date)
   WITH recursive tmp (a, b) AS
   (SELECT
     1,
     '2021-04-20'
   UNION
   ALL
   SELECT
     ROUND(RAND() * 10),
     b - INTERVAL ROUND(RAND() * 1000) DAY
   FROM
     tmp
   LIMIT 10)
select * from  tmp;

result:

4.2.3. Update data with and update

WITH recursive tmp (a, b, c) AS
    (SELECT
      1,
      1,
      '2021-04-20'
    UNION ALL
    SELECT
      a + 2,
      100,
      DATE_SUB(
        CURRENT_DATE(),
        INTERVAL ROUND(RAND() * 1000, 0) DAY
      )
    FROM
      tmp
    WHERE a < 10)
UPDATE
     tmp AS a,
     y1 AS b
   SET
     b.r1 = a.b
   WHERE a.a = b.id;

4.2.4, with and delete delete rows with odd numbers

For example, to delete rows with odd IDs, you can use the WITH DELETE form of the delete statement:

WITH recursive tmp (a) AS
    (SELECT
      1
    UNION
    ALL
    SELECT
      a + 2
    FROM
      tmp
    WHERE a < 10)
    DELETE FROM y1 WHERE id IN (select * from tmp);

When used together with DELETE, one thing to note: the data of the WITH expression itself is read-only, so the WITH expression cannot be included in a multi-table DELETE. For example, changing the above statement to a multi-table deletion form will directly report the error that WITH expression cannot be updated.

WITH recursive tmp (a) AS
     (SELECT
       1
     UNION
     ALL
     SELECT
       a + 2
     FROM
       tmp
     WHERE a < 100)
     delete a,b from y1 a join tmp b where a.id = b.a;

error: [HY000][1288] The target table b of the DELETE is not updatable

4.2.5, with generate date sequence

Use the WITH expression to generate a date series, similar to the generate_series table function of POSTGRESQL. For example, starting from '2020-01-01', generate a date series for one month:

WITH recursive seq_date (log_date) AS
      (SELECT
        '2023-07-09'
      UNION
      ALL
      SELECT
        log_date + INTERVAL 1 DAY
      FROM
        seq_date
      WHERE log_date + INTERVAL 1 DAY < '2023-07-20')
      SELECT
        log_date
      FROM
        seq_date;

+-----------+
|   log_date| 
+-----------+
| 2023-07-09|
| 2023-07-10| 
| 2023-07-11| 
| 2023-07-12| 
| 2023-07-13| 
| 2023-07-14| 
| 2023-07-15| 
| 2023-07-16| 
| 2023-07-17| 
| 2023-07-18| 
| 2023-07-19| 
+------+

Reference documentation
https://docs.oracle.com/cd/E17952_01/mysql-8.0-en/with.html#common-table-expressions
https://dev.mysql.com/doc/refman/8.0/en/with .html#common-table-expressions-recursive

https://blog.csdn.net/weixin_43194885/article/details/122199299?utm_medium=distribute.pc_relevant.none-task-blog-2_defaultbaidujs_baidulandingword~default-1-122199299-blog-74002447.235^v38pc_relevant_anti_t3_base&spm=1001.2101.3001.4242.2&utm_relevant_index=4

https://www.jb51.net/article/236061.htm