Introduction|Remember a thought process of locating SQL problems and performance optimization from the execution plan
1. Problem recurrence:
As written earlier, before the Dragon Boat Festival holiday, children's shoes encountered such a problem- SQL slow query , querying in 300,000 data, a certain interface response took about 30s , and the page directly hung up. Among them, SQL introduced the view, here I won't post the code for now.
2. Hypothetical conjecture:
1. In the sql query condition, the condition field is not indexed ?
2. In the view, are the table access methods , connection order and connection methods reasonable?
3. In the established index, whether the index is hit , whether there is a failure , and use the full table scan ?
3. Thinking process:
It takes time from the request initiated by the page to the response of the backend service interface until the page hangs directly:
1. First , check the page loading time through F12 and add time-consuming logs to the back-end interface to analyze the longest time-consuming stuck in the DAO layer, that is, SQL business logic query-response results.
2. Next , since this is a remote on-site investigation, we log in through the bastion host, then open the Oracle client PL/SQL Developer, and copy the SQL statement printed in the log to query on the client.
3. Then , if you are on a laptop, you can use the Fn+F5 shortcut key to open the current SQL query plan on the client side for specific investigation and further analysis.
Figure 1
Figure II
Figure three
Here, some concepts in the execution plan are explained in detail :
Resource Cost Consumption (COST)
Full table scan (TABLE ACCESS FULL)
Index scan (INDEX SCAN)
Nested loops (NESTED LOOPS)
Hash join (HASH JOIN)
Sort-merge connection (SORT MERGE JOIN)
Among them, each execution step has a corresponding COST. From the level of the single-step COST, and the single-step estimated result set-corresponding ROWS/cardinality, we can analyze whether the table access method, connection sequence, and connection method are reasonable.
We know from the execution plan that two of the tables have been scanned in full - TABLE ACCESS FULL, and the number of scanned rows ROWS and COST are also the highest. At this point in the analysis, the root of the problem is one step closer to us. But when we check the printed SQL statement and do not find the current table name, will it exist in the view view?
We checked each table carefully on the PL/SQL client, and finally found the table that was scanned by the full table in the view view. Now the problem can be solved easily?
The complex SQL statement is resolved as a query step - "use the big to make the small, use the small to make nothing", disassemble it into multiple single-step SQL fragments, and analyze the query plan for each SQL fragment. But here, we can give priority to further investigation on the table that has been scanned by the full table. After the analysis, the previous conjecture was verified.
The tables scanned by the above-mentioned full table are nicknamed A and B tables respectively. There are two fields in A and B tables respectively as the query condition attributes. A fuzzy query like '%A' does not hit the index, and the other Table B is a conditional query without indexing. According to the results of the above investigation, make like 'A%' for table A, and create an INDEX index for the query condition field of table B, and the type is Normal.
Figure four
Ok, now, let's continue to look at the effect of select. The page normally requests the back-end service interface, completes data loading and rendering, and the page finally runs. The back-end service interface responds to time-consuming log printing, which is also time-consuming from the previous 30s request . It was shortened to 1-2s , and the execution plan continued to be viewed on the Oracle client, and there was no full table scan.
4. Solution:
For the invalid index or the index that has not been established for the query condition field, locate the problem and the root cause according to the query plan, and at the same time adjust and establish the appropriate index and type for the fields participating in the condition query.
5. Reflection summary:
Of course, there are many optimization points. The above is just a reasonable adjustment of the index through the query plan . When the data base is very large, the relational database can be divided into databases and tables , and the non-relational database can be used to model the data. Design , select Elastic Search-distributed full-text search -inverted index or MongoDb, etc., and choose the best according to the appropriate business scenario. For details, please refer to the editor's CSDN blog post :
https://blog.csdn.net/yxd179It leads to such a small question-thinking: When you go to the library to borrow books or bookstores, you must have used the type of book and the name of the book to narrow down the scope of your search, so why search through the catalog and index? quick? Under what circumstances will the index become invalid? How are the bottom layers of Oracle, Mysql, and ES indexes implemented? ^_^
Of course, we can also look at the underlying data structure, B-tree, B+ tree, Hash index, inverted index...
Note: On the way back from the high-speed rail, it is better to mark the previous thinking process. Persistence is not easy-self-driven. Due to time and other reasons, the explanation may not be complete. This article is here for the time being. For SQL performance tuning, there are many ways. I hope that readers can be helpful in the process of locating database SQL problems. Valuable suggestions are welcome^_^