oracle-based sql optimization

oracle-based sql optimization

[Blogger] Gao Ruilin 

[Blog address] http://www.cnblogs.com/grl214 

one. Write a description of the original intention

In the early stage of the development of the due system, due to the lack of database data, the performance of SQL cannot be reflected in the writing of various SQL statements. There is a hundredfold gap, which shows the importance of sql optimization

two. Sql statement performance optimization

2.1 Understand the execution process of Oracle

2.2 Oracle Optimization Rule---Funnel Rule

2.3 Oracle Execution Plan

2.3.1 What is an Oracle Execution Plan

The execution plan is a description of the execution process or access path of a query statement in Oracle.

2.3.2 View Oracle Execution Plan

1. Explanation of column fields commonly used in execution plans

Cardinality: The number of result set rows returned

bytes: The number of bytes returned after executing the step

Cost (cust), CPU cost: The execution cost of this step estimated by Oracle is used to illustrate the cost of SQL execution. In theory, the smaller the better.

2.3.3 Understanding the Oracle Execution Plan

 

2.3.3.1 Execution order

Judging by the indentation, the one with the most indentation is executed first (when the indentation is the same, the topmost one is executed first)

2.4 How to access the table

• TABLE ACCESS FULL (full table scan)
• TABLE ACCESS BY ROWID (table access by rowid)
• TABLE ACCESS BY INDEX SCAN (index scan)

 

2.4.1 ABLE ACCESS FULL (full table scan)

Oracle will read all the rows in the table and check whether the conditions in the where statement are met;

Recommendation for use: full table scan is not recommended for tables with too much data

2.4.2 TABLE ACCESS BY ROWID (table access via ROWID)

Explanation of ROWID: Oracle will automatically add a pseudo-column in the last column of each row of the table. The value of ROWID is not physically stored in the table. Once a row of data is inserted, its corresponding ROWID is unique in the life cycle of the row. Yes, even if a row migration occurs, the row's ROWID value does not change.

2.4.3 TABLE ACCESS BY INDEX SCAN (index scan)

In the index block, the key value of each index is stored, and the ROWID with the key value pair is also stored.

The scan of the index is divided into two steps: the first is to find the ROWID to which the index is paired, and the second is to read the changed row data through the ROWID

There are five types of index scans:

• INDEX UNIQUE SCAN (index unique scan)
• INDEX RANGE SCAN
• INDEX FULL SCAN
• INDEX FAST FULL SCAN
• INDEX SKIP SCAN

(a).INDEX UNIQUE SCAN (index unique scan) :

For the scan of the unique index (UNIQUE INDEX), at most one record is returned at a time, mainly for the primary key or unique of this field;

(b). INDEX RANGE SCAN

Use an index to access multiple rows of data;

There are three situations in which an index range scan occurs:

• A range operator is used on a unique indexed column (eg: > < <> >= <= between)
• On the composite index, only some columns are used for query (the leading column must be included in the query, otherwise a full table scan will be taken)
• Any query on a non-unique indexed column

(c). INDEX FULL SCAN (index full scan)

• When performing a full index scan, the data queried must be directly available from the index

(d). INDEX FAST FULL SCAN

• Scans all data blocks in the index, similar to INDEX FULL SCAN, but with one notable difference is that it does not sort the queried data (that is, the data is not returned in sorted order)

(e). INDEX SKIP SCAN :

Provided after Oracle 9i, sometimes the leading column of the composite index (the first column contained in the index) does not appear in the query statement, and the composite index will also be used by oralce. At this time, the INDEX SKIP SCAN is used;

When Oracle finds that the number of unique values ​​in the leading column is small, it will use each unique value as the entry of a regular scan, perform a search on this basis, and finally merge these queries;

E.g:

Suppose the table emp has three fields: ename (employee name), job (job name), sex (gender), and a composite index such as create index idx_emp on emp (sex, ename, job) is established;

Because the gender has only two values, 'male' and 'female', in order to improve the utilization of the index, Oracle can split this composite index into ('male', ename, job), ('female', ename, job) two compound indexes;

When the query select * from emp where job = 'Programmer', after the query is issued:

Oracle first enters the entry where sex is 'male'. At this time, the composite index ('male', ename, job) is used to find the entry with job = 'Programmer';

Then enter the entry where sex is 'female', at this time, the composite index ('female', ename, job) is used to find the entry of job = 'Programmer';

Finally, merge the query result sets from the two portals.

2.5 Sql statement processing process

 

1. Find the SQL statement in the shared pool

2. Check grammar

3. Check semantics and related permissions

4. Merge (MERGE) view definitions and subqueries

5. Determine the execution plan

Binding (BIND) :

1. Find the bind variable in the statement

2. Assignment (or reassignment)

Execute (EXECUTE) :

1. Application execution plan

2. Perform necessary I/O and sorting operations

Extract (FETCH) :

1. Return records from query results

2. Sort if necessary

3. Use ARRAY FETCH mechanism

Shared Cursors: Benefits

1. Reduce parsing

2. Dynamic memory adjustment

3. Improve memory usage

2.5.1 Sql sharing principle

Oracle puts the sql statement during execution in the shared pool of memory, which can be shared by all database users. When executing a sql statement, if it is exactly the same as the previous sql execution statement, oracle will quickly get it and be parsed statement and the best way to execute it.

This system belongs to a global area, but Oracle only provides cache for simple tables. If it is a multi-table join query, the database administrator must set appropriate parameters for this area in the startup parameter file to increase the possibility of sharing.

2.5.2 Conditions for Sql sharing (precautions)

1. The execution statement must be exactly the same as the shared pool statement, including (case, spaces, newlines, etc.).

2. The objects referred to by the two statements must be exactly the same.

3. The names of the bind variables of the two SQL statements must be the same.

Example: Character-level comparison

SELECT * FROM UR_USER_INFO

Select * from ur_user_info

Example: same bind variable name

select pay_fee,pay_method from bal_payment_info where pay_sn= : pay_sn;

select pay_fee,pay_method from bal_payment_info where pay_sn= : pay_no;

Bind variables are different and cannot be shared.

2.5.3 Shared SQL area

 

2.5.4 Sql parsing and sharing sql statement

When an Oracle instance receives an sql

1. Create a Cursor

2. Parse the Statement

3. Describe Results of a Query

4. Define Output of a Query Define the output data of the query

5. Bind Any Variables

6. Parallelize the Statement to execute the statement in parallel

7. Run the Statement

8. Fetch Rows of a Query

9. Close the Cursor

2.6 Bind variables

2.6.1 Recompilation problems

E.g:

select  *from ur_user_info where contract_no = 32013484095139

The following statement needs to be hard parsed in SHARE POOL every time it is executed.

times, a million users is a million times, consuming CPU and memory, if the business

The large amount is likely to lead to the downtime of the library...

If you bind a variable, you only need to hard parse it once and call it repeatedly

2.6.2 Bind variables to solve the problem of recompilation

E.g:

select  *from ur_user_info where contract_no = 32013484095139

select  *from ur_user_info where contract_no = 12013481213149

Use bind variables

select  *from ur_user_info where contract_no =：contract_no

2.6.3 Notes on bind variables

a. Do not use the database-level variable binding parameter cursor_sharing to force

control binding, whether its value is force or similar

b. Some statements with >< may cause the optimizer to fail to correctly bind variables.

use index

2.5 Principles and precautions for SQL optimization

• Target:

(1). General principles of SQL optimization design:

• Design:

(1). Try to rely on oracle's optimizer and provide conditions for it;

(2). Appropriate index, double effect of index, selectivity of column;

• Coding:

(1). Use indexes to avoid large tables FULL TABLE SCAN;

(2). Reasonable use of temporary tables;

(3). Avoid writing overly complex SQL, not necessarily one SQL to solve the problem;

(4) Reduce the granularity of transactions without affecting the business;

2.5.1 IS NULL 与IS NOT NULL

Any SQL statement as long as adding is null or is not null after the where statement, then the oracle optimizer will no longer use the index.

2.5.2 Using statements with wildcards (%)

Two examples are given to illustrate the problem:

Query the service number with 10 in phone_no in the ur_user_info table

例子1：Select *from ur_user_info where phone_no like ‘%10%’；

例子2：Select *from ur_user_info where phone_no like ‘10%’；

Since the wildcard (%) appears at the beginning of the search term in Example 1, the oracle system does not use the index of phone_no, and the wildcard will reduce the efficiency of the query, but when the wildcard does not appear first, the index can be used, as shown in Example 2.

 

three. ORACLE statement optimization rules

 

3.1 Choose the most efficient table name order

For example: TAB1 1000 records, TAB2 1 record

Select the table with the fewest records as the base table

Select count(*) from tab1,tab2;

If there are 3 or more tables, select the cross table as the base table

3.2 Join order in where clause

The parsing of oracle is parsed from top to bottom, so the connection between tables must be written before the where condition:

E.g:

ineffective:

select .. from

             emp e

             where sal > 50000 and job = 'manager'

             and 25 < (select count(*) from emp where mgr=e.empno);

high efficiency:

select .. from

emp e

where 25 < (select count(*) from emp where mgr=e.empno)

             and sal > 50000

             and job = 'manager';

3.3 Use of wildcard '*'

When SQL executes a statement with wildcards, if '%' is in the first place, the primary key or index established on the field will be invalid!

Similar statements should be avoided

Select name from user_info where name=’%A’;

3.4 Use truncate instead of delete

When the table is deleted, use delete to perform the operation, and the rollback terminal is used to store recoverable information. When the transaction is not committed, the rollback transaction is executed, and the data will be restored to before the delete operation is performed, and when truncate is used, the rollback is performed. The terminal will not store recoverable information, reducing the invocation of resources.

3.5 Replacing the HAVING clause with a where clause

Avoid using the HAVING clause, HAVING will only filter the result set after all records have been retrieved. This processing requires sorting, totaling, etc. If you can limit the number of records by the WHERE clause, you can reduce this overhead. .

3.6 Reduce queries to tables

Inefficient:

Select tab_name from tables where tab_name = ( select

tab_name from tab_columns where version = 604) and db_ver=

( select db_ver from tab_columns where version = 604)

Efficient:

select tab_name from tables where (tab_name,db_ver) =

( select tab_name,db_ver) from tab_columns where version =604)

3.7 Use in instead of or

Inefficient :

Select.. from location where loc_id = 10 or loc_id = 20 or loc_id = 30

Efficient :

Select..from location where loc_in in (10,20,30);

3.8 Deduplication

The most efficient way to delete duplicate records

Delete from ur_user_info a

     Where a.rowid>(select min(b.rowid)

              From ur_user_info b

         Where b. uid=a. uid);

 

3.9 Avoid resource-intensive operations

SQL statements with DISTINCT, UNION, MINUS, INTERSECT, ORDER BY will start the SQL engine to perform a resource-intensive sorting (SORT) function. DISTINCT requires one sorting operation, while others require at least two sorting operations.

For example, a UNION query, where each query has a GROUP BY clause, GROUP BY will trigger a nested sort (NESTED SORT); thus, each query needs to perform a sort, and then when the UNION is performed, another unique sort ( SORT UNIQUE) operation is executed and it can only start executing after the previous embedded sort has finished. The depth of the embedded sort can greatly affect the efficiency of the query.

3.10 Automatic index selection

If there are more than two (including two) indexes in the table, one of them is a unique index, and the others are non-unique. In this case, ORACLE will use the unique index and completely ignore the non-unique index.

Example:

select ename from emp where empno = 2326 and deptno = 20 ; here, only the index on empno is unique, so the empno index will be used to retrieve records.

table access by rowid on emp index unique scan on emp_no_idx；

3.11 At least the first column of the composite index must be included

If the index is built on multiple columns, the optimizer will choose to use the index only if its first column (leading column) is referenced by the where clause. When only the second column of the index is referenced, the optimizer will The server used a full table scan and ignored the index.

3.12 Avoid using functions on indexed columns

Inefficient:

select ..

from dept

where sal * 12 > 25000;

Efficient :

select ..

from dept

where sal  > 25000/12;

3.13 Avoid automatic conversion of index columns

When comparing data of different data types, ORACLE automatically performs simple type conversions on columns.

Suppose EMP_TYPE is an indexed column of type character.

select user_no,user_name,address

from user_files

where user_no = 109204421

This statement is converted by ORACLE to:

select user_no,user_name,address

from user_files

where to_number(user_no) = 109204421 This index will not be used because of the type conversion that happens internally!

3.14 Avoid automatic conversion of index columns

For use:

where a.order_no = b.order_no

 

Need not :

where to_number (substr(a.order_no, instr(b.order_no, '.') - 1)

= to_number (substr(a.order_no, instr(b.order_no, '.') - 1)

3.15 Use DECODE to reduce processing time

E.g:

select count(*) sum(sal)

   from emp

  where dept_no = 0020

and ename like 'smith%';

 

 select count(*) sum(sal)

   from emp

  where dept_no = 0030

    and ename like 'smith%';

 

You can get the same result efficiently with the DECODE function

select count(decode(dept_no, 0020, 'x', null)) d0020_count,

       count(decode(dept_no, 0030, 'x', null)) d0030_count,

       sum(decode(dept_no, 0020, sal, null)) d0020_sal,

       sum(decode(dept_no, 0030, sal, null)) d0030_sal

  from emp

 where ename like 'smith%';

 

3.16 Reduce queries to tables

 inefficient

          select tab_name

          from tables

          where tab_name = ( select tab_name

                                from tab_columns

                                where version = 604)

          and db_ver= ( select db_ver

                           from tab_columns

                           where version = 604)

   efficient

      select tab_name

          from tables

          where  (tab_name,db_ver)

        = ( select tab_name,db_ver)

                   from tab_columns

                   where version = 604)    

3.17 Order by statement

(a). The ORDER BY statement determines how Oracle will sort the returned query results. The Order by statement has no special restrictions on the columns to be sorted, and functions can also be added to the columns (such as joins or appends, etc.). Any non-indexed items or calculated expressions in the Order by statement will slow down the query.

(b). order by statement to find non-indexed items or expressions, which can degrade performance. The solution to this problem is to rewrite the order by statement to use an index, or you can create another index for the column used, and you should absolutely avoid using expressions in the order by clause.

3.18 Using Indexes to Improve Efficiency

An index is a conceptual part of a table that is used to improve the efficiency of retrieving data. ORACLE uses a complex self-balancing B-tree structure. Generally, querying data through an index is faster than a full table scan. When ORACLE finds out, execute the query and Update The ORACLE optimizer will use an index when determining the best path for a statement. Also, using an index can improve efficiency when joining multiple tables. Another benefit of using an index is that it provides primary key uniqueness verification. In general, using an index is particularly effective on large tables. Of course, you will also find that using an index can also improve efficiency when scanning small tables. Although using an index can improve query efficiency, we must also be aware of its cost. . The index needs space to store and also needs to be maintained regularly. Whenever a record is added or removed in the table or the index column is modified, the index itself will also be modified. This means that the INSERT, DELETE, UPDATE of each record will be for this 4 or 5 extra disk I/Os. Because indexes require additional storage space and processing, those unnecessary indexes will actually slow down query response time. Periodic reconstruction of the index is necessary.

3.19 Avoid using calculations on indexed columns

In the WHERE clause, if the indexed column is part of the function. The optimizer will use full table scans instead of indexes.

Inefficient:

     SELECT … FROM  DEPT  WHERE SAL * 12 > 25000;

Efficient:

     SELECT … FROM DEPT WHERE SAL > 25000/12;

 

3.20 Replacing > with >=

If there is an index on DEPTNO.

  Efficient:

   SELECT *

   FROM EMP

   WHERE DEPTNO >=4

  

   Inefficient:

   SELECT *

   FROM EMP

   WHERE DEPTNO >3

3.21 Avoid using NOT commands by using >=, <=, etc.

example:

select * from employee where salary <> 3000;

This query can be rewritten to not use NOT:

select * from employee where salary<3000 or salary>3000;

         Although the results of the two queries are the same, the second query scheme will be faster than the first query scheme. The second query allows Oracle to use an index on the salary column, while the first query cannot.

3.22 Quotes for character fields

For example, the PHONE_NO field of some tables is of type CHAR, and an index is created.

But if you forget to put quotes in the WHERE condition, the index will not be used.

WHERE PHONE_NO=‘13920202022’

WHERE PHONE_NO=13920202022

 

Four. Optimization summary

 

a. When creating the table. The primary key should be established as much as possible, and the PCTFREE and PCTUSED parameters of the data table should be adjusted as much as possible according to actual needs; if the large data table is deleted, use truncate table instead of delete.

b. Use indexes reasonably. In OLTP applications, there should not be too many indexes on a table. Do not build a binary tree index for columns with a large amount of data repetition, but use a bitmap index; the column order of the combined index should be as consistent as possible with the query condition column order; for tables with frequent data operations, the index needs to be periodically rebuilt to reduce invalid indexes and fragmentation .

c. The query should use a certain column name as much as possible, and use less *.

select count(key)from tab where key> 0性能优于select count(*)from tab；

d. Try to nest subqueries as little as possible, which will consume a lot of CPU resources; for queries with more or operations, it is recommended to divide them into multiple queries and join them with union all; in the query statement of multi-table query, choose the most Efficient table name ordering. The Oracle parser parses tables from right to left, so tables with fewer records are placed on the right.

e. Use commit statements to submit transactions as much as possible, which can release resources, unlock, release log space in time, and reduce management costs; in frequent data operations with high performance requirements, try to avoid remote access, such as database chains, etc. Tables can be resident in memory: alter table. . . cache;

f. To dynamically execute SQL in Oracle, try to use the execute method instead of the dbms_sql package.

references

Oracle SQL Statement Optimization 2010 by Black_Snail

"Analysis of Typical Cases of Oracle-based SQL Optimization" 2013 Author: dbsnake @dbsnake