Query optimization rules for millions of data:
1. For query optimization, full table scan should be avoided as much as possible, and indexes should be established on the columns involved in where and order by.
2. Try to avoid the null value judgment of the field in the where clause, otherwise the engine will give up the use of the index and perform a full table scan, such as:
select id from t where num is null
You can set the default value of 0 on num to ensure that There is no null value in the num column in the table, and then query like this:
select id from t where num=0
3. Try to avoid using the != or <> operator in the where clause, otherwise the engine will give up the use of the index and perform the full Table scan.
4. Try to avoid using or in the where clause to connect the conditions, otherwise the engine will give up the use of the index and perform a full table scan, such as:
select id from t where num=10 or num=20
can be queried like this:
select id from t where num=10
union all
select id from t where num=20
5. In and not in should also be used with caution, otherwise it will result in a full table scan, such as:
select id from t where num in(1,2,3)
For For consecutive values, you can use between instead of in:
select id from t where num between 1 and 3
6. The following query will also result in a full table scan:
select id from t where name like '%abc%'
7. Try to avoid expression operations on fields in the where clause, which will cause the engine to give up the use of indexes and perform full table scans. For example:
select id from t where num/2=100
should be changed to:
select id from t where num=100*2
8. You should try to avoid performing functional operations on fields in the where clause, which will cause the engine to abandon the use of indexes instead of Do a full table scan. Such as:
select id from t where substring(name,1,3)='abc'--name id starting with abc
should be changed to:
select id from t where name like 'abc%'
9. Don't use it in the where clause Functions, arithmetic operations, or other expression operations are performed on the left side of the "=", otherwise the system may not use the index correctly.
10. When using an index field as a condition, if the index is a composite index, the first field in the index must be used as a condition to ensure that the system uses the index,
otherwise the index will not be used and should be As much as possible, make the field order consistent with the index order.
11. Don't write some meaningless queries. If you need to generate an empty table structure:
select col1,col2 into #t from t where 1=0
This kind of code will not return any result set, but it will consume system resources and should be changed Like this:
create table #t(...)
12. Many times using exists instead of in is a good choice:
replace select num from a where num in(select num from b)
with the following statement:
select num from a where exists(select 1 from b where num=a.num)
13. Not all indexes are valid for queries, the SQL is Query optimization is performed based on the data in the table. When a large amount of data in the index column is repeated, the SQL query may not use the index. For example,
if there are fields in the table sex, male and female are almost half and half, then even if an index is built on sex It also has no effect on query efficiency.
14. The more the index, the better. Although the index can improve the efficiency of the corresponding select, it also reduces the efficiency of the insert and update,
because the index may be rebuilt during the insert or update, so how to build the index needs to be carefully considered. As the case may be.
The number of indexes in a table should not exceed 6. If there are too many indexes, you should consider whether it is necessary to build indexes on some infrequently used columns.
15. Try to use numeric fields as much as possible. If the fields only contain numeric information, try not to design them as character fields, which will reduce the performance of query and connection and increase the storage overhead.
This is because the engine compares each character of the string one by one when processing queries and joins, whereas only one comparison is required for numbers.
16. Use varchar instead of char as much as possible, because first of all, the storage space of variable-length fields is small, which can save storage space.
Secondly, for queries, the search efficiency in a relatively small field is obviously higher.
17. Do not use select * from t anywhere, replace "*" with a list of specific fields, and do not return any fields that are not used.
18. Avoid frequent creation and deletion of temporary tables to reduce the consumption of system table resources.
19. When creating a new temporary table, if a large amount of data is inserted at one time, you can use select into instead of create table to avoid causing a large number of logs
to improve the speed; if the amount of data is not large, in order to ease the resources of the system table, you should first create table, then insert.
20. Try to avoid using the cursor, because the efficiency of the cursor is poor, if the data operated by the cursor exceeds 10,000 rows, then you should consider rewriting.
21. Before using the cursor-based method or the temporary table method, you should look for a set-based solution to solve the problem. The set-based method is usually more efficient.
22. Try to avoid large transaction operations and improve system concurrency capabilities.
23. Try to avoid returning a large amount of data to the client. If the amount of data is too large, you should consider whether the corresponding demand is reasonable.