Query performance optimization
Why is the query slow
A query is a task that consists of a series of subtasks, each of which consumes a certain amount of time. If you want to optimize a query, you actually want to optimize its subtasks, either by eliminating some of them, or by reducing the number of subtasks that are executed and the response time. The tasks are: client-server-generate execution plan-execute-return, the most important of which is execution, including a large number of calls to retrieve data to the storage engine and data processing after calls, including sorting, grouping, etc.
When completing these tasks, the time it takes to query, including network, CPU calculation, generating statistics, execution plans, locks (mutual exclusion wait) and other operations. In particular, calls to retrieve data from the underlying engine require time-consuming memory operations, CPU operations, and I/O operations caused by insufficient memory.
Slow Query Basics: Optimizing Data Access
The most basic reason for low query performance is that too much data is accessed. To analyze query performance, follow the steps below:
1. Confirm whether the application is retrieving more data than needed, which usually means that too many rows, possibly too many columns, are accessed.
2. Confirm whether the mysql service layer is analyzing a large number of data rows that exceed the requirements.
Whether unwanted data is requested from the database
Sometimes the query will request more data than the actual data. The consequences of these are, the MySQL server brings additional burden, increases the network overhead, and consumes the CPU and memory resources of the application server.
query unneeded columns
When using select *, you need to look at it with a skeptical eye. Do you really need to return all the columns and take out all the columns, which will make the optimizer unable to complete the optimization such as index coverage scan, and will also bring additional I to the service. /O, memory and CPU consumption.
Repeatedly query the same data
To repeatedly execute the same query, and then return the same data each time, you can cache the queried data first, and retrieve it from the cache when needed. When needed, it will be fetched from the cache, so the performance will be better.
Is MySQL scanning for extra records
After confirming that the query returns the required data, the next step is to see if the query scans too much data in order to return the results. For MySQL, the three simplest indicators of query cost are as follows:
1. Response time
2. The number of lines scanned
3. The number of rows returned
No indicator can perfectly balance the query cost, but it roughly reflects how much data the query needs to access, and calculates the query running time. Checking the slow log is a way to find out if there are too many scanned lines. .
Response time
Response time includes service time and queuing time. Service time refers to how long the database really took to process the query. Queue time is the time the server does not actually execute the query because it is waiting for some resource, either waiting for an I/O operation to complete, or waiting for a lock.
The number of rows scanned and the number of rows returned
Ideally, the number of rows scanned and the number of rows returned should be the same, but it is difficult to achieve in practice. For example, in a relational query, the server must scan multiple rows to generate one row in the result set.
Number of rows scanned and access type
The TYPE column in the EXPLAIN statement reflects the access type. There are many access types, from full table scan to index scan, range scan, unique index query, constant reference, and so on. The speed is from slow to fast, and the number of lines scanned is also from large to small. If the query can not find a suitable access type, then the best solution is to add a suitable index. Indexes allow MySQL to find the records it needs in the most efficient way that scans the fewest rows. General MySQL can apply the WHERE condition in three ways, from good to bad:
Use where conditions in the index to filter unmatched records, which is done by the storage engine layer.
Use an index coverage query (using inex in the Extra column to return records) to return records, filter unwanted records directly from the index and return the hits.
Return data from the data table, then filter records that do not meet the criteria (Using Where appears in the Extra column). This is done in the MySQL service layer, and there is no need to return to the table to query records.
If you find that a query needs to scan a large amount of data but returns only a few rows, you can usually try the following tricks to optimize it:
1. Use index coverage scan to put all required columns into the index, so that the storage engine can return without going back to the table to obtain the corresponding row.
2. Change the library table structure. e.g. using a separate summary table
3. Rewrite the complex query so that the MySQL optimizer can execute the query in a more optimized way.