After repeated failures, I summarized this million-character "MySQL Performance Tuning Notes"

Summary of "MySQL Performance Tuning Learning Map"

Note: This article is limited in space, so only part of the content is shown. "MySQL Performance Tuning Learning Map" is not easy to collect and organize. Friends in need can help please like it, and then click here to get the download method for free

One, performance monitoring

1. Use the show profile query analysis tool to specify a specific type

This tool is disabled by default and can be dynamically modified at the drawing level through server variables

set profiling=1;

When the setting is completed, all statements executed on the server will measure the time spent and other data related to the change of the execution state of the query.

select * from emp;

In the mysql command line mode, only two decimal places of time can be displayed. You can use the following command to view the specific execution time

show profiles;

Execute the following command to view the detailed time of each step:

show profile for query 1;

2. Use performance schema to monitor mysql more easily

Detailed MYSQL performance schema

• Introduction to performance_schema
• Getting started with performance schema
• Classification of performance_schema table
• Simple configuration and use of performance_schema
• Parameter description of common configuration items
• Description of important configuration tables
• performance_schema practice operation

3. Use show processlist to view the number of connected threads to observe whether a large number of threads are in an abnormal state or other abnormal characteristics

• id means session id
• user represents the user of the operation
• host represents the host of the operation
• db represents the operating database
• command indicates the current state
• info indicates the detailed sq1 language direction
• time represents the execution time of the corresponding command
• state indicates the command execution state

Two, schema and data type optimization

1. Optimization of data types

Smaller is usually better: Try to use the smallest data type that can store data correctly. Smaller data types are usually faster because they take up less disk, memory, and CPU cache, and require fewer CPU cycles for processing. , But make sure you don’t underestimate the range of values ​​that need to be stored. If you can’t confirm which data type, choose the smallest type that you think will not exceed the range

Simple is better: operations of simple data types usually require less CPU cycles, for example, ①, integers are cheaper than character operations, because the character set and collation rules are that character comparison is more complicated than integer comparison; ②, use mysql Self-built type instead of string to store date and time; ③, use integer to store IP address

Try to avoid null: If the query contains a column that can be NULL, it is difficult to optimize for mysql, because the column that can be null makes indexes, index statistics, and value comparison more complicated. Frankly speaking, usually null columns The performance improvement brought by changing to not null is relatively small, so there is no need to modify the schema of all tables, but you should try to avoid designing columns that can be null

Actual rules:

2. Fair use of paradigm and anti-paradigm

①, paradigm

advantage:

• Normalization updates are usually faster than anti-paradigm
• When the data is better normalized, there is little or no duplicate data
• Normalized data is relatively small, can be stored in memory, and the operation is faster

Disadvantages:

• Usually need to be associated

②, anti-paradigm

advantage

• All data are in the same table, which can avoid association
• Can design an effective index;

Disadvantage

• There is a lot of redundancy in the table, and some useful information in the table will be lost when data is deleted.

③, pay attention

3. Selection of Primary Key

Surrogate primary key: a non-business-related, meaningless sequence of numbers

Natural primary key: the natural unique identifier in the properties of things

It is recommended to use a surrogate primary key

• They are not coupled to the business, so they are easier to maintain
• A majority table, preferably all tables, a common key strategy can reduce the amount of source code that needs to be written and reduce the total cost of ownership of the system

4. Selection of storage engine

5. Appropriate data redundancy

• Independent small fields that are frequently quoted and can only be obtained by joining 2 (or more) large tables.
• In such a scenario, each time Join is only to obtain the value of a small field, and the record of Join is large, which will cause a lot of unnecessary IO, which can be optimized by the way of space for time. However, while redundancy, it is necessary to ensure that the consistency of the data will not be destroyed, and to ensure that the redundant fields are also updated during the update.

6. Properly split

When our table has a large field similar to TEXT or a very large VARCHAR type, if most of our access to this table does not need this field, we should split it into another without hesitation In a separate table, to reduce the storage space occupied by frequently used data. An obvious benefit of this is that the number of data items that can be stored in each data block can be greatly increased, which not only reduces the number of physical IOs, but also greatly improves the cache hit rate in the memory.

Three, implementation plan

Fourth, optimize through the index

1. Basic knowledge of indexing

• Advantages of indexing
• Usefulness of the index
• Index classification
• Interview technical terms
• Data structure used by the index
• Index matching method

2. Hash Index

• Based on the implementation of the hash table, only queries that exactly match all the columns of the index are valid
• In mysql, only memory storage engine explicitly supports hash index
• The hash index itself only needs to store the corresponding hash value, so the structure of the index is very compact, which makes the hash index lookup very fast
• Free theme

3. Combined Index

When multiple columns are included as an index, it should be noted that the correct order depends on the query of the index, and it is necessary to consider how to better meet the needs of sorting and grouping

4. Clustered index and non-clustered index

• Clustered index: not a separate index type, but a data storage method, which refers to the compact storage of data rows with adjacent key values
• Non-clustered index: data files are stored separately from index files

5. Covering Index

• basic introduction
• Advantage

6. Optimize small details

• When using index columns for queries, try not to use expressions, and put calculations in the business layer instead of the database layer
• Try to use the primary key query instead of other indexes, so the primary key query will not trigger back to the table query
• Use prefix index
• Use index scan to sort
• Union all, in, or can use indexes, but in is recommended
• Range column can use index
• Forced type conversion will scan the entire table
• Updates are very frequent, and indexing is not suitable for fields with low data discrimination
• Create an index column, it is not allowed to be null, you may get results that do not meet expectations
• When you need to join tables, it is best not to exceed three tables, because the data types of the fields that need to be joined must be consistent
• Try to use limit when you can use limit
• The single table index is recommended to be controlled within 5
• The number of single index fields is not allowed to exceed 5 (combined index)
• The following misconceptions should be avoided when creating an index

7. Index Monitoring

Five, query optimization

1. Reasons for slow query

• The internet
• CPU
• I
• Context switch
• System call
• Generate statistics
• Lock wait time

2. Optimize data access

The main reason for poor query performance is too much data to be accessed. Some queries inevitably need to filter a large amount of data. We can optimize by reducing the amount of data accessed.

Did you request unneeded data from the database

• Query unwanted records
• Return all columns when multiple tables are associated
• Always fetch all columns
• Query the same data repeatedly

3. Optimization of the execution process

Query cache: Before parsing a query statement, if the query cache is turned on, mysql will first check whether the query hits the data in the query cache. If the query happens to hit the query cache, it will check user permissions before returning the result , If there is no problem with the permissions, then mysql will skip all stages and get the result directly from the cache and return it to the client

Query optimization processing: After mysql queries the cache, it will go through the following steps: parsing SQL, preprocessing, and optimizing the SQL execution plan. Any errors in these steps may terminate the query

4. Optimize specific types of queries

• Optimize count() query
• Optimize related queries
• Optimize subqueries
• Optimize limit paging
• Optimize union query
• Recommend the use of user-defined variables

Six, partition table

1. Application Scenarios of Partition Table

• The table is so large that it cannot be all stored in memory, or there is hot data only in the last part of the table, and the rest are historical data
• Partition table data is easier to maintain
• The data of the partition table can be distributed on different physical devices, thereby efficiently using multiple hardware devices
• Partition table can be used to avoid some special bottlenecks
• Independent partition can be backed up and restored

2. Limitations of the partition table

• A table can only have 1024 partitions at most, and 8196 partitions can be supported in version 5.7
• In the early MySQL, the partition expression must be an integer or an expression that returns an integer. In MySQL 5.5, columns can be used directly for partitioning in some scenarios.
• If there are primary key or unique index columns in the partition field, then all primary key columns and unique index columns must be included
• Partitioned tables cannot use foreign key constraints

3. The principle of partition table

4. Types of partition table

• Range partition
• List partition
• Column partition
• hash partition
• key partition
• Subpartition

5. How to use partition table

If you need to query the records of a certain period of time from a very large table, and this table contains many years of historical data, the data is sorted by time, how to query the data at this time?

Because of the huge amount of data, it is certainly not possible to scan the entire table every time you query. Taking into account the space and maintenance consumption of the index, I do not want to use the index. Even if you use the index, you will find that a large number of fragments will be generated and a large amount of random IO will be generated. However, when the amount of data is too large, the index cannot be used. Function, at this time you can consider using partitions to solve

6. Issues that need to be paid attention to when using partition tables

• nul1 value will invalidate partition filtering
• The partition column and index column do not match, which will cause the query to fail to partition and filter
• The cost of choosing a partition can be high
• The cost of opening and locking all underlying tables can be high
• The cost of maintaining partitions can be high

Seven, server parameter settings

1. general

2. character

3. connection

4. log

5. cache

6. INNODB

Learning testimonials

Success is not in the future, but is accumulated from the moment you decide to do it.

Remember to help like + comment, click here to get the download method for free