mysql optimization entry point:
- Advance:
When designing a database: design of database tables and fields (vertical sub-table), storage engine.
Horizontal expansion: MySQL cluster, load balancing, separation of reads and writes, and sub-databases and tables.
- In the event:
To optimize SQL statements, make good use of the functions provided by MySQL itself, such as indexes.
- afterwards:
The slow query Explian in mysql looks at the execution plan, analyzes, and optimizes.
Database Design:
Selection of field types, design specifications, paradigms, common design cases
1. Table-level design specifications:
1. The character set of the database and the table uniformly use UTF8
2. The naming of the fields should be seen in the name, all tables and fields need to add comments
3. Single table fields should not be too many, separate hot and cold data
Mysql limits each table to store up to 4096 columns, and the size of each row of data cannot exceed 65535 bytes. Reduce disk IO and ensure the memory cache hit rate of hot data (the wider the table, the memory occupied when the table is loaded into the memory buffer pool The larger it is, the more IO will be consumed.) More efficient use of the cache, to avoid reading useless cold data and columns that are often used together in a table (avoid more associated operations)
4. Too much data in a single table should consider sub-table or historical data archiving
It is recommended that the data volume of a single table be controlled within 5 million. If the table structure is too large, there will be big problems in modifying the table structure, backup, and recovery.
You can use historical data to archive (apply to log data), sub-database and sub-table (apply to business data), etc. Means to control the amount of data
2. Field-level design specifications:
1. Preferentially select the smallest data type that meets storage needs
-
Convert character strings to digital storage, such as: Convert IP address to plastic data
-
For non-negative data (such as self-incrementing ID, integer IP), unsigned integer should be used for storage
-
Unsigned can double the storage space compared to signed
SIGNED INT -2147483648~2147483647
UNSIGNED INT 0~4294967295 -
The N in VARCHAR(N) represents the number of characters, not the number of bytes. Use UTF8 to store 255 Chinese characters. Varchar(255)=765 bytes. Too large length will consume more memory
-
2. Use not null as much as possible
The processing of non-null fields is more efficient than the processing of null fields! And there is no need to judge whether it is null.
Null is not easy to handle in MySQL, storage requires extra space, and operations also require special operators. For example, select null = null and select null <> null (<> is an inequality sign) have the same result, and you can only judge whether the field is null through is null and is not null.
How to store it? Each record in MySQL requires additional storage space to indicate whether each field is null. Therefore, special data is usually used for placeholders, such as int not null default 0, string not null default ''.
3. Avoid using ENUM type
Modifying the ENUM value requires the use of the ALTER statement
. The ORDER BY operation of the ENUM type is inefficient and requires additional operations. It is
forbidden to use numerical values as the enumeration value of ENUM.
4. Use TIMESTAMP (4 bytes) or DATETIME type (8 bytes) to store time
TIMESTAMP storage time range: 1970-01-01 00:00:01 ~ 2038-01-19-03:14:07
TIMESTAMP occupies 4 bytes the same as INT, but it is more readable than INT. It
exceeds the value range of TIMESTAMP Use DATETIME type storage
People often use strings to store date-based data (incorrect practice)
Disadvantage 1: Cannot use date functions for calculation and comparison
Disadvantage 2: Using strings to store dates takes up more space
5. Amount data related to finance must use decimal type
- Non-precision floating point: float, double
- Precision floating point: decimal
Decimal type is a precision floating point number, which will not lose precision during calculation. The
occupied space is determined by the defined width. Every 4 bytes can store 9 digits, and the decimal point takes up one byte
to store integers larger than bigint data
3. Index design specification
1. Limit the number of indexes on each table, it is recommended that no more than 5 indexes on a single table
2. It is forbidden to create a separate index for each column in the table
3. Each Innodb table must have a primary key
4. Common index column suggestions
-
Columns that appear in the WHERE clause of SELECT, UPDATE, and DELETE statements
-
For the fields included in ORDER BY, GROUP BY, and DISTINCT, it is not necessary to create an index for all the columns that match the fields in 1 and 2. It is usually better to create a joint index on the fields in 1 and 2
-
Multi-table join…on…associated columns
5. How to choose the order of index columns
The purpose of indexing is to search for data through the index, reduce random IO, and increase query performance. The less data the index can filter out, the less data will be read from the disk.
- The highest discrimination is placed on the leftmost side of the joint index (discrimination = the number of different values in the column/the total number of rows in the column)
- Try to put the column with a small field length on the leftmost side of the joint index (because the smaller the field length, the larger the amount of data that can be stored on a page, the better the IO performance)
- The most frequently used columns are placed on the left side of the joint index (so that you can build fewer indexes)
6. Avoid creating redundant indexes and duplicate indexes
Establishing redundant indexes and duplicate indexes will increase the time it takes for the query optimizer to generate an execution plan.
Examples of duplicate indexes: primary key(id), index(id), unique index(id) Examples of redundant indexes: index(a,b,c), index(a,b), index(a)
7. For frequent queries, give priority to using a covering index
Covering index: the index that contains all query fields (fields included in where, select, ordery by, group by)
Benefits of covering index:
- Avoid the secondary query of the Innodb table index
Innodb is stored in the order of the clustered index. For Innodb, the secondary index stored in the leaf node is the primary key information of the row.
If the data is queried by the secondary index, after finding the corresponding key value , We need to perform a secondary query through the primary key to get the data we really need.
In the covering index, all the data can be obtained from the key value of the secondary index, which avoids the secondary query of the primary key and reduces the IO operation. Improved query efficiency
- Random IO can be turned into sequential IO to speed up query efficiency
Since the coverage index is stored in the order of key values, for IO-intensive range search, it is much less IO than reading each row of data from the disk randomly.
Therefore, the coverage index can also be used to randomize the disk during access. The read IO is converted to the sequential IO of the index search
Optimization of SQL statements:
1. Avoid implicit conversion of data types
All the column names and column types that store the same data must be consistent (usually as associated columns, if the associated column types are inconsistent during query, implicit conversion of data types will be performed automatically, which will cause the index on the column to fail and reduce query efficiency)
//todo to be continued