A pit on the table design
1, field design
1.1 Field Type Design
Try to use integer representation string:
`INET_ATON(str)`，address to number
`INET_NTOA (number)`, number or address
1.2 fixed-length and non-selected fixed length data type
1, decimal no loss of accuracy, storage will increase with increasing data. double the space occupied by the fixed, larger numbers of memory loses precision.
2, fixed-length `char`, non-fixed length` varchar, text` (limit 65535, which also consume 1-3 `varchar` byte record length, and the additional space` text` record length) field length Do not set too long, take up space.
1.3 Use as much as possible not null
1, than the processing of non-null field null fields more efficient! And is not required to judge whether or not null.
2, null in MySQL, good handling, the need for additional storage space, but also requires a special arithmetic operator. The select null = null and select null <> null (<> unequal number) has the same result, can only be judged by is null and is not null field is null.
1.4 single table field not too much, can refer to the use of the reserved field, or look for inefficiencies (sub-library sub-table).
2, paradigm required (to reduce data redundancy)
- The first paradigm: Field atomicity
- The second paradigm: the elimination of part of the primary key dependency
- Third Pattern: Elimination of transmission dependence on the primary key
Second, the selection of used storage engine (engine = ***)
| MyISAM | InnoDB | |
| Index Type | Non-clustered index | Clustered index |
| Support Services | no | Yes |
| Support table lock | Yes | Yes |
| Support row lock | no | Yes |
| Support foreign keys | no | Yes |
| Support for full-text indexing full text | Yes | It is (after 5.6 support) |
| Suitable for the type of operation | A large number of select | 大量insert、delete、update |
Third, the indexing mechanism
1, the index Introduction
MySQL index is to help efficiently get the data structure of the data.
Index is stored in the file system.
The index files are stored in the form storage engine related.
2, the index Category:
Mysql index of five types: primary key index, the only index, the general index and full-text index, a composite index. By adding to the index fields can increase the data read speed, concurrency and under pressure to improve the project.
Primary key index
Primary key is a unique index, but it must be specified as PRIMARY KEY, each table can have only one primary key. If no primary key, then selects a unique key, if there is no unique key, then generates a 6-bit row_id to as the primary key.
The only index
All values are indexed columns once, that must be unique, value can only appear empty.
General index
The basic types of index values may be empty, without limitation uniqueness.
Full-text index
Full-text index of the index type FULLTEXT. Full-text index can be created on varchar, char, text type columns.
Composite index
Multi-column values make up an index, specifically for combinatorial search. The most left index principle.
3, the index data structure
the hash, binary trees, red-black tree, B tree, B + trees (B + tree Why use Refer https://www.cnblogs.com/jing99/p/11760993.html )
4, the index of scenarios
- where possible query the index column to the conditional expression.
- order by the time of the query will use the index column.
- join on conditions column to the index column.
- Index covering: columns query if all established through an index, the index will be queried directly, without query all data.
- Appears to be independent field: select * from user where id = 20-1; select * from user where id + 1 = 20;
- When fuzzy query, try not to match the expression begins with statements like wildcards (like '% abc'), but as far as possible ( 'abc%), full scan will trigger the beginning of a wildcard.
- Do not create an index in the use of state value.
Fourth, the cache settings
1, open the cache in the configuration file
In the configuration query_cache_size [mysqld] in the group:
0: Do not open
1: ON, default caches all, the need to increase select sql-no-cache prompt SQL statement to give up Cache
2: Open the default cache is not necessary to increase the select sql-cache in the SQL statement to proactively cache (common)
2, set at the client cache size
By using the configuration set global query_cache_size = 64x1024x1024
Fifth, data partition
MySQL high performance standalone cases, but when a large amount of data and is distributed when accessed, MySQL performance began to decline, the data needs to be dispersed into a plurality of groups storage file, in order to ensure the efficiency of a single file.
Partitioning algorithm:
1, hash (field) applied to integer field
2, key (field) applied to the string type
3, range partitioning algorithm according to the data size range
Create Table article_range ( ID int AUTO_INCREMENT, title VARCHAR ( 64 ), Content text , CREATED_TIME int , - Published 1970-1-1 number of milliseconds to a PRIMARY KEY (ID, CREATED_TIME) - requires the partitioning of the primary key field must be part ) charset = UTF8 the PARTITION BY the RANGE (CREATED_TIME) ( the PARTITION p201808 the VALUES less Within last ( 1535731199 ), - SELECT UNIX_TIMESTAMP ( '2018-8-31 23:59:59') the PARTITION p201809VALUES less than (1538323199), -- 2018-9-30 23:59:59 PARTITION p201810 VALUES less than (1541001599) -- 2018-10-31 23:59:59 );
4, list algorithm
Create Table article_list ( ID int AUTO_INCREMENT, title VARCHAR ( 64 ), Content text , Status TINYINT ( . 1 ), - Base Status: Draft 0-, 1- completed but not published, 2- published a PRIMARY KEY (ID, Status) - requires the partitioning field must be part of the primary key ) charset = UTF8 the pARTITION BY List (Status) ( the pARTITION Writing values in ( 0 , . 1 ), - unpublished on a partition Published the PARTITION values in ( 2 ) - published in a partition ); INSERT INTO article_list values ( null , ' MySQL optimization ' , ' content exemplary ' , 0 ); the flush Tables;
Six clusters
1, from the master copy
mysql master replication process is asynchronous from
- master open bin-log function, log files for reading and writing the database records deletions.
- We need to open three threads, master IO thread, slave IO thread open, SQL thread.
- Slave IO thread connection master, and requests a bin-log, after the contents position.
- MASTER server receives the slave IO thread sent logging request information, io thread to the bin-log content, position returned to the slave IO thread.
- slave server receives a bin-log log content that will write the contents of bin-log log relay-log relay log, create a master.info the file that records the master ip username password master bin-log name, bin- log position.
- slave SQL thread open end, real-time monitoring whether relay-log log content is updated, the file parsing SQL statements, to execute the slave database.
2, separate read and write
Is dependent on the separate read and write from the master copy, and the copying master is to separate read and write services. MySQL to read and write separation refers to the process to make master write operation, so that slave processing read operation, ideal for larger scene than read operations, reduce stress master's.
3, load balancing
polling
WRR
Load distribution
4, high availability
When server architecture, in order to ensure that no server downtime 7x24 online, you need a single point for each server (server served by a server, such as writing server, database middleware) to provide redundancy machine.
For the write server, the need to provide a similar write - redundant servers, when writing server health (write - through redundancy heartbeat), write - redundancy as a copy write content server machine from its role to do a synchronous; when the server goes down to write, write - top to redundant servers will continue to serve as the write server. To the outside world this process is transparent, that is, the outside world only access the service through a IP.
Seven, SQL statement optimization
1, database import statement
1.1, when importing disable indexes and constraints:
alter table table-name disable keys
After the completion of the data to be imported, then turn indexes and constraints, to create a one-time index
alter table table-name enable keys
1.2 database engine if you are using innodb, it defaults will each write command plus transaction (which will consume certain amount of time), it is recommended to manually open a transaction, and then perform the bulk import a certain amount of the last manual submission affairs.
1.3, if the same batch of SQL command to import data in different formats only, then you should pre-compiled prepare it, so can save a lot of time recompiling.
2、limit offsets,row
Try to ensure that no large offset, such as limit 10000,10 equivalent to the first 10,000 lines have been discarded on the number of lines of inquiry and then take out 10 lines, plus a number of conditions can filter it (complete screening), and should not limit the use of skips to query data.
3, select * to use less
That is, you need to select a field `select`, but the impact is not great, because the network transmission more tens of hundreds of bytes not much delay,` select and the now popular ORM frameworks are used * ` but we pay attention to the separation of large amounts of data fields in the design table.
4, single-table and multi-table queries
Multi-table query: join, subqueries are involved in the multi-table queries. If you use the explain analyze the execution plan you will find multi-table query is a table of a processing table, the final result of the merger. Therefore we can say single-table queries to calculate the pressure placed on applications, and multi-table queries to calculate the pressure placed on the database.
Predicate pushdown: the WHERE clause predicates into outer query block included in the lower layer query block (e.g., view), thereby enabling early data filtering and possible to better use the index in the art.
-- 有表如下： -- t_student(student_id, student_name, student_age) -- t_score(score_id, student_id, course_id, score_number) -- 优化前的SQL SELECT * FROM t_student,t_score WHERE t_score.student_id=t_student.student_id AND t_score.score_id=2; -- 优化后的SQL SELECT * FROM t_student t1 right JOIN ( SELECT * from t_score WHERE score_id=2 ) t2 ON t1.student_id=t2.student_id;
5、count(*)
In the MyISAM storage engine, it will automatically record the number of rows in the table, so using count (*) to quickly return. And without such a innodb internal counter, we need to manually count the number of records. Thus using the count (1) or the count (column).
6, turn on slow query log
Configuration Item: slow_query_log
Can show variables like 'slov_query_log' to see if open, if the state is OFF, may be used to set GLOBAL slow_query_log = on open, it generates a file xxx-slow.log at datadir. After facilitates slow SQL query execution optimization.
Eight, MySQL configuration
1, max_connections The maximum number of client connections
2, table_open_cache table file cache
3, key_buffer_size index cache size
4, innodb_buffer_pool_size innodb storage engine cache pool size
5, innodb_file_per_table each table of data in a single file rather than a shared space table