table of Contents
Three, high concurrency database optimization
1. Engine comparison
(1) MySQL engine: supports five engine types : ISAM, MYISAM , MEMORY (HEAP), INNODB and BERKLEY .
ISAM (disk table): It is suitable for database design in which the number of queries is much greater than the number of updates. The read operation is fast and does not take up a lot of memory and storage resources. ISAM has shortcomings: it does not support transaction processing, row-level locks, fault tolerance, and data cannot be recovered.
MyISAM (disk table): It still emphasizes fast read operation (the fastest read speed), it is MySQL's ISAM extended format and the default database engine, which not only provides index and field management, but also uses a table locking mechanism , To optimize multiple concurrent read and write operations. MyISAM has shortcomings: it does not support transaction processing, does not support row-level locks, and data cannot be recovered after the table is damaged.
MEMORY (HEAP) (memory table) : Because it is in the memory, its operation speed is very fast, but the managed data is unstable, and it is easy to cause data loss. If the data is deleted, it is easy to cause a waste of space.
InnoDB and BERKLEY (disk tables) : InnoDB is the default engine of MySQL, which creates the flexibility of MySQL and perfectly solves the problems of the first two types of unsupported transactions, unsupported foreign keys, and unsupported row-level locks. Through MySQL+API Implementation engine (when DELETE FROM table, InnoDB will not re-create the table, but delete row by row).
(2) Comparison of engine index structure:
B+tree (B+ tree) and B-tree (B tree), most of us use these two as the index structure.
B-tree: It is a kind of multi-channel search tree, that is, the use of B-tree can reduce the intermediate process experienced when positioning records, thereby speeding up access.
Its characteristics are as follows:
1. The keyword set is distributed among all the nodes of the entire tree;
2. The search may end at a non-leaf node;
3. Any keyword appears and only appears in one node.
The B-tree search starts from the root node, and performs binary search on the keyword sequence in the node. If it hits, it ends, otherwise it enters the son node of the query keyword range, and repeats this operation until the corresponding node The pointer is empty or is already a leaf node.
B+tree is a variant of B-tree. Compared with B-tree, it has better advantages:
1. The IO cost of the disk is lower;
2. The query efficiency is more stable;
3. All leaf nodes form an ordered linked list, which is more convenient to find.
InnoDB and MyISAM use B+tree as the index structure.
(3) Oracle engine: Oracle does not have the concept of an engine. Its data processing can be divided into two categories: OLTP (on-line transaction processing) and OLAP (On-Line Analytical Processing).
OLTP : Emphasizes database memory efficiency, emphasizes the command rate of various indicators of memory, emphasizes bind variables, and emphasizes concurrent operations. It is the main application of the traditional relational database, mainly for basic and daily transaction processing, such as bank transactions.
OLAP : Emphasizes data analysis, emphasizes the SQL execution market, emphasizes disk I/O, emphasizes partitioning, etc. It is the main application of the data warehouse system, supports complex analysis operations, focuses on decision support, and provides intuitive and easy-to-understand query results.
2. Transaction support
The concept of transaction: A transaction is to treat a series of operations on the database as a whole, either all succeed or all fail. Using transactions, we can ensure the integrity of the database, and the transaction is atomic.
(1) The characteristics of the transaction (ACID)
1. Atomicity: The modification operations on the database by the programs involved in the transaction either all succeed or all fail.
2. Consistency: The source and destination of the transaction are balanced before and after execution.
3. Isolation: When concurrent, each transaction is isolated and does not affect each other.
4. Durability: Once the transaction is successfully submitted, the integrity of the data should be guaranteed.
(2). Transaction isolation level
1, read uncommitted uncommitted read (dirty read).
2, read committed The submission has been submitted for reading.
3. Repeatable can be read repeatedly.
4. Serializable can be serialized
(3) MySQL and Oracle support transaction level
Oracle supports two transaction levels: READ COMMITTED: read committed and SERIALIZABLE: serial read, the default isolation level is : read committed (READ COMMITTED)
MySQL supports four isolation levels, the default isolation level : Repeated read (Repeated read)
Three, high concurrency database optimization
(1) Table design optimization: If the database model design is unreasonable, it will not only increase the maintenance difficulty of the client and server, but also affect the performance of the system. Generally, in the early stage of system operation, the amount of data is small, and it is easy to overlook the rationality of model design and system performance. Therefore, at the beginning of the system design, the operating performance under high concurrency should be considered. When designing, we must consider as many aspects as possible. While ensuring the lowest possible redundancy, we must also consider the performance of related queries. When the data redundancy is low, the integrity of the data is easily guaranteed, which increases the data throughput speed and ensures the integrity of the data. However, when multiple tables are associated with queries, the performance will be reduced, and the programming difficulty of the client program will also be increased.
The optimization points are as follows:
1. Reasonably increase effective indexes (such as aggregate indexes, single-column indexes, etc.). Indexes help improve retrieval performance, but too many or improper indexes can also cause system inefficiency. Because every time the user adds an index to the table, the database has to do more work. Too many indexes can even lead to index fragmentation .
2. Flexible selection of reasonable field types (both 8000 bytes for immutable character type char and variable character type varchar, char query is fast, but consumes storage space, varchar query is relatively slow but saves storage space. For example, user name, password If the length changes little, you can choose CHAR, and you can choose VARCHAR for fields with large length changes, such as comments).
3. The field length should be as short as possible under reasonable circumstances.
(Two), SQL optimization
1. When using select query, put the index column in the first column of the selection as much as possible, use select * from table less, and select a few columns if you want to use them, such as SELECT COL1, COL2 FROM table.
2. Use the index correctly when querying (if it is a composite index, you must use the first field in the index as a condition to ensure that the system uses the index, otherwise the index will not be used, and should be used as much as possible The field order is consistent with the index order) to increase query efficiency.
3. Avoid the null value judgment of the field in the where clause.
4. Avoid using the != or <> operator in the where clause.
5. Avoid using or in the where clause to connect conditions, you can use union all to solve it.
6. Use in and not in as little as possible. In will make the system unable to use the index. If it is unavoidable, pay attention to the difference between the reasonable use of in and exists. For example, select * from A where id in(select id from B); in() is suitable for the case where the data in table B is smaller than that in table A, and exists() is suitable for the case where the data in table B is larger than that in table A.
7. Avoid using non-initial search in the indexed character data. This also makes the engine unable to use the index. As follows: SELECT * FROM T1 WHERE NAME LIKE'L%'; like a like query, only this kind of index will be used.
8. Avoid expression operations and function operations on fields in the where clause.
9. Avoid performing functions, arithmetic operations or other expression operations on the left side of the "=" in the where clause.
10. Whether there is a certain record in the check table, don't use count(*) which is very inefficient, you can use EXISTS instead.
11. Don't use UNION if you can use UNION ALL.
12. Try not to use the SELECT INTO statement. It will cause the table to be locked, preventing other users from accessing the table.
13. Use DISTINCT instead of GROUP BY as much as possible. If you must use GROUP BY, you can improve the efficiency of the GROUP BY statement by filtering out unwanted records before the GROUP BY.
14. The query speed can be increased through the view.
15. Avoid frequent creation and deletion of temporary tables to reduce the consumption of system table resources.