Storage engine
concept
database storage engine is the underlying database software organization, database management systems (DBMS) using the data engine to create, query, update, and delete data. Different storage engines provide different storage mechanism, indexing techniques, lock level and other functions, the use of different storage engines, you can also obtain a specific function. Many different database management systems support a variety of different data engine.
Main storage engine: 1. MyIsam, 2. InnoDB, 3. Memory, 4. Archive, 5. Federated.
InnoDB (B + tree)
InnoDB underlying storage structure B + tree, B-tree each node corresponds to a page innodb, page size is fixed, usually set to 16k. In which the non-leaf nodes only key, leaf nodes contain complete data.
Database: Storage Engine + InnoDB + TokuDB + MyIASM + Memory + index + three paradigms such as
application scenarios :
1) frequently updated table, the update request processing for multiple concurrent.
2) support transactions.
3) can recover from a disaster (by bin-log log, etc.).
4) foreign key constraints. Only his support foreign keys.
5) Support automatic increase column attributes auto_increment.
TokuDB (Fractal Tree- band data node)
TokuDB underlying storage Fractal Tree structure, somewhat similar Fractal Tree and B + tree structure, in the Fractal Tree, each child pointer pointing to a child node in addition to the outside, but also with a Message Buffer, this is a Message Buffer FIFO's queue, to update the cache.
For example, one need only insert a node falls Message Buffer will be returned immediately, you do not need to search for the leaf nodes. These updates are cached in the background asynchronous query or merger application to the corresponding node.
Storage engine + InnoDB + TokuDB + MyIASM + Memory + index + three paradigms such as: database
Add TokuDB online index does not affect the read and write operations, very fast write performance, Fractal-tree has an advantage in transaction implementation. He mainly applicable to visit infrequently or historical data archiving.
MyIASM
MyIASM is MySQL default engine, but it does not provide support for database transactions, does not support row-level locking, and foreign keys, so when INSERT (insert) or UPDATE (update) data-write operations need to lock the entire table, then efficiency It will be lower.
ISAM speed read operation is performed quickly, and do not take up a lot of memory and storage resources. Expected early in the design data are organized into fixed-length records, stored sequentially.
--- ISAM is a static index structure, the disadvantage is that it does not support transactions.
Memory
Memory (also known as HEAP) heap memory: the presence of the contents of memory to use to create the table. Each MEMORY table only actually corresponds to a disk file. MEMORY very fast table access type, data as it is in memory, and the default indexes using HASH. But once the service is shut down, the data will be lost off the table. Memory support and hash index B-tree indexes, B-tree index may be used with the partial query and pass the query, may be used <,>, and> = operators like to facilitate data mining, the hash index equal to the range, but relatively fast relatively much slower.
Index
Index (Index) to help MySQL efficiently get the data structure of the data. Common query algorithm, sequential search, binary search, binary sort tree search, hash hashing, block search, multiple search trees balanced B-tree (B-tree)
The principle of common index
1. Select a unique index
Unique index value is unique and can be more quickly determined by the particular record index.
2. often need sorting, grouping and joint field operations indexing:
3. Indexed field often as a query.
4. Limit the number of indexes:
The more the index, will update the table becomes a waste of time.
The possible use of a small amount of data index
If the value of the index is very long, so the speed of queries will be affected.
6. Try to use the prefix to index
If the value of the index field is very long, a prefix is preferably used to index values.
7. Remove unused or little-used index
8. The most left-prefix matching principle, a very important principle.
9 to select a high-sensitive column as an index
10. The discrimination of the formula is a ratio of the field will not be repeated
11. indexed columns can not participate in the calculation to keep columns "clean": queries function does not participate in the index.
12. Try to expansion index, do not create a new index.
Three database paradigm
Paradigm table structure having a minimal redundancy. Paradigm as follows:
Paradigm first (1st NF - are listed can not be divided)
The first objective is to ensure that paradigm atoms of each column: If the minimum data unit in each column are not subdivided (also referred to as the smallest atomic units), the first normal form (1NF)
Database: Storage Engine + InnoDB + TokuDB + MyIASM + Memory + index + three paradigms, etc.
The second paradigm (2nd NF- each table only describe one thing)
First, in first normal form, and the table does not exist the primary key column Africa partially dependent on the primary key. The second paradigm requires that each table describes only one thing.
Database: Storage Engine + InnoDB + TokuDB + MyIASM + Memory + index + three paradigms like
third normal form (3rd NF- absence of transmission dependence on the non-primary key column)
The third paradigm is defined, satisfies a second paradigm, and the column of the table depend on the absence of transmission of non-primary key column. In addition to the primary key order number, customer name depends on the non-primary key customer ID.
Database: Storage Engine + InnoDB + TokuDB + MyIASM + Memory + index + three paradigms such as
database transactions
Transaction (TRANSACTION) are a series of operations as a single logical unit of work performed, these operations as a whole system to be submitted together, are either executed or not executed. A transaction is a logical unit of work indivisible
The transaction must have the following four properties, referred to as the ACID properties:
Atomicity (Atomicity)
- A transaction is a complete operating. Each transaction step operation is indivisible (atoms); or are executed or not executed.
Consistency (Consistency)
-
When the transaction is completed, the data must be in a consistent state. Isolation (Isolation)
- All data can be modified concurrent transactions are isolated from each other, indicating that the transaction must be independent, it should not in any way affect or depend on other matters.
Persistent (Durability)
- After the transaction is complete, it modifies its database is permanently kept, the transaction log to maintain permanent transaction.
Stored Procedure (SQL statement to set a specific function)
In order to complete a set of SQL statements set specific functions, stored in a database, compiled after the first call does not need to re-compile again, the user name specified by the stored procedure parameters are given (if the stored procedure with parameters) to execute it. A stored procedure is an important object in the database.
Storage process optimization ideas:
-
Try to use some sql statement to replace some small cycles, e.g. aggregation function, averaging functions and the like.
-
Intermediate results stored in a temporary table, add an index.
-
Less use cursors. sql language is set, for set operation having a higher performance. The cursors are process operations. For example, a 1 million lines of data to be queried. Cursor need to read the table one million times, instead of using the cursor only need a small amount several times to read.
-
The shorter the transaction as possible. sqlserver support concurrent operations. If the transaction too long, or isolation level is too high, it will result in concurrent operation of blocking, deadlocks. Resulting in slow query, cpu occupancy rate Polar.
-
Use try-catch error exception handling.
- Try not find the statement placed in circulation.
Trigger (for a program to automatically execute)
Automatically triggers a program section executed, is a special storage procedures, triggers, and the difference between the ordinary procedure is stored: the trigger is triggered when one of the operating table. Such as: update, insert, delete these operations, the system will automatically perform the corresponding call trigger on the table. SQL Server 2005 triggers can be divided into two categories: DML triggers and DDL triggers, DDL triggers which they will affect a variety of data definition language statements are excited, these statements have create, alter, drop statement.
Database concurrency strategy
General Concurrency Control Three methods, namely, optimistic and pessimistic locking and a time stamp.
Optimistic locking
Optimistic locking think a user when reading data, no one else to write their own data read; pessimistic lock on the contrary, that he read the database, others may just write their own just read the data, in fact, hold a conservative attitudes; a time stamp that is not locked to control problems occur concurrently through the stamp.
Pessimistic locking
Pessimistic locking is, when reading data, to prevent others to modify the data themselves read, they will read their own data first lock, only you can read the data, and only allow others to modify the portion of the data, or anti came over and said, is to make changes to certain pieces of data when the data do not allow others to read, and so only their entire transaction submitted before the release of their plus lock before allowing other users to access that part of the data.
Timestamp
Prior to the time stamp is in a separate database table plus a time stamp, such as "TimeStamp", each read out of time, also read out to the field, when write back when, to the field plus 1, to submit, with the database the comparison of a field, if greater than the value of the database, then it allows you to save, otherwise not allowed to save, although this approach does not use locking mechanism provided by the database system, but this method can greatly increase the amount of concurrent database processing, and more pessimistic locking said adding "lock" is actually divided into several locks, which are: exclusive lock (write lock) and shared lock (read lock).
Lock database
row-level locking
Row-level locking is an exclusive lock to prevent other transactions from modifying this line; when using the following statement, Oracle will automatically use row-level locking:
-
INSERT、UPDATE、DELETE、SELECT … FOR UPDATE [OF columns] [WAIT n | NOWAIT];
-
SELECT ... FOR UPDATE statement allows the user to lock multiple records to be updated
- Use COMMIT or ROLLBACK statement to release the lock.
Table-level locking
Representation of the entire table to lock the current operation, it is simple, low resource consumption, supported by most of the MySQL engine. MYISAM and INNODB most commonly used support table-level locking. Table table-level locking into a shared read lock (shared lock) and exclusive write locks table
(Exclusive lock).
Page-level locking
MySQL page-level locking is locked and a particle size between row-level locks intermediate a table lock latch. Table-level locking is fast, but more conflict, less row-level conflict, but slower. So take a page-level compromise, once locked a group of adjacent records. BDB supports page-level locking
To be continued ............
Finishing is not easy, need the full pdf document junior partner please poke:
https://shimo.im/docs/TC9Jq63Tp6HvTXdg/