Partitioning and sub-table
1. Why do we need to partition and sub-table?
When we encounter a table with a large amount of data stored in the database, the pressure on a single table is huge at this time.
This will not only cause slower queries, but also operations such as updates, modifications, and deletions due to the table's locking mechanism will be severely affected (even row-level locks will also have an impact), so there is a database performance bottleneck.
2. Sub-table
Sub-table is broken down into a large table having a plurality of independent entity table storage according to certain rules, each table corresponding to the three documents: MYD 数据文件，.MYI 索引文件，.frm 表结构文件. These tables can be distributed on the same disk or on different machines. When the application accesses the database for reading and writing, the corresponding table name is obtained according to the pre-defined rules, and then it is operated.
Split a single database table into multiple data tables, and then when the user accesses it, according to a certain algorithm (such as using a hash method, you can also use the remainder (modulo) method) to allow users to access different In this way, the data is distributed to multiple data tables, reducing the access pressure of a single data table. The concurrency of a single table is improved, and the disk I/O performance is also improved, which improves the database access performance . This is the purpose of sub-tables, reducing the burden on the database and shortening the query time.
The sub-table is also divided into two sub-table methods, namely vertical split and horizontal split:
3.1 Level score table
Under normal circumstances, we use hash, modulo, etc. to split the table.
For example : When the number of users of the application is very large, there is a 100w user table, we can divide it into 4 independent identical independent sub-tables, each table 25w, and then we pass certain rules (such as hash, modulus, etc.) ) Random access to these three tables, so that the pressure of one table is shared among the three tables, which is similar to the way of clustering.
(Note: The 4 independent sub-tables here are exactly the same as the original table, including the same data structure)
3.2 Vertical sub-table
Vertical splitting is more often a step that should be performed at the beginning of the data table design. Vertical splitting refers to the splitting of data table columns, splitting a table with more columns into multiple tables.
Generally, we split vertically according to the following principles:
- Put the infrequently used fields in a separate table ;
- Split text, blob (binary large object, binary large object) and other large fields into the attached table ;
- Columns that are frequently combined and queried are placed in a table ;
3. Partition
The partition is similar to the sub-table, and the table is decomposed according to the rules. The difference is that the sub-table decomposes the large table into several independent entity tables, while the partition divides the data into multiple locations for storage. After the partition, the table is still one table, but the data is scattered to various scattered locations .
However, partitioning is generally not recommended, because the disadvantages of the partition table are written in the mysql specification: the partition table has strict requirements on the partition key; after the partition table becomes larger, the execution of DDL, SHARDING, single table recovery, etc. becomes More difficult . Therefore, it is forbidden to use the partition table, and it is recommended that the business end manually SHARDING.