Database table partitioning techniques include the following:
- split horizontally
- vertical split
- library table hash
1.1. Horizontal split
What is split horizon? To use a more vivid analogy, when eating in the cafeteria, there is only one window, and the queue for the meal is too long, and they are all lined up in an S shape. At this time, it is easy to cause anxiety and confusion among the people in the queue. A manager stood up, added multiple dining windows, and cut the long line into several teams. To understand it more vividly, you take a "scalpel" and slash a large watch a few times. As a result, the large watch turns into several small watches.
Split horizon puts data into two or more separate tables based on some criteria . That is, the records are divided into points, different records can be saved separately, and each subtable has the same number of columns. A horizontal cut divides a table into multiple tables. Each table contains the same number of columns, but fewer rows of data. For example, a table with a billion rows can be horizontally partitioned into 12 tables, each of which represents a month's worth of data for a specific year. Any query that requires data for a specific month simply refers to the corresponding month's table.
The conditions that are usually used to split tables horizontally are: date and time dimension, region dimension, etc., of course, there are more business dimensions.
Let me give a few examples to explain
- Case 1: The data volume of a company's sales records is too large, we can divide it horizontally by month, and the sales records of each month are separated into a table.
- Case 2: A group has branches in various regions, and the group's order data table is too large. We can cut horizontally by the regions where the branches are located.
- Case 3: After a telecommunications company's bills were cut horizontally by date and city, they found that the amount of data was too large, and then they cut them horizontally by brand and number segment.
Horizontal splitting is usually used in the following situations:
(1) The amount of table data is large. After splitting, the number of data and index pages that need to be read during query can be reduced, and the number of index layers can also be reduced to speed up the query.
(2) The data in the table is inherently independent, for example, the data of each region or the data of different periods are recorded in the table, especially some data are commonly used while others are not.
(3) The data needs to be stored on multiple media.
(4) It is necessary to separate historical data from current data.
The advantages of split horizon:
1: Reduce the number of data and index pages that need to be read when querying, and also reduce the number of layers of the index, speeding up the query speed.
Disadvantages of horizontal splitting:
1: Horizontal splitting will increase the complexity of the application. It usually requires multiple table names when querying, and querying all data requires union operations. In many database applications, this complexity outweighs its advantages, because as long as the index key is not large, when the index is used for a query, the amount of data in the table increases by two or three times, and the number of reads in the query increases. The number of disk counts for an index tier.
1.2, vertical division
What is vertical division? To use an image metaphor, a small company has developed into a large multinational enterprise in just a few years. The previous departmental structure obviously cannot meet the current business development. The CEO crackly divided the company into the finance department, the personnel department, and the production department. , sales department ......, all of a sudden set up a number of departments, each performing its own duties. This is relatively image, right? Is there any wood? Ha ha
, split the table vertically (without destroying the third normal form), put the primary key column and some columns in one table, and then put the primary key column and some other columns in another table. Split the original table into multiple tables with fewer columns. If some columns in a table are commonly used while others are not, vertical splitting can be used .
Advantages of vertical splitting:
1: Vertical splitting can make row data smaller, a block of data can store more data, and the number of I/Os during query will be reduced (the number of blocks read per query will be less) ).
2: The vertical partition table can achieve the purpose of maximizing the use of Cache.
Disadvantages of vertical division:
1: After the table is divided vertically, the primary key (primary key) is redundant, and redundant columns need to be managed
2: It will cause the JOIN operation of table connection (increase CPU overhead), which needs to be avoided from the business
1.3, library table hash
表散列与水平分割相似，但没有水平分割那样的明显分割界限，采用Hash算法把数据分散到各个分表中, 这样IO更加均衡。一般来说，我们会按照业务或者功能模块将数据库进行分离，不同的模块对应不同的数据库或者表，再按照一定的策略对某个页面或者功能进行更小的数据库散列，比如用户表，按照用户ID进行表散列，散列128张表，则应就能够低成本的提升系统的性能并且有很好的扩展性