Author: Zen and the Art of Computer Programming
1 Introduction
For web applications, the amount of data tends to become larger and larger, and more and more data needs to be stored. Therefore, how to reasonably segment data, optimize query performance, and improve system stability is particularly important.
The current mainstream data segmentation methods include the following:
- Horizontal sharding: Split data into different databases or tables based on business logic or time dimensions. For example, the database is divided according to user ID, and the data of different users is stored in different databases.
- Vertical segmentation: Save data of the same business into different databases. For example, order-related data and logistics information are stored in a separate database.
- Mixed slicing: A combination of horizontal and vertical slicing. For example, the membership data of a website can be divided into multiple databases. Each database only stores the data of ordinary users of the website, while the data of VIP users is stored in another database.
- Multi-level index: When querying, retrieve data through multiple index fields at the same time. Query efficiency can be effectively improved through multi-level indexes.
However, each of these solutions has its own advantages and disadvantages. This article will discuss the advantages and implementation solutions of sub-database and sub-table based on practical cases.
2. Explanation of basic concepts and terms
(1) Data Partitioning
Data Partitioning is to divide a large set of data into multiple smaller subsets according to certain rules. Data sharding is generally divided into two types - vertical sharding and horizontal sharding. Among them, vertical segmentation refers to segmenting data according to the characteristics of the data (such as identity, age, etc.); horizontal segmentation refers to segmenting data with the same attributes according to the distribution of data on physical storage. For example: on an e-commerce website