Analysis & Answer
Sub-library? sub-table? Or is it divided into databases and tables?
If it is necessary to divide the tables, how many tables should be divided?
Since all technologies serve the business, let's first review the business background from the data aspect.
If 80,000 transaction orders are generated every day, each transaction will generate about 8 operation logs on average, and the system design service life is 5 years, then the data volume of transaction orders is approximately = 5 years, 365 days/year, 8w/day = 146 million, then it is estimated The output tables are as follows:
- The transaction order needs: 146 million/500w = 29.2 tables, we will divide it by 32 tables;
- The operation log needs: 32 * 10 = 320 tables, and we split the data according to 32 * 16 = 512 tables for binary splitting.
If sub-library is required, how many sub-libraries are appropriate?
In addition to the usual business peak reading and writing QPS, when sub-database, you also need to consider the peak that may be reached during the Double 11 promotion period, which needs to be estimated in advance.
According to our actual business scenario, the data query source of the problem list mainly comes from the Xiaomi homepage of Ali customer service. Therefore, it can be evaluated based on historical QPS, RT and other data. Suppose we only need 3500 database connections. If a single database can bear up to 1000 database connections, then we can split it into 4 databases.
Reflect & Expand
How to split the data?
split horizontally
This is a way of horizontally segmenting by business dimension, such as the common segmentation by member dimension, where different member-related data are scattered in different database tables according to certain rules. Since our business scenario decides to read and write data from the perspective of members, we choose to split the database horizontally.
vertical split
Vertical splitting can be simply understood as splitting different fields of a table into different tables.
For example: Suppose there is a small e-commerce business, and the product information, buyer information, and payment information related to an order are all placed in a large table. You can consider splitting the product information, buyer information, seller information, and payment information into separate tables through vertical segmentation, and associate the order number with the basic information of the order.
There is also a situation, if a table has 10 fields, and only 3 fields need to be modified frequently, then you can consider splitting these 3 fields into sub-tables. Avoid affecting the query row locking of the remaining 7 fields when modifying these 3 data.
Meow Interview Assistant: One-stop solution to interview questions, you can search the WeChat applet [Meow Interview Assistant] or follow [Meow Brush Questions] -> Interview Assistant free questions. If you have good interview knowledge or skills, look forward to your sharing!