mysql processing scheme at the time of massive data

1. Partition

2. Cache

3. isolated from a main plurality of read-write little pressure

4. Write the pressure: sub-library sub-table

4.1 points table

Level segmentation:

Vertical database segmentation: According to the closeness of the table function and data association; massive data table caused by too much consideration by business logic is divided; the amount of data and can alleviate problems caused by traffic but can not cure;

Vertical segmentation table: library can be assigned to multiple, non-compliance Paradigm

Level of segmentation table: a single table to solve the problem of too much data, improved stability and load capacity, low end application transformation

 

No matter the kind of segmentation disadvantages are distributed transaction, node join, merge sort paging across nodes, multi-source data management issues

 

5. Middleware

1. Role: parsing sql, separate read and write, read from the load balancing of the library, the library sub-sub-table support operations, support pants collapse association, support the transaction, the master key ID generation, management of multiple data sources

2. Type: client mode (sharding-jdbc TDDL), server proxy mode (mycat cobar atlas heinsberge vitess kingshard)

 

6. fragment advantages

1. reduce the impact of incremental lock when data is written to the query, reduce the query result of a long table lock, lock contention, queuing time expenses

2. The base of smaller single-table queries, io reduce, delay shorter

 

 

7. The actual size of the control

The amount of data traffic and the tightness of the table

Table 7.1 and the close relationship between the amount of data, slow growth, then put together without segmentation level;

7.2 If placed under the table with the rapid growth of the sub-library will need to be divided table

Vertical Table: according to the frequency of use of business functions, the table is divided into primary and secondary

 

 

8. Split Principle

8.1 can not be removed without removal

8.2 segmentation rules planned in advance

8.3 to reduce cross-database may join the data tables or redundant packet

8.4 High Performance middleware database join operations difficult to achieve, so as little as possible join

 

Guess you like

Origin www.cnblogs.com/hzq3554055/p/12000980.html