And sub-sub-library database tables slicing scheme
Database query is inefficient when data reaches millions of levels, sub-library sub-table is a very effective solution.
Vertical division and the horizontal division
Vertical division: Vertical vertical division is divided into sub-table and vertical two kinds of libraries, sub-libraries vertical low degree of association is to a variety of different databases in the table, the vertical column for the sub-table is a table of the field split to form a new table to another table, so that the recording sheet becomes small, the index will reduce the space consumption, improve performance. Vertical division manageable business logic clear, high concurrency improve performance, but can not connect table queries to distributed transaction technology, and a large amount of data from the table can not be reduced essentially, but also by horizontal division.
Horizontal division: sub-table is divided into sub-library and library sub-table, a shared library or sub-table due to the cpu and IO is not obvious, the record table in a logical distributed to a plurality of the plurality of database tables, horizontal division completely large table to solve the problem, but still there is a connection query is not convenient, the need to introduce a distributed transaction and other shortcomings.
Data fragmentation rules are divided into two main levels: first sub-library according to different time, such as every day's data to generate a database, data separation, or cold, depending on the frequency sub-library data queries, followed by a field in accordance with calculated hash value to divide, but vulnerable to complex inquiries, if the query is not involved in this field will lead directly to the database could not be located.
Sub-library sub-table problems caused by
1, a distributed transaction: cross-database transaction operations, distributed transaction requires the coordination of multiple nodes to extend the execution time of the transaction, concurrent access conflicts and increase the probability of deadlocks, do not ask for transactional consistency of the system can not be demanding Real-time consistency, and the pursuit of eventual consistency can be, that is, transactions compensation: checking the data, logs and contrast.
(Distributed Transaction reason may be because the generated sub-library sub-table, it may be because the application of SOA)
2, join query problem
Solutions include: using the global table (each module are likely to rely on each table in a database are stored), field redundancy (an anti-paradigm design, added to the table based on the information inquiry service is to connect join query) , assembling data (the join query split into two), the associated table together, frequent connection query may be established corresponding to the mapping table
Complex queries may also be used in gene approach:
If you use a name check id but does not know which id in the library, allowing you to follow a name tag at design time, you can take it directly to the library location where id according to the mark or marks calculated hash value
3, across nodes paging, sorting, function
Have to perform again at their table, then the results are summarized again perform together again
4, avoid heavy primary key global issues
Can UUID (32 generates a hexadecimal number), a special table with the primary key to generate (set to increment, in each case taking this value), the establishment of a plurality of ID generating server (first generation 1,3 / 5 , .. ... second 2/4/6), can also use the cache (an ID by generating a time table, and then expanding a plurality of ID values in memory access wait, then access is completed continue generation), according to a design based on actual business string (millisecond precision bits, each of 1000 independent redesign millisecond time number generation)
5, the data expansion and migration
If the primary key is a sub-library of range, then expansion is very simple, if a modulo hash slice, then use the hash consistency
Slicing scheme
Logical database fragment may be encapsulated in the persistence layer of the application side, may be added to a layer between the application and the proxy data, maintenance logic slice in the middleware.