Data slice: a dimension according to the stored data in a single database to store a plurality of libraries or dispersing tables, performance bottlenecks and improve usability

Effective means: relational database sub-library sub-table: Avoid the amount of data exceeds the threshold value can withstand generate a query bottlenecks

Sub-libraries: an effective dispersing amount of access to the database of a single point

Points table: as far as possible into the local Distributed Transaction transaction may

Vertical Split: Split by business, vertical split: special purposes; not really solve the bottleneck of a single point, can alleviate problems caused by the amount of data traffic to and

Horizontal resolution: horizontal split, pass 1 / * field data according to certain rules or dispersed * library table: It is a primary key into the even slice 0 library / libraries in Table 1 into odd / table

Theoretically break through the bottleneck single amount of data processed, expand relatively free, sub-sub-table in the library solution

SQL concepts

Binding table: fragmentation rules consistent master and child tables: t_order and t_order_item are by order_id fragmentation, the link between them is not Cartesian product inquiry

SELECT i.* FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id WHERE o.order_id in (10, 11);
绑定后，路由的sql应该是：
SELECT i.* FROM t_order_0 o JOIN t_order_item_0 i ON o.order_id=i.order_id WHERE o.order_id in (10, 11);

SELECT i.* FROM t_order_1 o JOIN t_order_item_1 i ON o.order_id=i.order_id WHERE o.order_id in (10, 11);

t_order在FROM的最左侧，将以它为整个绑定表的主表。 所有路由计算只用主表的策略，t_order_item表的分片计算使用t_order的条件，绑定表之间的分区键要完全相同。

Broadcast table: all of fragmented data sources exist table structure and data exactly the same amount need not query massive data management table: the dictionary table

Logic Index: sub-table scene with library allowed the same index name, the index name different tables need to be rewritten index + table name, index name to be rewritten before the logical index

Fragmentation

Shard key: the level of demolition of key fields, without the full implementation of the route: poor performance, support for multi-field slice

Slicing algorithm:

Common denominator piece Strategy refining a variety of scenarios to provide a higher level of abstraction and provides an interface

Accurate slice

PreciseShardingAlgorithm, as a process using a single key = fragmentation scene with IN slice bond? Use must comply with the StandardShardingStrategy

Range fragmentation

RangeShardingAlgorithm, as a single key a key BETWEEN AND fragmented fragmented, with the need to use StandardShardingStrategy

Composite fragment

ComplexKeysShardingAlgorithm, multi-key key fragments as fragment, with the use ComplexShardingStrategy

hint fragment

HintShardingAlgorithm, using slicing Hint line, with the use HintShardingStrategy

In the middle of this large segment in my earlier point in time to save all gone, I keep the page open what, csdn but also a token of Mody

This is in part: core routing engine as well as analysis of the rewrite engine

The following pages can not restore what to write directly to the execution engine

Execution Engine: automation and efficiency to balance resources

Connection Mode

Query maintain a separate database connections for each slice, more efficient use of multi-threading

Maintain a separate database connection for each slice, to avoid premature query results are loaded into memory

Open connection a separate thread for each library, the I / O consumption parallel processing

Streaming merge:

Cursor result set down to merge a result, the result is not a fully loaded memory;

When the query can not guarantee that each tile has a separate database connections required in connection multiplexing the library to remove a score sheet before the query results, the full load current query results to memory

Memory Limit mode

Provided: shardingsphere not limit the number of database connections consumed in one operation

Used for OLAP, through relaxation of restrictions on system throughput database connected to lift

Connection limit mode

Provided: shardingsphere strict control of the number of database connections time-consuming operation: the opposite

Preventing fragmentation of a single database connection request to occupy too much problematic, for use in OLTP, OLTP usually with shard key route to

Automated Execution Engine

The selection of connection mode to refine the particle size of each SQL operation, real-time calculations according to the route result and weighed, independent use of appropriate connection mode

Preparation phase: data ready for execution

The key concept of the connection mode of implementation, execution engine according maxConnectionSizePerQuery , with the current routing results, select an appropriate connection mode

Results sql routing packets by a data source name
Through the following formula, the result set is eligible sql routing database instance within each maxConnectionSizePerQuery allowable range, each of the connection to be performed, this request computing optimal connection mode

Avoid deadlock: acquiring sync database connection, creates an execution unit, a one-time acquisition of all atoms database requests needed in this connection sql

Avoid locking the disposable simply acquiring a database connection, once the connection request without waiting for each of the two scenes, using the most OLTP shard key unique to the data routing node, such that the system becomes completely locked state, to enhance the concurrent efficiency, separate read and write times are visible
Resource lock mode when the connection limit, all query results will be released after the library installed memory connected to the memory limit mode only for resources

The implementation phase

Perform sql, and performs packet merge result set

Performing packet: the stage of generating an execution unit delivers the packet to the bottom of the concurrent execution engine, for sending an event during execution of each critical step

The execution engine is concerned only send events, do not care about the subscriber events

Connection mode acquired by performing the preparation phase, to generate a memory flow merge merge result set or result sets, and passes it to a merge engine results

Merge engine

Results merge: the respective plurality of data acquisition nodes result set of data, and are combined into a correct result set is returned to the requesting client

classification

Functionally divided traversal / sorting / grouping / tab / polymerization: Non-exclusive combination

Structure : flow merge, merge memory, merging Decorator: Flow and memory mutex, decoration can be further processed in the streaming memory and merged

Merge : the library one by one to return a result set, all without the load-time data memory, the result follows the libraries return a result set merging merge manner: Save memory consumption

Flow : each time data is acquired from the result set, one by one can obtain a single correct way to return data to fit the native database returns a result set manner

Traverse, sorting, streaming packet

Memory : traversing the result set all the data stored in memory, and then by uniform grouping, sorting, and then packaged as polymerization result set of data by one access

Decorator : all merge unified result set enhancements: paging, aggregation

Traversal merge: the simplest

After the plurality of result sets and is singly linked list, the linked list in the current data been traversed, the list element will shift one, continue traversing

Sort merge

Presence sql order by, each of the ordered result set itself, only need to sort the set of data points to the current cursor value: sorting a plurality of ordered arrays

When sorting merge query, the result set for each data value comparison current (java comparable) and in the priority queue, each time to obtain the next data: The result set cursor down the top of the queue, according to re-enter the new cursor prioritizing queue find their place: every next get only a single correct data

Packet merge

Streaming groups merge: SQL sorting items to be consistent with the entry field grouping and sorting type (ASC or DESC), whether only guarantee the accuracy of its data through memory to merge.

Same data packet it disposable items plurality of data taken full result set.
It is needed for the polymerization is calculated according to the type of aggregate function.

Memory packet merge: the set of all the result data packet loaded into memory and the polymerization is carried out

Sorting statement deletion, shardingsphere sql by rewriting automatically increase is consistent with the sort item entry packet, a streaming memory module,