1. Fragmentation of Mycat -- Node -- Host
Sharding node (dataNode)
After the data is divided, a large table is divided into different sharded databases, and the database where each table shard is located is the shard node (dataNode).
Node host (dataHost)
After the data is split, each shard node (dataNode) does not necessarily have a single machine, and there can be multiple shard databases on the same machine.
The machine where one or more shard nodes (dataNode) are located is the node host (dataHost), in order to avoid the concurrency limit of a single node host
Try to balance the shard nodes (dataNode) with high read and write pressure on different node hosts (dataHost) .
Sharding rules (rule)
We talked about data segmentation earlier. If a large table is divided into several sharded tables, certain rules are required, so that the data can be divided into different parts according to certain business rules.
The rule of a shard is the shard rule. It is very important to select the appropriate shard rule for data segmentation, which will greatly avoid the difficulty of subsequent data processing.
Global sequence number (sequence)
After the data is split, the primary key constraints in the original relational database will not be available under distributed conditions, so an external mechanism needs to be introduced to ensure data
Unique identification, this mechanism to ensure the unique identification of global data is the global sequence number (sequence).
2. Multi-tenancy of Mycat
Multi-tenancy technology, or multi-tenancy technology, is a software architecture technology that explores and implements how to share the same in a multi-user environment.
system or program components and still ensure data isolation between users. In the era of cloud computing, multi-tenant technology provides the same or even customizable services for most clients with a single system architecture and services in a shared data center, and can still ensure customer data isolation. At present, various cloud computing services belong to this category of technology, such as Alibaba Cloud Database Service (RDS), Alibaba Cloud Server and so on.
There are three main solutions for multi-tenancy in data storage, which are:
independent database
This is the first solution, that is, one database per tenant . This solution has the highest level of user data isolation and the best security, but it is also costly.
advantage:
Provide independent databases for different tenants, which helps to simplify the extension design of the data model and meet the unique needs of different tenants;
In the event of a failure, restoring data is simpler.
shortcoming:
The number of installations of the database is increased, and the maintenance cost and the acquisition cost are also increased.
This solution is similar to the traditional one customer, one set of data, and one set of deployment, the only difference is that the software is uniformly deployed at the operator. if faced with
Tenants that require a very high level of data isolation, such as banks and hospitals, can choose this model to increase rental pricing. If the price is low and the product takes the low-price route, this solution is generally unaffordable for operators.
Shared database, isolated data schema
This is the second scheme, that is, multiple or all tenants share the Database, but each tenant has one Schema .
advantage:
Provides a certain degree of logical data isolation for tenants with high security requirements, not complete isolation; each database can support more
number of households.
shortcoming:
If there is a failure, data recovery is more difficult, because the recovery of the database will involve data of other tenants;
If you need cross-tenant statistics, there are certain difficulties.
Shared database, shared data schema
This is the third solution, that is, the tenants share the same Database and the same Schema, but distinguish the number of tenants by the TenantID in the table.
according to. This is the mode with the highest degree of sharing and the lowest level of isolation.
advantage:
Comparing the three schemes, the third scheme has the lowest maintenance and acquisition costs and allows the maximum number of tenants supported by each database.
shortcoming:
The isolation level is the lowest and the security is the lowest, and it is necessary to increase the amount of security development during design and development;
Data backup and recovery is the most difficult, and it needs to be backed up and restored table by table and item by item.
If you want to serve the most tenants with the fewest servers, and the tenants accept the cost of sacrificing isolation levels, this is the best solution.
Suit.
3. Mycat multi-tenant solution:
In the case of configuring multiple schemas through annotations, specify which configured schema to go.
Mycat multi-tenancy implementation principle
1) Modify the web part:
a. When the user logs in, record the tenant's id in the thread variable (ThreadLocal)
b. Modify the implementation of jdbc: when submitting sql, get the tenant id from ThreadLocal, add sql comments, and put the tenant's schema
put in a note. For example: /*!mycat : schema = test_01 */ sql ;
2) DB layer proxy forwarding:
A proxy layer is established in front of the db to proxy all database requests from the web. The proxy layer is implemented with mycat, and the web is submitted
When the sql comes, specify the schema in the comments, and the proxy layer forwards the sql request according to the specified schema.
/*!mycat : schema = test_01 */ sql ;
Fourth, Mycat annotation introduction
MyCat provides a solution for Sql statements that are not supported by itself - add an extra paragraph before the SQL statement to be executed by the annotation
The code that SQL organizes so that Sql can execute correctly, this code is called "annotation". The use of annotations is equivalent to a layer of transparent proxy forwarding of SQL statements that are not supported by mycat, and directly handed over to the target data node for SQL statement execution. The form of the annotation is:
/*!mycat: sql=Annotate Sql statement*/
Annotations are used in the following way:
/*!mycat: sql=Annotate Sql statement*/Real execute Sql
When using, replace the "annotated Sql statement" after the = sign with the required Sql statement, and the specific usage will be mentioned later.
Annotation Principle
The process of MyCat executing SQL statements is to first perform SQL parsing processing, parse out the shard information (routing information), and then execute it in the physical database corresponding to the shard; if the incoming SQL statement cannot be parsed by MyCat, MyCat will not The annotation tells MyCat to perform parsing processing according to the SQL in the annotation (called the annotation SQL). After parsing the fragmentation information, it sends the SQL statement to be executed after the annotation (called the original SQL). Go to the physical library corresponding to the shard to execute.
Annotation usage example
The '!' supported by the annotation is not compatible with the mysql single library,
Annotation supported '#' is not compatible with mybatis
Added mycat character prefix flag Hintsql:"/** mycat: */"
Since 1.6, three annotation methods are supported:
/*#mycat:db_type=master*/ select * from travelrecord
/*!mycat:db_type=slave*/ select * from travelrecord
/**mycat:db_type=master*/ select * from travelrecord
1. The Mycat side executes the storage creation table or stored procedure as follows:
Stored procedure:
/*!mycat: sql=select 1 from test */ CREATE PROCEDURE `test_proc`() BEGIN END ;
surface:
/*!mycat: sql=select 1 from test */create table test2(id int);
Note that the statement in the annotation is the table of the node, please replace it with your own table such as select 1 from table, which shard is the data found by the statement in the annotation, and the number of
It depends on which node to build at that node.
2. Custom sharding with special statement:
/*!mycat: sql=select 1 from test */insert into t_user(id,name) select id,name from t_user2;
3. Read-write separation
After the Mycat read-write separation is configured, the default query will obtain data from the read node, but some scenarios need to obtain real-time data.
The data obtained by the point may not be realized in real time due to the delay. Mycat supports the annotation /*balance*/ to force the data query from the write node:
a. The SQL in the transaction, the default write node, starts with the annotation /*balance*/, will be based on the dataHost tag attribute of schema.xml
balance="1" or "2" to get nodes
b. For non-transactional SQL, the read- write separation is enabled by default to obtain according to balance=“1” or “2”, and the beginning of the annotation /*balance*/ will write the section.
The point solution part has enabled read-write separation, but the scene that requires strong consistent data acquisition in real time walks the write node
/*balance*/ select a.* from customer a where a.company_id=1;
4. Multi-table ShareJoin
/*!mycat:catlet=demo.catlets.ShareJoin */ select a.*,b.id, b.name as tit from customer a,company b on
a.company_id=b.id;
5. Read-write separation data source selection
/*!mycat:db_type=master*/ select * from travelrecord
/*!mycat:db_type=slave*/ select * from travelrecord
/*#mycat:db_type=master*/ select * from travelrecord
/*#mycat:db_type=slave*/ select * from travelrecord
6. Multi-tenant use
Remarks: Not all annotations are supported by Mycat. Mycat only supports specific annotations. You need to see different versions and refer to the relevant documents to build an experimental environment for verification.