【mycat】mycat level score table

mycat completes horizontal split

Introduction

Compared with vertical splitting, horizontal splitting does not classify tables, but disperses them into multiple databases according to certain rules of a certain field, and each table contains a part of the data. To put it simply, we can understand the horizontal segmentation of data as segmentation according to data rows, that is, segmenting some rows in the table into one database, and segmenting other rows into other databases. , as shown in the figure:

Select the table to split

There is a bottleneck in the number of data stored in a single MySQL table. The bottleneck will be reached when a single table reaches 1 million pieces of data, which will affect the query efficiency and requires horizontal splitting (table splitting) for optimization. For example: the orders and orders_detail in the example have reached 6 million rows of data, and need to be optimized by sub-tables.

Table segmentation fields, taking the orders table as an example, can be divided into tables based on different fields.

Modify the schema.xml file, set the data nodes for the orders table as dn1, dn2, and specify the sharding rule as mod_rule (customized name):

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">

	<schema name="TESTDB" checkSQLschema="true" sqlMaxLimit="100" dataNode="dn1">
		  <table name="tbl_consumer" dataNode="dn2"></table>
		  <!-- rule:水平分表得规则名称该名称可以随便定义。-->
		  <table name="tbl_orders" dataNode="dn1,dn2" rule="mod_rule"></table>

	</schema>
	
	<dataNode name="dn1" dataHost="host1" database="orders" />
	
	<dataNode name="dn2" dataHost="host2" database="consumer" />
	
	<dataHost name="host1" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM1" url="192.168.179.131:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
	
	<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM2" url="192.168.179.132:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
</mycat:schema>

Main points to note:

Configure our rules in rule.xml

	<tableRule name="mod_rule">
		<rule>
			<columns>customer_id</columns>
			<algorithm>mod-long</algorithm>
		</rule>
	</tableRule>
	
	<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
		<!-- 数据库的个数 -->
		<property name="count">2</property>
	</function>

Make sure there are corresponding databases in both machines (in this case: tbl_orders)

INSERT INTO tbl_orders values(1,101,100,100100);

报错： partition table, insert must provide ColumnList

When adding data to a sharded table, you need to provide a list of columns . If you do not specify it, you cannot determine which value is the field of the sharded table.

We can add relevant data to the table through mycat

INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES (1,101,100,100100);
INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES(2,101,100,100300);
INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES(3,101,101,120000);
INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES(4,101,101,103000);
INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES(5,102,101,100400);
INSERT INTO tbl_orders(id,order_type,customer_id,amount) VALUES(6,102,100,100020);

Discover:

144 is 192.168.179.131, 145 is 192.168.179.132. The picture has not changed.

Mycat's sharding "join"

join: joint table query

The Orders order table has been sharded. How to perform join query on the orders_detail order details table associated with it? We need to shard the orders_detail as well. The principle of Join is as follows:

Use the ER table to solve the problem of the above word table association query, which willThe storage location of the sub-table depends on the main table, and are stored physically close to each other, thus completely solving the efficiency and performance problems of JION. Based on this idea, a data fragmentation strategy based on the ER relationship is proposed. The records of the child table and the records of the associated parent table are stored inThe same data fragmentsuperior.

Modify the schema.xml file

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">

	<schema name="TESTDB" checkSQLschema="true" sqlMaxLimit="100" dataNode="dn1">
		  <table name="tbl_consumer" dataNode="dn2"></table>
		  <!-- rule:水平分表得规则名称该名称可以随便定义。-->
		  <table name="tbl_orders" dataNode="dn1,dn2" rule="mod_rule">
			  <!-- childTale:定义子表得标签
			          name: 关联得子表名称
					  primaryKey:子表中得主键列名
					  joinKey: 子表中外键得列名
					  parentKey: 关联得父表得主键名称
			  -->
			   <childTable name="tbl_orders_detail" primaryKey="id" joinKey="order_id" parentKey="id"/>
		  </table>
	</schema>
	
	<dataNode name="dn1" dataHost="host1" database="orders" />
	
	<dataNode name="dn2" dataHost="host2" database="orders" />
	
	<dataHost name="host1" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM1" url="192.168.179.131:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
	
	<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM2" url="192.168.179.132:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
</mycat:schema>

Main points to note:

Make sure you have the tbl_orders_detail table on both machines

CREATE TABLE tbl_orders_detail(
 id INT AUTO_INCREMENT,
 detail VARCHAR(2000),
 order_id INT,
 PRIMARY KEY(id)
);

Restart mycat

Add data to the tbl_orders_detail table through mycat

INSERT INTO tbl_orders_detail(id,detail,order_id) values(7,'detail1',1);
INSERT INTO tbl_orders_detail(id,detail,order_id) VALUES(8,'detail1',2);
INSERT INTO tbl_orders_detail(id,detail,order_id) VALUES(9,'detail1',3);
INSERT INTO tbl_orders_detail(id,detail,order_id) VALUES(10,'detail1',4);
INSERT INTO tbl_orders_detail(id,detail,order_id) VALUES(11,'detail1',5);
INSERT INTO tbl_orders_detail(id,detail,order_id) VALUES(12,'detail1',6);

test

global table

Order data field table.—Status of stored orders---->Payment Unpaid Cancel Pending Shipment Shipped Confirmed. . . .

Because the order data dictionary table is required on each node. So we define the data dictionary table as a global table.

When tables are needed, global tables should be made.

• Changes are infrequent
• The overall amount of data has not changed much
• The data size is small, rarely exceeding hundreds of thousands of records

In view of this, Mycat defines a special table called a "global table". The global table has the following characteristics:

• The insertion and update operations of the global table will be executed on all nodes in real time to maintain the data consistency of each shard.
• The query operation of the global table is only obtained from one node.
• The global table can perform JOIN operations with any table. Defining dictionary tables or some tables that meet the characteristics of dictionary tables as global tables solves the problem of data JOIN from another aspect. Through global tables + sharding strategy based on ER relationships, Mycat can satisfy more than 80% of enterprise application development needs

Modify schema.xml

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">

	<schema name="TESTDB" checkSQLschema="true" sqlMaxLimit="100" dataNode="dn1">
		  <table name="tbl_consumer" dataNode="dn2"></table>
		  <!-- rule:水平分表得规则名称该名称可以随便定义。-->
		  <table name="tbl_orders" dataNode="dn1,dn2" rule="mod_rule">
		  			  <!-- childTale:定义子表得标签
			          name: 关联得子表名称
					  primaryKey:子表中得主键列名
					  joinKey: 子表中外键得列名
					  parentKey: 关联得父表得主键名称
			  -->
			   <childTable name="tbl_orders_detail" primaryKey="id" joinKey="order_id" parentKey="id"/>
			</table>
			<!-- type:global表示全局表得意思-->
		  <table name="tbl_dict_order_type" dataNode="dn1,dn2" type="global" ></table>
	</schema>
	
	<dataNode name="dn1" dataHost="host1" database="orders" />
	
	<dataNode name="dn2" dataHost="host2" database="consumer" />
	
	<dataHost name="host1" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM1" url="192.168.179.131:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
	
	<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
			  writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="hostM2" url="192.168.179.132:3306" user="root"
				   password="123456">
		</writeHost>
	</dataHost>
</mycat:schema>

Main points to note:

Restart mycat

Make sure there are tbl_dict_order_type databases on both server machines. Note: You need to create the table through mycat.

CREATE TABLE tbl_dict_order_type(
 id INT AUTO_INCREMENT,
 order_type VARCHAR(200),
 PRIMARY KEY(id)
);

Add data through mycat

INSERT INTO tbl_dict_order_type(id,order_type) VALUES(101,'type1');
INSERT INTO tbl_dict_order_type(id,order_type) VALUES(102,'type2');

There is data in both databases, global tables