1. Environment construction

Mycat is the use of open-source database middleware java language development , support for Windows and Linux operating environment, here MyCat in the Linux environment to build .

A. MySQL
B. JDK
C. MyCat

1. Install and configure MySQL

1) Install MySQL

A). 卸载 centos 中预安装的 mysql 

	rpm -qa | grep -i mysql 
	
	rpm -e mysql-libs-5.1.71-1.el6.x86_64 --nodeps 

B). 上传 mysql 的安装包 

	alt + p -------> put E:/test/MySQL-5.6.22-1.el6.i686.rpm-bundle.tar 

C). 解压 mysql 的安装包 

	mkdir mysql 
	
	tar -xvf MySQL-5.6.22-1.el6.i686.rpm-bundle.tar -C /root/mysql 

D). 安装依赖包 

	yum -y install libaio.so.1 libgcc_s.so.1 libstdc++.so.6 libncurses.so.5 -- setopt=protected_multilib=false

	yum update libstdc++-4.4.7-4.el6.x86_64 

E). 安装 mysql-client 

	rpm -ivh MySQL-client-5.6.22-1.el6.i686.rpm 

F). 安装 mysql-server 

	rpm -ivh MySQL-server-5.6.22-1.el6.i686.rpm

2) Start and stop MySQL

service mysql start 

service mysql stop 

service mysql status 

service mysql restart

3) Log in to MySQL

mysql 安装完成之后, 会自动生成一个随机的密码, 并且保存在一个密码文件中 : 
	/root/.mysql_secret 
	
	mysql -u root -p 
	
	登录之后, 修改密码 : 
	
	set password = password('itcast'); 
	
	授权远程访问 : 

	grant all privileges on *.* to 'root' @'%' identified by 'itcast'; 
	flush privileges;

After the authorized remote access, you can sqlYogconnect Linuxon MySQL, but remember to turn off the firewall on a Linux (or configure a firewall):

2. Install JDK1.8

A. 上传JDK的安装包到Linux的root目录下 

	alt + p -----------> put D:/jdk-8u181-linux-x64.tar.gz 

B. 解压压缩包 , 到 /usr/share 目录下 
	tar -zxvf jdk-8u181-linux-x64.tar.gz -C /usr/share/ 

C. 配置PATH环境变量 , 在该配置文件(/etc/profile)的最后加入如下配置 
	export JAVA_HOME=/usr/share/jdk1.8.0_181 
	export PATH=$PATH:$JAVA_HOME/bin

3. Install MyCat

Upload the compressed package of MyCat alt + p --------> put D:/Mycat-server-1.6.7.3- release-20190927161129-linux.tar.gz
Unzip the compressed package of MyCat tar -zxvf Mycat-server-1.6.7.3-release-20190927161129- linux.tar.gz -C /usr/local
Introduction to MyCat directory structure

Reference materials: 《深入理解JVM ＆ G1 GC》
Students who want to get information quickly: Please add assistant VX: C18173184271,备注一下CSDN+工作年限！Get it for free
so that you can fully understand and learn!

Two, the core concept of MyCat

1. Sharding

In simple terms, refers to certain conditions, we will be stored in a database with data stored in a plurality of dispersed database (host) above, to achieve the effect of dispersing the load a single device. Data segmentation ( Sharding) according to the type of segmentation rules can be divided into two split mode.

One is Schemato segment to different databases (hosts) according to different tables (or ). This segmentation can be called vertical (vertical) segmentation of data.
The other is to split the data in the same table into multiple databases (hosts) according to certain conditions according to the logical relationship of the data in the table. This segmentation is called horizontal (horizontal) data cutting. Minute.

MyCat fragmentation strategy:

Above the dotted line is the logical structure diagram, and below the dotted line is the physical structure diagram;

2. Logic library (schema)

MyCat is a database middleware . Usually, for practical applications, you do not need to know the existence of middleware. Business developers only need to know the concept of a database, so database middleware can be regarded as one or more database clusters. Logic library .

3. Logical table (table)

Since there is a logic library, there will be logic tables. In distributed databases, for applications, the tables that read and write data are logical tables. Logical tables can be distributed in one or more sharding libraries after data segmentation, or they can be composed of only one table without data segmentation or sharding.

Fragmented tables
refer to the original large data tables that need to be divided into multiple database tables . In this way, each fragment has a part of data, and all fragments constitute complete data. All in all, it is the table that needs to be fragmented. For example, the tb_ordertable is a fragmented table, and the data is divided into two nodes dn1 and dn2 according to the rules.
Non-fragmented tables
Not all tables in a database are very large. Some tables can be partitioned. Non-fragmented tables are relative to fragmented tables, that is, tables that do not require data partitioning. Such as: tb_citya non-fragmented table , data is only stored on one node dn1 therein.
ER table
relational database is based on the entity-relationship model (Entity Relationship Model) , and MyCatthe ER table will come from this. MyCat presents data partitioning strategy based on the relationship between ER, recording the parent table of its associated sub-word table stored in a single data slice, by group tables (Table Group) to ensure that data is not associated with cross-database queries.
Global table
in a large project, there will be a part of the dictionary table (code table) , in which stored data is the basis for some of the projects, which are the basis of the data, the amount of data is not large, probably in all business table There are connections. When the business table is fragmented due to the large amount of data, the associated query between the business table and the attached data dictionary table becomes a more difficult problem. In MyCat, the associated query of this type of table can be solved through data redundancy. , That is, all shards copy this piece of data ( data dictionary table ), so these redundant data tables can be defined as global tables.

4. Shard node (dataNode)

After the data segmentation, a large table is divided into different pieces of the above databases, database tables for each slice is located slice node (Datanode) .

5. Node host (dataHost)

After the data segmentation, each (Datanode) fragment node is not necessarily a machine are exclusive, the same machine may have a plurality of slices above databases, one or more such fragments nodes (Datanode) machine is located the node host ( DataHost) , in order to avoid limiting the number of concurrent single-master node, try to write the high pressure slice node (Datanode) equalization in a different host node (DataHost) .

6. Fragmentation rules (rule)

We talked about data segmentation. If a large table is divided into several fragment tables, certain rules are required. In this way, the rules for dividing data into a certain fragment according to a certain business rule are the fragmentation rules, and the data fragmentation is selected appropriately The sharding rules are very important and will greatly avoid the difficulty of subsequent data processing.

Three, shard configuration test

1. Demand

Since TB_TESTa large amount of data in the table, now need TB_TESTa table of data pieces, is divided into three data nodes, each node located on a different server host, a specific configuration, with reference to the following figure:

2. Environmental preparation

Prepare three virtual machines, install them MySQL, and configure them:

IP 地址列表 : 
	192.168.192.157 
	192.168.192.158 
	192.168.192.159

3. Placement schema.xml

schema.xmlAs one of the important configuration files in MyCat, it manages the logic library , logic table and corresponding fragmentation rules , DataNode and DataSource of MyCat . Understanding these configurations is the prerequisite for the correct use of MyCat. Here the file is parsed layer by layer.

Create 3 databases on the server, named db1

Modified schema.xmlas follows:

<?xml version="1.0"?> 
<!DOCTYPE mycat:schema SYSTEM "schema.dtd"> 
<mycat:schema xmlns:mycat="http://io.mycat/"> 
	<!-- 逻辑库配置 --> 
	<schema name="ITCAST" checkSQLschema="false" sqlMaxLimit="100"> 
		<!-- 逻辑表配置 --> 
		<table name="TB_TEST" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" /> 
	</schema> 
	
	<!-- 数据节点配置 --> 
	<dataNode name="dn1" dataHost="host1" database="db1" /> 
	<dataNode name="dn2" dataHost="host2" database="db1" /> 
	<dataNode name="dn3" dataHost="host3" database="db1" /> 
	
	<!-- 节点主机配置 --> 
	<dataHost name="host1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> 
		<heartbeat>select user()</heartbeat> 
		<writeHost host="hostM1" url="192.168.192.157:3306" user="root" password="itcast"></writeHost>
	</dataHost> 
	
	<dataHost name="host2" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> 
		<heartbeat>select user()</heartbeat> 
		<writeHost host="hostM1" url="192.168.192.158:3306" user="root" password="itcast"></writeHost> 
	</dataHost> 
	
	<dataHost name="host3" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100"> 
		<heartbeat>select user()</heartbeat> 
		<writeHost host="hostM1" url="192.168.192.159:3306" user="root" password="itcast"></writeHost> 
	</dataHost> 

</mycat:schema>

4. Configure server.xml

server.xmlAlmost all the system configuration information required by mycat is saved. The most commonly used is to configure the user name, password and permissions here. Add the UTF-8 character set setting in the system, otherwise a question mark will appear when storing Chinese

<property name="charset">utf8</property>

Modify user settings, we are here to ITCASTset up two users:

<user name="root"> 
	<property name="password">123456</property> 
	<property name="schemas">ITCAST</property> 
</user> 

<user name="test"> 
	<property name="password">123456</property> 
	<property name="schemas">ITCAST</property> 
</user>

And need to replace the original logic library configuration with the ITCAST logic library ;

5. Start MyCat

start up:

bin/mycat start 
bin/mycat stop 
bin/mycat status

View MyCat:

Connection port number 8066

Via command line

mysql -h 127.0.0.1 -P 8066 -u root -p

Connect via sqlyog

6. MyCat fragmentation test

Enter mycat, execute the following statement to create a table

CREATE TABLE TB_TEST ( 
	id BIGINT(20) NOT NULL, 
	title VARCHAR(100) NOT NULL , 
	PRIMARY KEY (id) 
) ENGINE=INNODB DEFAULT CHARSET=utf8 ;

Let's check the 3 databases of MySQL and find that the tables are created automatically. so amazing.

Next is inserted into the table data, note that writing INSERTmust be written to the field list when written statement, otherwise, the following error message appears:

Error code: 1064

partition table, insert must provide ColumnList

We try to insert some data:

INSERT INTO TB_TEST(ID,TITLE) VALUES(1,'goods1'); 
INSERT INTO TB_TEST(ID,TITLE) VALUES(2,'goods2'); 
INSERT INTO TB_TEST(ID,TITLE) VALUES(3,'goods3');

We will find that the data is written to the first node. When will the data be written to the second node?

We can insert the second node by inserting the following data

INSERT INTO TB_TEST(ID,TITLE) VALUES(5000001,'goods5000001');

Because the sharding rule we adopted is to store 5 million pieces of data per node, when the ID is greater than 5000000, it will be stored on the second node.

Currently only two nodes are set up. What happens if the data is greater than 10 million? Execute the following statement to test

INSERT INTO TB_TEST(ID,TITLE) VALUES(10000001,'goods10000001');

Fourth, the principle of MyCat is introduced

MyCat most important principle is a verb , "interception" , which intercepts the SQL statements sent from the user, the first SQL statement to do some specific analysis, such as fragmentation analysis , route analysis , read and write separation and analysis , buffer analysis and so on, Then send this SQL statement to the real database on the back end, and process the returned result appropriately, and finally return it to the user, as shown in the figure.

In the figure, the user table is divided into three sharding nodes dn1, dn2, and dn3. They are distributed on three MySQL Server(dataHost). Therefore, 1-N servers can be used for sharding. The sharding rule is typical enumeration string fragmentation rules, a rule is defined fragment field + fragmentation function . Here field slice status, slice function was string enumerating .

When MyCat receives a SQL statement, it first parses the table involved in the SQL statement, and then looks at the definition of this table. If the table has a fragmentation rule, it will get the value of the fragment field in the SQL statement and match the fragmentation function to get The shard list corresponding to the SQL statement is then sent to the corresponding shard for execution, and finally the data returned by all shards are processed and returned to the client. Take " select * from user where status='0'" as an example, search status='0', according to the sharding function, the value of '0' is stored in dn1, so the SQL statement is sent to the first node for execution, and then the result of the query is returned to the user.

If the sent SQL statement is "select * from user where status in ('0','1')", then the SQL statement will be sent to the host corresponding to dn1 and dn2 for execution, and then the results will be collected and output to the user.

If you need this full version 《开源数据库中间件MyCat实战笔记》, you only need to support me in this article.

A lot of support, you can get information for free-after three consecutive years (promise: 100% free)

Quick start channel: add assistant VX: C18173184271，备注一下CSDN+工作年限！get it for free! Full of sincerity! ! !

Database middleware MyCat actual combat notes (first step): Introduction to MyCat (environment construction, core concepts of MyCat, shard configuration test, introduction to the principle of MyCat)

table of Contents