Demo Getting Started with Mycat

Preface
Mycat is an open source distributed database system. It is a Server that implements the MySQL protocol. Front-end users can regard it as a database proxy, which can be accessed with MySQL client tools and command lines, and its back-end can use MySQL The native protocol communicates with multiple MySQL servers, and it can also communicate with most mainstream database servers using the JDBC protocol. Its core function is to divide a large table into N small tables horizontally and store them in the backend. In the MySQL server or other databases;
the following will briefly analyze the demo provided by Mycat:

Prepare
1.Jdk1.7.0_80
2.Mysql 5.5
3.Mycat-server-1.6-release http://dl.mycat.io/1.6-RELEASE/
4.Mycat configures the writeHost and readHost of the backend, and configures the local machine. Under conf/schema.xml:

<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
              writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
        <heartbeat>select user()</heartbeat>
        <!-- can have multi write hosts -->
        <writeHost host="hostM1" url="localhost:3306" user="root"
                   password="root">
            <!-- can have multi read hosts -->
            <readHost host="hostS2" url="localhost:3306" user="root" password="root" />
        </writeHost>
</dataHost>

5. Configure the log level to debug, under conf/log4j2.xml:

<asyncRoot level="debug" includeLocation="true">
     <AppenderRef ref="Console" />
     <AppenderRef ref="RollingFile"/>
</asyncRoot>

Start Mycat
1. Start Mycat, run the bin/startup_nowrap.bat executable file
2. Connect to the Mycat server, the default port is 8066

C:\Users\hui.zhao.cfs>mysql -uroot -proot -P8066 -h127.0.0.1

3. Simple view of Mycat server, including database, data table

mysql> show databases;
+----------+
| DATABASE |
+----------+
| TESTDB   |
+----------+
1 row in set (0.00 sec)
 
mysql> use TESTDB;
Database changed
mysql> show tables;
+------------------+
| Tables in TESTDB |
+------------------+
| company          |
| customer         |
| customer_addr    |
| employee         |
| goods            |
| hotnews          |
| orders           |
| order_items      |
| travelrecord     |
+------------------+
9 rows in set (0.01 sec)

The databases and data tables shown above are all configured in conf/schema.xml, and the related ones are conf/server.xml and conf/rule.xml.
schema.xml mainly defines the logic library, logic table and other related information;
server.xml mainly configures some system parameters;
rule.xml mainly defines some rules for sub-database and sub-table.
The following mainly uses the default logic library and logic table configured in schema.xml to do some simple operations to understand Mycat.

Demo shows that
schema.xml defines the background Mysql database db1, db2, db3; so these three databases need to be created in the Mysql database first;

1. Table travelrecord (sharding rules), defined as follows:

<table name="travelrecord" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />

name: defines the table name of the logical table;
dataNode: defines the dataNode to which this logical table belongs, which needs to correspond to the value of the name attribute in the dataNode tag, that is, the corresponding background database;
rule: used to specify the logical table to be used Rule name, the rule name is defined in rule.xml;

1.1 Create tables in three databases respectively

CREATE TABLE `travelrecord` (
  `id` int(11) NOT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8

1.2 Rules defined in rule.xml

<tableRule name="auto-sharding-long">
    <rule>
        <columns>id</columns>
        <algorithm>rang-long</algorithm>
    </rule>
</tableRule>
<function name="rang-long" class="io.mycat.route.function.AutoPartitionByLong">
    <property name="mapFile">autopartition-long.txt</property>
</function>

This sharding rule plans in advance which shard a certain range of the sharding field belongs to, which is specifically defined in the conf/autopartition-long.txt file; and specifies the id as the sharding field;

1.3 Simulate the id range to insert data

insert into travelrecord (id,name) values(1,'hehe');
insert into travelrecord (id,name) values(5000001,'hehe');
insert into travelrecord (id,name) values(10000001,'hehe');

1.4 Query data and observe logs

select * from travelrecord where id=5000001;
select * from travelrecord;

id=5000001 should be routed to the dn2 node, check the log:

route={1 -> dn2{SELECT * FROM travelrecord WHERE id = 5000001 LIMIT 100}

If there is no query condition, it should be routed to three nodes, and check the log:

route={
   1 -> dn1{SELECT * FROM travelrecord LIMIT 100} 
   2 -> dn2{SELECT * FROM travelrecord LIMIT 100}
   3 -> dn3{SELECT * FROM travelrecord LIMIT 100}
}

2. Table company (global table), defined as follows:

<table name="company" primaryKey="ID" type="global" dataNode="dn1,dn2,dn3" />

2.1 Create tables in three databases respectively

CREATE TABLE `company` (
  `ID` int(11) NOT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8

2.2 Logical table type
type defines the type of logical table. Currently, there are only two types of logical table: "global table" and "normal table". If it is a global table, all shards have the same data;

insert into company (ID,name) values(1,'hehe');

View the log as follows:

route={
   1 -> dn1{insert into company (ID,name) values(1,'hehe')}
   2 -> dn2{insert into company (ID,name) values(1,'hehe')}
   3 -> dn3{insert into company (ID,name) values(1,'hehe')}
} rrs

2.3 View the global table

select * from company;

Execute multiple times to view the global table, and check the log to find that one query statement is randomly selected from the three shards each time to execute the query; the following three logs are the results of the three executions:

route={1 -> dn3{SELECT * FROM company LIMIT 100}} rrs
route={1 -> dn2{SELECT * FROM company LIMIT 100}} rrs
route={1 -> dn1{SELECT * FROM company LIMIT 100}} rrs

3. The table hotnews (self-incrementing primary key) is defined as follows:

<table name="hotnews" primaryKey="ID" autoIncrement="true" dataNode="dn1,dn2,dn3" rule="mod-long" />

3.1 Create tables in three databases respectively

CREATE TABLE `hotnews` (
  `id` bigint(20) DEFAULT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8

3.2 Self-increasing primary key
In the case of sub-database and sub-table, the default MySQL self-increasing primary key cannot satisfy the uniqueness of the primary key. Mycat provides the function of global serial number to ensure the uniqueness of the table's primary key;
Mycat provides a variety of global The serial number method includes: local file method, database method, local timestamp method, distributed ZK ID generator, Zk increment method;
the default sequenceHandlerType=2 in server.xml indicates the local timestamp method. The specific usage is as follows:

insert into hotnews (id,name) values(next value for MYCATSEQ_GLOBAL,'hehe');

At the same time, rule=”mod-long” is configured, and the specified sharding rule is modulo. You can view the data results. The following data is inserted under db1:

968418212909813760  hehe

4. Table employee (configure primaryKey), defined as follows:

<table name="employee" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-intfile" />

4.1 Create tables in three databases respectively

CREATE TABLE `employee` (
  `id` int(11) NOT NULL,
  `sharding_id` int(11) NOT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8

4.2
The sharding rule specified by the employee for inserting data is sharding-by-intfile, specifically conf/rule.xml:

<tableRule name="sharding-by-intfile">
    <rule>
        <columns>sharding_id</columns>
        <algorithm>hash-int</algorithm>
    </rule>
</tableRule>
<function name="hash-int" class="io.mycat.route.function.PartitionByFileMap">
    <property name="mapFile">partition-hash-int.txt</property>
</function>

Use sharding_id as the sharding field, and the sharding rule is "sharding enumeration": configure sharding yourself by configuring possible enumeration ids in the configuration file, which is configured here in conf/partition-hash-int.txt, Prepare to insert two pieces of data into the library dn1 and dn2 respectively:

insert into employee (id,sharding_id,name) values(1,10000,'hehe0');
insert into employee (id,sharding_id,name) values(2,10010,'hehe1');

4.3 The primaryKey attribute
indicates that the logical table corresponds to the primary key of the real table. When the sharding rule uses a non-primary key for sharding, when using the primary key to query, the query statement will be sent to all sharding nodes. If this attribute is configured, then Mycat will cache the primary key and specific dataNode information;

select * from employee where id=1;

Execute the above query statement for the first time, and check the log to find that the query statement is sent to both shard nodes:

route={
   1 -> dn1{select * from employee where id=1}
   2 -> dn2{select * from employee where id=1}
} rrs

When this query statement is executed for the second time, check the log and only send the query statement to one node:

route={
   1 -> dn1{select * from employee where id=1}
} rrs

5. The table customer and orders (parent-child table relationship) are defined as follows:

<table name="customer" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-intfile">
    <childTable name="orders" primaryKey="ID" joinKey="customer_id" parentKey="id">
    </childTable>
</table>

5.1 Create tables in three databases respectively

CREATE TABLE `customer` (
  `id` int(11) NOT NULL,
  `sharding_id` int(11) NOT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `orders` (
  `id` int(11) NOT NULL,
  `customer_id` int(11) NOT NULL,
  `name` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8

5.2 Parent-child table relationship
The childTable tag is used to define the child table of the ER fragment, and is associated with the parent table through the attributes on the tag;
joinKey: When inserting the child table, the value of this column will be used to find the data node stored in the parent table;
parentKey: The value specified by the attribute is generally the column name associated with the parent table. The program first obtains the value of joinkey, and then generates a query statement through the column name specified by the parentKey attribute, and executes the statement to obtain which shard the parent table is stored on, thereby determining the storage location of the child table;

5.3 Insertion of simulated data

insert into customer (id,sharding_id,name) values(1,10000,'hehe0');
insert into customer (id,sharding_id,name) values(2,10010,'hehe1');

A piece of data is inserted into dn1 and dn2 respectively through the sharding rules, and then the data is inserted into orders to see if it can be inserted into the associated node;

insert into orders (id,customer_id,name) values(1,1,'order1');

The id of the customer table corresponding to customer_id=1 should be inserted into the dn1 node and check the log:

route={
   1 -> dn1{insert into orders (id,customer_id,name) values(1,1,'order1')}
} rrs

Similarly specify customer_id=2:

insert into orders (id,customer_id,name) values(2,2,'order1');
 
route={
   1 -> dn2{insert into orders (id,customer_id,name) values(2,2,'order2')}
} rrs

Summary
In this article, I mainly learned about the related functions of Mycat from the demo that comes with Mycat and combined with the official documents. Many functions are not in-depth, and it is a simple introduction; I will prepare to have a more comprehensive understanding of Mycat in the future, and at the same time, I can go deeper. to the source code level.

Demo Getting Started with Mycat

Guess you like