Dubbo Learning Series Thirteen (Mycat database agent)

Software industry had a cat, I do not say, Tell me what you know for sure what is just, and that is the famous Tomcat, and now again a cat, said to be the sister site Meng East, for the time being recognized as Tom cat's cousin, had called OpencloudDB, later renamed mycat, perhaps more cordial Cat? So, now we come to know it at the kitten.

The core of the database will not say, but the question now is, all kinds of RDB, various NoSQL intertwined, and a distributed, multi-tenant scenarios, and my heart there is no full grasp such a situation can stabilize it. When there is demand, there is market! Naturally, appropriate technology have emerged, Mycat as a DB middleware can be used as a "bridge" between the application and DB, DB make the background transparent to the application of complex composition, processing sub-library sub-table, multi-tenant architecture and large Real-time data query and so a cinch!

tool:

Idea201902/JDK11/ZK3.5.5/Gradle5.4.1/RabbitMQ3.7.13/Lombok0.26/Erlang21.2/RocketMQ4.5.2/Sentinel1.6.3/SpringBoot2.1.6/RHEL7.6/VMware15.0.4/Mysql8.0.17/Mycat1.6.7.3/MysqlWorkbench6.3

Difficulty:  Novice - soldiers - veterans - Master

aims:

Use Mycat connection Mysql clusters (two master-slave) 1.linux, the separate read and write applications

---

step:

1. Establish Mysql cluster, refer to the steps of the article ( Linux cluster use under Mysql ).

2. Download Mycat, put under Linux in / usr / mycat, remember to build this directory.

3. Go to the directory, unzip:

[root@localhost mycat]# tar -zxvf Mycat-server-1.6.7.3-release-20190828135747-linux.tar.gz

4. You can see the directory structure is as follows, two little cats really like:

• bin-Mycat various management commands;
• catlet- extensions;
• conf- configuration information, this is the current focus of use;
• lib-jar package repository, because Mycat is in Java;
• logs- log file;

5.Mycat application integration architecture, that is the goal of this architecture:

If you do need to extend high availability, it can be turned into something like this:

It's that simple!

 

6. In fact, Mycat from the application perspective, is to do the configuration, according to the source preferences study, said to be very complicated!

Mainly three documents core file rule.xml, schema.xml, server.xml configuration:

• server.xml: Mycat configuration file that can be viewed as DBServer Mycat agent,
• schema.xml: physical and logical table DB / mapping configuration fragment sub-library,
• rule.xml: sub-library sub-table rules,

 

7. look at long-sawed one by one, there are instructions on the basic parameters of the meaning of the comment, here are the global configuration parameters:

• <system>Provided Mycat global properties;

• <firewall>Set black and white lists;

• <user>Set the user login account information Mycat;

• <privileges>DML permission table is provided separately;

server.xml original sample:

 1 <?xml version="1.0" encoding="UTF-8"?>
 2 <!DOCTYPE mycat:server SYSTEM "server.dtd">
 3 <mycat:server xmlns:mycat="http://io.mycat/">
 4   <system>
 5     <property name="nonePasswordLogin">0</property> <!-- 0为需要密码登陆、1为不需要密码登陆 ,默认为0，设置为1则需要指定默认账户-->
 6     <property name="useHandshakeV10">1</property>
 7     <property name="useSqlStat">0</property>  <!-- 1为开启实时统计、0为关闭 -->
 8     <property name="useGlobleTableCheck">0</property>  <!-- 1为开启全加班一致性检测、0为关闭 -->
 9     <property name="sqlExecuteTimeout">300</property>  <!-- SQL 执行超时 单位:秒-->
10     <property name="sequnceHandlerType">2</property>
11     <!--<property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>-->
12     <!--必须带有MYCATSEQ_或者 mycatseq_进入序列匹配流程 注意MYCATSEQ_有空格的情况-->
13     <property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>
14     <property name="subqueryRelationshipCheck">false</property> <!-- 子查询中存在关联查询的情况下,检查关联字段中是否有分片字段 .默认 false -->
15       <!--  <property name="useCompression">1</property>--> <!--1为开启mysql压缩协议-->
16         <!--  <property name="fakeMySQLVersion">5.6.20</property>--> <!--设置模拟的MySQL版本号-->
17   <!-- <property name="processorBufferChunk">40960</property> -->
18   <!-- 
19   <property name="processors">1</property> 
20   <property name="processorExecutor">32</property> 
21    -->
22         <!--默认为type 0: DirectByteBufferPool | type 1 ByteBufferArena | type 2 NettyBufferPool -->
23     <property name="processorBufferPoolType">0</property>
24     <!--默认是65535 64K 用于sql解析时最大文本长度 -->
25     <!--<property name="maxStringLiteralLength">65535</property>-->
26     <!--<property name="sequnceHandlerType">0</property>-->
27     <!--<property name="backSocketNoDelay">1</property>-->
28     <!--<property name="frontSocketNoDelay">1</property>-->
29     <!--<property name="processorExecutor">16</property>-->
30     <!--
31       <property name="serverPort">8066</property> <property name="managerPort">9066</property> 
32       <property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property>
33       <property name="dataNodeIdleCheckPeriod">300000</property> 5 * 60 * 1000L; //连接空闲检查
34       <property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
35     <!--分布式事务开关，0为不过滤分布式事务，1为过滤分布式事务（如果分布式事务内只涉及全局表，则不过滤），2为不过滤分布式事务,但是记录分布式事务日志-->
36     <property name="handleDistributedTransactions">0</property>   
37       <!--off heap for merge/order/group/limit      1开启   0关闭-->
38     <property name="useOffHeapForMerge">0</property>
39     <!--单位为m-->
40     <property name="memoryPageSize">64k</property>
41     <!--单位为k-->
42     <property name="spillsFileBufferSize">1k</property>
43     <property name="useStreamOutput">0</property>
44     <!--单位为m-->
45     <property name="systemReserveMemorySize">384m</property>
46     <!--是否采用zookeeper协调切换  -->
47     <property name="useZKSwitch">false</property>
48     <!-- XA Recovery Log日志路径 -->
49     <!--<property name="XARecoveryLogBaseDir">./</property>-->
50     <!-- XA Recovery Log日志名称 -->
51     <!--<property name="XARecoveryLogBaseName">tmlog</property>-->
52     <!--如果为 true的话 严格遵守隔离级别,不会在仅仅只有select语句的时候在事务中切换连接-->
53     <property name="strictTxIsolation">false</property>   
54     <property name="useZKSwitch">true</property>    
55   </system>
56   
57   <!-- 全局SQL防火墙设置 -->
58   <!--白名单可以使用通配符%或着*-->
59   <!--例如<host host="127.0.0.*" user="root"/>-->
60   <!--例如<host host="127.0.*" user="root"/>-->
61   <!--例如<host host="127.*" user="root"/>-->
62   <!--例如<host host="1*7.*" user="root"/>-->
63   <!--这些配置情况下对于127.0.0.1都能以root账户登录-->
64   <!--
65   <firewall>
66      <whitehost>
67         <host host="1*7.0.0.*" user="root"/>
68      </whitehost>
69        <blacklist check="false">
70        </blacklist>
71   </firewall>
72   -->
73   <user name="root" defaultAccount="true">
74     <property name="password">123456</property>
75     <property name="schemas">TESTDB</property>    
76     <!-- 表级 DML 权限设置 -->
77     <!--     
78     <privileges check="false">
79       <schema name="TESTDB" dml="0110" >
80         <table name="tb01" dml="0000"></table>
81         <table name="tb02" dml="1111"></table>
82       </schema>
83     </privileges>    
84      -->
85   </user>
86   <user name="user">
87     <property name="password">user</property>
88     <property name="schemas">TESTDB</property>
89     <property name="readOnly">true</property>
90   </user>
91 </mycat:server>

 

8.schema.xml，配置schema下各个table的分片/分库，以及物理DB：

• <schema>+<table>租户和子表配置，

• <dataNode>分片，

• <dataHost>物理DB，

原版样例：

 1 <?xml version="1.0"?>
 2 <!DOCTYPE mycat:schema SYSTEM "schema.dtd">
 3 <mycat:schema xmlns:mycat="http://io.mycat/">
 4   <schema name="TESTDB" checkSQLschema="true" sqlMaxLimit="100">
 5     <!-- auto sharding by id (long) -->
 6     <table name="travelrecord" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
 7     <!-- global table is auto cloned to all defined data nodes ,so can join
 8       with any table whose sharding node is in the same data node -->
 9     <table name="company" primaryKey="ID" type="global" dataNode="dn1,dn2,dn3" />
10     <table name="goods" primaryKey="ID" type="global" dataNode="dn1,dn2" />
11     <!-- random sharding using mod sharind rule -->
12     <table name="hotnews" primaryKey="ID" autoIncrement="true" dataNode="dn1,dn2,dn3"
13          rule="mod-long" />
14     <!-- <table name="dual" primaryKey="ID" dataNode="dnx,dnoracle2" type="global"
15       needAddLimit="false"/> <table name="worker" primaryKey="ID" dataNode="jdbc_dn1,jdbc_dn2,jdbc_dn3"
16       rule="mod-long" /> -->
17     <table name="employee" primaryKey="ID" dataNode="dn1,dn2"
18          rule="sharding-by-intfile" />
19     <table name="customer" primaryKey="ID" dataNode="dn1,dn2"
20          rule="sharding-by-intfile">
21       <childTable name="orders" primaryKey="ID" joinKey="customer_id"
22             parentKey="id">
23         <childTable name="order_items" joinKey="order_id"
24               parentKey="id" />
25       </childTable>
26       <childTable name="customer_addr" primaryKey="ID" joinKey="customer_id"
27             parentKey="id" />
28     </table>
29     <!-- <table name="oc_call" primaryKey="ID" dataNode="dn1$0-743" rule="latest-month-calldate"
30       /> -->
31   </schema>
32   <!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743"
33     /> -->
34   <dataNode name="dn1" dataHost="localhost1" database="db1" />
35   <dataNode name="dn2" dataHost="localhost1" database="db2" />
36   <dataNode name="dn3" dataHost="localhost1" database="db3" />
37   <!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
38    <dataNode name="jdbc_dn1" dataHost="jdbchost" database="db1" />
39   <dataNode  name="jdbc_dn2" dataHost="jdbchost" database="db2" />
40   <dataNode name="jdbc_dn3"   dataHost="jdbchost" database="db3" /> -->
41   <dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
42         writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
43     <heartbeat>select user()</heartbeat>
44     <!-- can have multi write hosts -->
45     <writeHost host="hostM1" url="localhost:3306" user="root"
46            password="123456">
47       <!-- can have multi read hosts -->
48       <readHost host="hostS2" url="192.168.1.200:3306" user="root" password="xxx" />
49     </writeHost>
50     <writeHost host="hostS1" url="localhost:3316" user="root"
51            password="123456" />
52     <!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
53   </dataHost>
54   <!--
55     <dataHost name="sequoiadb1" maxCon="1000" minCon="1" balance="0" dbType="sequoiadb" dbDriver="jdbc">
56     <heartbeat>     </heartbeat>
57      <writeHost host="hostM1" url="sequoiadb://1426587161.dbaas.sequoialab.net:11920/SAMPLE" user="jifeng"   password="jifeng"></writeHost>
58      </dataHost>
59 ​
60     <dataHost name="oracle1" maxCon="1000" minCon="1" balance="0" writeType="0"   dbType="oracle" dbDriver="jdbc"> <heartbeat>select 1 from dual</heartbeat>
61     <connectionInitSql>alter session set nls_date_format='yyyy-mm-dd hh24:mi:ss'</connectionInitSql>
62     <writeHost host="hostM1" url="jdbc:oracle:thin:@127.0.0.1:1521:nange" user="base"   password="123456" > </writeHost> </dataHost>
63 ​
64     <dataHost name="jdbchost" maxCon="1000"   minCon="1" balance="0" writeType="0" dbType="mongodb" dbDriver="jdbc">
65     <heartbeat>select   user()</heartbeat>
66     <writeHost host="hostM" url="mongodb://192.168.0.99/test" user="admin" password="123456" ></writeHost> </dataHost>
67 ​
68     <dataHost name="sparksql" maxCon="1000" minCon="1" balance="0" dbType="spark" dbDriver="jdbc">
69     <heartbeat> </heartbeat>
70      <writeHost host="hostM1" url="jdbc:hive2://feng01:10000" user="jifeng"   password="jifeng"></writeHost> </dataHost> -->
71 ​
72   <!-- <dataHost name="jdbchost" maxCon="1000" minCon="10" balance="0" dbType="mysql"
73     dbDriver="jdbc"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1"
74     url="jdbc:mysql://localhost:3306" user="root" password="123456"> </writeHost>
75     </dataHost> -->
76 </mycat:schema>

 

9.rule.xml详细描述表的分片规则，格式如下：

1   <tableRule name="分片规则名">
2         <rule>
3               <columns>分片的列</columns>
4               <algorithm>分片算法名</algorithm>
5         </rule>
6   </tableRule>
7   <function name="分片算法名" class="算法实现类">
8         <property name="算法参数">参数值</property>
9   </function>

 

原版样例：

  1 <?xml version="1.0" encoding="UTF-8"?>
  2 <!DOCTYPE mycat:rule SYSTEM "rule.dtd">
  3 <mycat:rule xmlns:mycat="http://io.mycat/">
  4   <tableRule name="rule1">
  5     <rule>
  6       <columns>id</columns>
  7       <algorithm>func1</algorithm>
  8     </rule>
  9   </tableRule>
 10   <tableRule name="rule2">
 11     <rule>
 12       <columns>user_id</columns>
 13       <algorithm>func1</algorithm>
 14     </rule>
 15   </tableRule>
 16   <tableRule name="sharding-by-intfile">
 17     <rule>
 18       <columns>sharding_id</columns>
 19       <algorithm>hash-int</algorithm>
 20     </rule>
 21   </tableRule>
 22   <tableRule name="auto-sharding-long">
 23     <rule>
 24       <columns>id</columns>
 25       <algorithm>rang-long</algorithm>
 26     </rule>
 27   </tableRule>
 28   <tableRule name="mod-long">
 29     <rule>
 30       <columns>id</columns>
 31       <algorithm>mod-long</algorithm>
 32     </rule>
 33   </tableRule>
 34   <tableRule name="sharding-by-murmur">
 35     <rule>
 36       <columns>id</columns>
 37       <algorithm>murmur</algorithm>
 38     </rule>
 39   </tableRule>
 40   <tableRule name="crc32slot">
 41     <rule>
 42       <columns>id</columns>
 43       <algorithm>crc32slot</algorithm>
 44     </rule>
 45   </tableRule>
 46   <tableRule name="sharding-by-month">
 47     <rule>
 48       <columns>create_time</columns>
 49       <algorithm>partbymonth</algorithm>
 50     </rule>
 51   </tableRule>
 52   <tableRule name="latest-month-calldate">
 53     <rule>
 54       <columns>calldate</columns>
 55       <algorithm>latestMonth</algorithm>
 56     </rule>
 57   </tableRule> 
 58   <tableRule name="auto-sharding-rang-mod">
 59     <rule>
 60       <columns>id</columns>
 61       <algorithm>rang-mod</algorithm>
 62     </rule>
 63   </tableRule>  
 64   <tableRule name="jch">
 65     <rule>
 66       <columns>id</columns>
 67       <algorithm>jump-consistent-hash</algorithm>
 68     </rule>
 69   </tableRule>
 70   <function name="murmur"
 71     class="io.mycat.route.function.PartitionByMurmurHash">
 72     <property name="seed">0</property><!-- 默认是0 -->
 73     <property name="count">2</property><!-- 要分片的数据库节点数量，必须指定，否则没法分片 -->
 74     <property name="virtualBucketTimes">160</property><!-- 一个实际的数据库节点被映射为这么多虚拟节点，默认是160倍，也就是虚拟节点数是物理节点数的160倍 -->
 75     <!-- <property name="weightMapFile">weightMapFile</property> 节点的权重，没有指定权重的节点默认是1。以properties文件的格式填写，以从0开始到count-1的整数值也就是节点索引为key，以节点权重值为值。所有权重值必须是正整数，否则以1代替 -->
 76     <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property> 
 77       用于测试时观察各物理节点与虚拟节点的分布情况，如果指定了这个属性，会把虚拟节点的murmur hash值与物理节点的映射按行输出到这个文件，没有默认值，如果不指定，就不会输出任何东西 -->
 78   </function>
 79   <function name="crc32slot"
 80         class="io.mycat.route.function.PartitionByCRC32PreSlot">
 81   </function>
 82   <function name="hash-int"
 83     class="io.mycat.route.function.PartitionByFileMap">
 84     <property name="mapFile">partition-hash-int.txt</property>
 85   </function>
 86   <function name="rang-long"
 87     class="io.mycat.route.function.AutoPartitionByLong">
 88     <property name="mapFile">autopartition-long.txt</property>
 89   </function>
 90   <function name="mod-long" class="io.mycat.route.function.PartitionByMod">
 91     <!-- how many data nodes -->
 92     <property name="count">3</property>
 93   </function>
 94 ​
 95   <function name="func1" class="io.mycat.route.function.PartitionByLong">
 96     <property name="partitionCount">8</property>
 97     <property name="partitionLength">128</property>
 98   </function>
 99   <function name="latestMonth"
100     class="io.mycat.route.function.LatestMonthPartion">
101     <property name="splitOneDay">24</property>
102   </function>
103   <function name="partbymonth"
104     class="io.mycat.route.function.PartitionByMonth">
105     <property name="dateFormat">yyyy-MM-dd</property>
106     <property name="sBeginDate">2015-01-01</property>
107   </function> 
108   <function name="rang-mod" class="io.mycat.route.function.PartitionByRangeMod">
109     <property name="mapFile">partition-range-mod.txt</property>
110   </function>  
111   <function name="jump-consistent-hash" class="io.mycat.route.function.PartitionByJumpConsistentHash">
112     <property name="totalBuckets">3</property>
113   </function>
114 </mycat:rule>

 

部分常用的分片规则算法说明：

• PartitionByMurmurHash(一致性hash)：将物理节点虚拟并映射为一个“一致性hash环”；

• PartitionByCRC32PreSlot(crc32slot 算法)：crc32(key)%102400=slot，slot 按照范围均匀分布在 dataNode 上；

• LatestMonthPartion(单月小时拆分)：单月内按照小时拆分，最小粒度是小时，可以一天最多 24 个分片，最少 1 个分片，一个月完后下月 从头开始循环；

• PartitionByMonth(自然月)：按自然月分片；

• PartitionByRangeMod(范围求模)：先进行范围分片计算出分片组，组内再求模；

• PartitionByJumpConsistentHash(一致性hash)：另一种一致性hash算法；

• PartitionByFileMap(枚举)：通过在配置文件中配置可能的枚举 id，自己配置分片，本规则适用于特定的场景，比如有些业务需要按照省份或区县来做保存，而全国省份区县是固定的；

• PartitionByLong(固定分片 hash 算法)：取 id 的二进制低 10 位取模运算，即( id 二进制) &1111111111，partitionCount分片个数，partitionLength分片长度，默认这两个参数的向量积为1024；

• AutoPartitionByLong(范围约定)：按照提前规划好分片字段范围计算属于哪个分片，start <= range <= end；

• PartitionByMod(求模)：即根据 id 进行十进制求模预算，相比固定分片 hash，此种在批量插入时可能存在批量插入单事务插入多数据分片，增大事务一致性难度；

• PartitionByDate(按天分片)：即根据指定的格式，起止日期，按日期划分，如果配置了 sEndDate 则代表数据达到了这个日期的分片后后循环从开始分片插入；

10.情况一：如果DB是一主一从：需注意这里的主从复制由Mysql实现，Mycat不负责数据复制功能。只需配置server.xml和schema.xml即可： 本次server.xml实例：

 1 <?xml version="1.0" encoding="UTF-8"?>
 2 <!DOCTYPE mycat:server SYSTEM "server.dtd">
 3 <mycat:server xmlns:mycat="http://io.mycat/">
 4   <system>
 5     <property name="nonePasswordLogin">0</property> 
 6     <property name="useHandshakeV10">1</property>
 7     <property name="useSqlStat">0</property> 
 8     <property name="useGlobleTableCheck">0</property>  
 9     <property name="sqlExecuteTimeout">300</property> 
10     <property name="sequnceHandlerType">2</property>
11     <property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>
12     <property name="subqueryRelationshipCheck">false</property> 
13     <property name="processorBufferPoolType">0</property>  
14     <property name="handleDistributedTransactions">0</property>    
15     <property name="useOffHeapForMerge">0</property>
16     <property name="memoryPageSize">64k</property>
17     <property name="spillsFileBufferSize">1k</property>
18     <property name="useStreamOutput">0</property>
19     <property name="systemReserveMemorySize">384m</property>
20     <property name="useZKSwitch">false</property>
21     <property name="strictTxIsolation">false</property>
22     <property name="useZKSwitch">true</property>   
23   </system>  
24   <user name="mycat" defaultAccount="true">
25     <property name="password">12345678</property>
26     <property name="schemas">dubbo_db</property>
27   </user>
28 </mycat:server>

 

schema.xml示例：

 1 <?xml version="1.0"?>
 2 <!DOCTYPE mycat:schema SYSTEM "schema.dtd">
 3 <!-- 数据库配置，与server.xml中的数据库对应 -->
 4 <mycat:schema xmlns:mycat="http://io.mycat/">
 5     <schema name="dubbo_db" checkSQLschema="true" sqlMaxLimit="100">
 6       <table name="dubbo_delivery" primaryKey="ID" dataNode="dn1"/>
 7       <table name="dubbo_finance" primaryKey="ID" dataNode="dn1 "/>
 8       <table name="dubbo_item" primaryKey="ID" dataNode="dn1 " />
 9       <table name="dubbo_order" primaryKey="ID" dataNode="dn1"/>
10       <table name="dubbo_order_detail" primaryKey="ID" dataNode="dn1 "/>
11       <table name="dubbo_stock" primaryKey="ID" dataNode="dn1 " />
12   </schema>
13   <!-- 分片配置 -->
14   <dataNode name="dn1" dataHost="localhost1" database="dubbo_db" />
15   <!-- 物理数据库配置 -->
16   <dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
17         writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
18       <heartbeat>select user()</heartbeat>
19       <writeHost host="hostM1" url="192.168.1.204:3306" user="root" password="abcd@1234">
20           <readHost host="hostS2" url="192.168.1.205:3306" user="root" password="abcd@1234" />
21       </writeHost>
22   </dataHost>
23 </mycat:schema>

 

11.情况二，即本期目标架构，DB是两主一从：server.xml不变, 本次schema.xml实例：

 1 <?xml version="1.0"?>
 2 <!DOCTYPE mycat:schema SYSTEM "schema.dtd">
 3 <!-- 数据库配置，与server.xml中的数据库对应 -->
 4 <mycat:schema xmlns:mycat="http://io.mycat/">
 5     <schema name="dubbo_db" checkSQLschema="true" sqlMaxLimit="100">
 6       <table name="dubbo_delivery" primaryKey="ID" dataNode="dn1"/>
 7       <table name="dubbo_finance" primaryKey="ID" dataNode="dn1,dn2" rule="rule1"/>
 8       <table name="dubbo_item" primaryKey="ID" dataNode="dn1,dn2" rule="rule2"/>
 9       <table name="dubbo_order" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-murmur"/>
10       <table name="dubbo_order_detail" primaryKey="ID" dataNode="dn1,dn2" rule="sharding-by-month"/>
11       <table name="dubbo_stock" primaryKey="ID" dataNode="dn1" />
12     </schema>
13   <!-- 分片配置 -->
14   <dataNode name="dn1" dataHost="localhost1" database="dubbo_db" />
15   <dataNode name="dn2" dataHost="localhost2" database="dubbo_db" />
16   <!-- 物理数据库配置 -->
17   <dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
18         writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
19       <heartbeat>select user()</heartbeat>
20       <writeHost host="hostM1" url="192.168.1.204:3306" user="root" password="abcd@1234">
21           <readHost host="hostS2" url="192.168.1.205:3306" user="root" password="abcd@1234" />
22       </writeHost>
23   </dataHost>
24   <dataHost name="localhost2" maxCon="1000" minCon="10" balance="0"
25         writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
26       <heartbeat>select user()</heartbeat>
27       <writeHost host="hostM2" url="192.168.1.206:3306" user="root" password="abcd@1234" />
28   </dataHost>
29 </mycat:schema>

 

本次rule.xml实例：只有使用了分片模式时，才需要配置rule规则，这里写了三种rule，其实也没全部用上：

 1 <?xml version="1.0" encoding="UTF-8"?>
 2 <!DOCTYPE mycat:rule SYSTEM "rule.dtd">
 3 <mycat:rule xmlns:mycat="http://io.mycat/">
 4 <!--规则定义-->
 5   <tableRule name="sharding-by-murmur">
 6     <rule>
 7       <columns>id</columns>
 8       <algorithm>murmur</algorithm>
 9     </rule>
10   </tableRule>
11     <!--自定义规则-->
12   <tableRule name="rule1">
13     <rule>
14       <columns>userr_id</columns>
15       <algorithm>func1</algorithm>
16     </rule>
17   </tableRule>
18     <tableRule name="rule2">
19     <rule>
20       <columns>id</columns>
21       <algorithm>func2</algorithm>
22     </rule>
23   </tableRule>
24   <tableRule name="sharding-by-month">
25     <rule>
26       <columns>create_time</columns>
27       <algorithm>partbymonth</algorithm>
28     </rule>
29   </tableRule>
30 <!--规则算法实现-->
31   <function name="murmur"
32     class="io.mycat.route.function.PartitionByMurmurHash">
33     <property name="seed">0</property><!-- 默认是0 -->
34     <property name="count">2</property><!-- 要分片的数据库节点数量，必须指定，否则没法分片 -->
35     <property name="virtualBucketTimes">160</property><!-- 一个实际的数据库节点被映射为这么多虚拟节点，默认是160倍，也就是虚拟节点数是物理节点数的160倍 -->
36     <!-- <property name="weightMapFile">weightMapFile</property> 节点的权重，没有指定权重的节点默认是1。以properties文件的格式填写，以从0开始到count-1的整数值也就是节点索引为key，以节点权重值为值。所有权重值必须是正整数，否则以1代替 -->
37     <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property> 
38       用于测试时观察各物理节点与虚拟节点的分布情况，如果指定了这个属性，会把虚拟节点的murmur hash值与物理节点的映射按行输出到这个文件，没有默认值，如果不指定，就不会输出任何东西 -->
39   </function>
40   <function name="func1" class="io.mycat.route.function.PartitionByLong">
41     <!--分片数量，partitionCount*partitionLength=1024-->
42     <property name="partitionCount">2</property>
43     <property name="partitionLength">512</property>
44   </function>
45     <function name="func2" class="io.mycat.route.function.PartitionByLong">
46     <property name="partitionCount">8</property>
47     <property name="partitionLength">128</property>
48   </function>
49   <function name="partbymonth"
50     class="io.mycat.route.function.PartitionByMonth">
51     <property name="dateFormat">yyyy-MM-dd</property>
52     <property name="sBeginDate">2015-01-01</property>
53   </function>
54 </mycat:rule>

 

12.测试：配置好mycat/conf/下的3个xml文件，即配置好了Mycat与物理DB的连接，应用端连接仅需修改连接串端口为Mycat的IP+端口，账号为server.xml中user信息，注意:要写上默认schema，否则启动应用报Mycat no chose错， 

 

13.此处有坑！如果Mysql是独立安装在linux上，需要对远程访问打开，否则访问默认仅限本地，导致远程连接一直报错，以开放root用户远程连接为例：

mysql> use mysql;
mysql> update user set Host='%' where User='root';
mysql> quit;

 

再重启mysql：

[root@localhost ~]# systemctl restart mysqld

 

启动Mycat：

[root@localhost ~]# cd /usr/mycat/mycat/bin
[root@localhost bin]# ./mycat start

 

mycat启动成功：

如果启动有问题，使用以下命令查看log：

[root@localhost conf]# tail -F  /usr/mycat/mycat/logs/wrapper.log
[root@localhost conf]# tail -F  /usr/mycat/mycat/logs/mycat.log

 

然后可以在window上使用如MysqlWorkbench,Navicat测试下是否连接正常，并测试下Mycat连接：  

为了集中测试代码，我只改写了finance模块，写个service方法：com.biao.mall.service.DubboFinanceServiceImpl中：

 1  //插入1000条数据，看data分布
 2 @Override
 3  public void testMycat(){
 4      DubboFinanceEntity financeEntity = new DubboFinanceEntity();
 5        for (int i = 0; i < 1000; i++) {
 6           financeEntity.setUserId(String.valueOf(i+100));
 7  financeDao.insert(financeEntity); 8  } 9 return "testMycat successfully"; 10 }

 

写个controller方法跑一跑：

@RestController
@RequestMapping("/finance")
public class DubboFinanceController {
    private DubboFinanceServiceImpl  financeService;
    @Autowired
    public DubboFinanceController(DubboFinanceServiceImpl  financeService) {
        this.financeService = financeService; } @RequestMapping("/mycat") public void testMycat(){ return financeService.testMycat(); } }

 

启动：ZK---> business -->  finance, URI来一个！

DB情况，请看数量和ID分布，红色数字是IP： 

这里只测试了两主一从和一种分片规则，其他请君自测！

13.代码地址：其中的day16，https://github.com/xiexiaobiao/dubbo-project.git

---

后记：

1.认识Mycat的关键特性：

• 支持Mysql原生协议，跨语言，跨平台，跨数据库的通用中间件代理；
• 基于心跳的自动故障切换，支持读写分离，支持MySQL主从；
• 基于Nio实现，有效管理线程，高并发问题；
• It supports multi-chip automatic routing of data with the polymerization, conventional polymerization support functions sum, count, max, etc., cross-database tab;
• Supported by a global table, fragmented ER relations strategy to achieve efficient single library multi-table join query;
• It supports multi-tenant solution that is multi-schema mode with DB;
• Support global serial number, primary key to solve the problem of distributed generation;
• Fragmentation rule rich, plug-in technology development, easy to expand, can be customized;
• ZooKeeper cluster-based management, online upgrade, expansion, intelligent optimization, large data processing (V2.0dev);
• Introduced Mycat Seamless switching, I think this is the biggest advantage;

2. Mycat recognize the limitations :

• Currently only for cross-database join2 tables, table 3 and above do not support cross-database join;

• Mycat did not submit the agreement to implement the XA transaction according to the second stage, but only to ensure data consistency prepare phase of weak XA transaction, a distributed transaction scenarios, strong consistency can not be guaranteed;

• The tab order scenes, all fragments will again query, and then sort and paging, there is a performance bottleneck;

• DB different types of adaptation in general, such as Oracle / SQLServer, etc., due to the differences in SQL syntax, the statement must do a thorough compatibility testing;

• No API configuration, only the configuration XML way, very out of date;

3.Mycat as a layer on the DB heavyweight middleware, unified entrance, actually undermines the definition of distributed, failed to give full play to the effectiveness of DB layer, so there are a lot of voices are not optimistic, DB used independently, and more can play freely flexible configuration, direct docking application layer is more efficient.

4. Summary: Use Mycat framework, to be cautious, so at least for now.

---

Recommended reading:

• The 12 Dubbo Learning Series (Quartz task scheduling)

• Under Linux clusters using Mysql

• Dubbo Learning Series in Eleven (Dashboard + Nacos rules push)

• Dubbo Learning Series Part Ten (Sentinel of limiting the downgrade)

• Dubbo Learning Series nine (Shiro + JWT Rights Management)

•