Foreword
While there is often a lure I let my "entertainment to death" of the video, there is a non-stop "spur" I let my big BOSS quickly learn. It had two 极端的爱before I often understand their own efforts at self-confidence. Ah, "the world is not worth it"!
Introduction SCHEMA.XML
Previous wrote: DBLE learning database middleware (a) a basic introduction and quickly build, introduces quick installation. During the installation, we configured the schema.xml file. schema.xml is a more important document. The document provides 逻辑库, 逻辑表, 分片规则, 数据分片, 物理数据库and other configurations. It mainly has the following tags.
schemaTag, mostly逻辑库, and逻辑表a configuration in which逻辑表there are many types, such as: the global table, the table fragment, the fragment table of the ER, and other non-fragmented table.dataNodeTag defines the main data pieces stored in the node, a label dataNode often corresponds to a data slice.dataHostTag, the main definition separate read and write configuration and physical database from the master.
SCHEMA.XML actual configuration
Next we look at actual combat, combat Take the classic PRODUCT table, ORDERS Orders table to do the experiment. We've already introduced 逻辑表the split can have a variety of types. First we plan, assuming that we are a new electricity supplier companies to sell their own company in the electric business platform for the production of goods, our goods on a total of 5-6, but the love of our customers can not stop the production of goods , sections are explosion models ah, once the sale orders on amazing. So how do we split the database table it?
- PRODUCTS product table, it sold several explosive materials, we want to make it
全局表. That is, the data has on each slice of this product PRODUCTS table. This will not have much consumption, after all, this small amount of table data. Also more convenient and fragmented association table. - ORDERS Orders table, there are tens of millions of orders. The large amount of data, we want to make the ORDERS table
分片表. There is one to many relationship PRODUCTS product table and the ORDERS table. We may use modulo fragment according order_id. So no data overheating problems, and产品表a small table on each fragment has, we can be very convenient to use the product table and the Order table associate.
So much, it is better to Photo everyone to look at.
As illustrated, directly connected to the database application, middleware, operating 逻辑库the testdb, 逻辑表PRODUCTS and ORDERS. And said 逻辑库and 逻辑表a corresponding database stored on a physical host A and the host B. MySQL sheet dn1 watched on host A, the host MySQL watched sheet dn2 B. and stores the global table PRODUCTS order_id by modulo the ORDERS table 1 dn1. and stores the global table PRODUCTS order_id by modulo 2 the ORDERS table dn2.
Sort out the whole context, it is our configuration environment.
| server | IP addresses | description |
|---|---|---|
| DBLE server | 192.168.56.185 | DBLE instance, middleware, database, SQL is responsible for receiving routing distribution |
| MySQL A server | 192.168.56.181 | Examples of physical A, there db_1 database |
| MySQL B1 main server | 192.168.56.182 | Examples of the physical B, there db_2 database |
| MySQL B2 from server | 192.168.56.183 | Example B from a library of physical |
MySQL configuration of the physical environment
A main mounting MySQL server and MySQL server B from the environment. Then on both sets of physical databases are created Buddy users.
create user 'buddy'@'%' identified by '123456';
GRANT ALL PRIVILEGES ON *.* TO 'buddy'@'%' IDENTIFIED BY '123456';Configuration schema tag
Dble enter the conf folder schema.xml configuration file. First, let's configure the schema tag. Main configuration as follows:
<schema name="testdb">
<!--er tables-->
<table name="orders" primaryKey="order_id" dataNode="dn1,dn2" rule="rule_mod"/>
<!--global tables-->
<table name="products" primaryKey="product_id" type="global" dataNode="dn1,dn2"/>
</schema>sehama label
The definition of logical libraries, name specifies the
逻辑库name.table label
对表的定义，name指定了表的名字，dataNode指定这张表涉及的数据节点，这里两张表都指定的是dn1,dn2。rule指定了分片的规则，这里是取模算法。如果是全局表需要在type属性列声明为global。
配置dataNode标签
<dataNode name="dn1" dataHost="dataHost1" database="db_1"/>
<dataNode name="dn2" dataHost="dataHost2" database="db_2"/>dataNode标签
name指定的是数据分片节点名称，dataHost指定对应的数据库实例，database对在mysql物理实例中的schema。
配置dataHost标签
<dataHost name="dataHost1" maxCon="1000" minCon="10" balance="0" switchType="-1" slaveThreshold="100">
<heartbeat>show slave status</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="192.168.56.181:3306" user="buddy" password="buddy">
</writeHost>
</dataHost>
<dataHost name="dataHost2" maxCon="1000" minCon="10" balance="0" switchType="-1" slaveThreshold="100">
<heartbeat>show slave status</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM2" url="192.168.56.182:3306" user="buddy" password="buddy">
<readHost host="hostS2" url="192.168.56.183:3306" user="buddy" password="buddy"/>
</writeHost>
</dataHost>dataHost标签
指定了数据库实例，这里name代表数据库实例的名称。maxCon定于了最大连接数，minCon定义了空闲时保有的最小连接数。balance是指读取操作的负载均衡模式，为0则不做均衡。switchType代表了写操作的高可用切换类型。等于-1则表示不自动切换。slaveThreshold指定了指定主从延迟的阀值，为100。该配置主要与读数据时的负载均衡有关，它会先取show slave status中的Seconds_Behind_Master值，如果该值大于slaveThreshold，则读取的时候就会过滤掉这个slave。防止读到旧的数据，影响前台的业务。
heartbeat标签
指定心跳检测，这里心跳检测的语句是show slave status
writeHost标签
指定写入节点。host是写节点的名称。url指定写入节点的ip和端口号。user指定写节点数据库的用户名，password指定写节点数据库的密码。
readHost标签
指定读取节点。host是读节点的名称。url指定读取节点的ip和端口号。user指定读节点数据库的用户名，password指定读节点数据库的密码。
启动dble
上述配置完成之后，我们就可以启动dble了。
[root@mycat bin]# ./dble start
Starting dble-server...
Removed stale pid file: /dble/dble.pid启动后查看wrapper日志居然报错了。
INFO | jvm 1 | 2019/12/27 00:34:15 | com.actiontech.dble.config.util.ConfigException: Illegal table conf : table [ orders ] rule function [ func_mod ] partition size : 4 > table datanode size : 2, please make sure table datanode size = function partition size这里报错很明显，主要是这个取模函数默认设置对4取模，大于了我们的dataNode。需要我们在rule.xml中把func_mod函数的partitionCount给改成2.
<function name="func_mod" class="Hash">
<property name="partitionCount">2</property>
<property name="partitionLength">1</property>
</function>重新启动dble，能够成功启动。
创建分片并验证
使用管理端口登录，进行创建datanode的操作。
[root@mysql5 ~]# mysql -uman1 -p -P9066 -h192.168.56.185 -p654321
mysql> create database @@dataNode='dn$1-2';
Query OK, 1 row affected (0.03 sec)分别登录MySQL服务器A和MySQL服务器B查看创建分片的情况。可以看到181上创建了db_1，而182上创建了db_2。
[root@mysql5 ~]# mysql -ubuddy -p -h192.168.56.181 -P3306 -p123456 -e "show databases"
+--------------------+
| Database |
+--------------------+
| information_schema |
| db_1 |
| mysql |
| performance_schema |
| sys |
+--------------------+
[root@mysql5 ~]# mysql -ubuddy -p -h192.168.56.182 -P3306 -p123456 -e "show databases"
+--------------------+
| Database |
+--------------------+
| information_schema |
| db_2 |
| mysql |
| performance_schema |
| sys |
+--------------------+创建表并插入数据
使用服务端口登录，进行创建表和插入数据的操作。此时我们操作的就是逻辑库和逻辑表。
[root@mysql5 ~]# mysql -uroot -h192.168.56.185 -P8066 -p123456
mysql> show databases;
+----------+
| DATABASE |
+----------+
| testdb |
+----------+
1 row in set (0.00 sec)
mysql> use testdb;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql> create table if not exists products (
-> product_id int not null,
-> product_name varchar(250),
-> expire_time date,
-> primary key(product_id)
-> )engine=innodb charset=utf8;
Query OK, 0 rows affected, 1 warning (0.08 sec)
mysql> create table if not exists orders (
-> order_id int not null,
-> product_id int not null,
-> effective_time date,
-> cust_name varchar(20),
-> cust_address varchar(200),
-> primary key(order_id)
-> )engine=innodb charset=utf8;
Query OK, 0 rows affected (0.04 sec)
insert into products values(1,'Package A','2015-12-31')
insert into products values(2,'Package B','2022-07-31');
insert into products values(3,'Package C','2025-12-31');
insert into products values(4,'Package D','2028-05-31');
insert into orders values(1,1,'2019-12-25','caocao','gz');
insert into orders values(1,1,'2019-12-25','caocao','gz');
insert into orders values(2,1,'2019-12-25','liubei','sz');
insert into orders values(3,2,'2019-12-25','sunquan','sh');
insert into orders values(4,2,'2019-12-25','zhugeliang','nj');
insert into orders values(5,3,'2019-12-25','simayi','hz');
insert into orders values(6,4,'2019-12-25','caopi','wh');
insert into orders values(7,1,'2019-12-25','guanyu','cd');
insert into orders values(8,1,'2019-12-25','zhengfei','bj');
insert into orders values(9,3,'2019-12-25','zhaoyun','cd');
insert into orders values(10,1,'2019-12-25','weiyan','cq');
insert into orders values(11,4,'2019-12-25','machao','cs');
insert into orders values(12,1,'2019-12-25','lvbu','cd');
insert into orders values(13,1,'2019-12-25','diaochan','sh');
insert into orders values(14,2,'2019-12-25','huangzhong','hz');
mysql> insert into orders values(1,1,'2019-12-25','caocao','gz');
ERROR 1064 (HY000): bad insert sql, sharding column/joinKey:ID not provided,INSERT INTO orders在插入orders表的时候再次报错，报错很明显，说分片键id没提供。这是因为我们在rule.xml中columns字段不是指定的分片字段order_id。在rule.xml配置一下columns字段为order_id就可以了。
<tableRule name="rule_mod">
<rule>
<columns>order_id</columns>
<algorithm>func_mod</algorithm>
</rule>
</tableRule>验证分片数据否正确
验证db_1分片上的数据。
[root@mysql5 ~]# mysql -ubuddy -h192.168.56.181 -P3306 -p123456 -D db_1 -e "select * from products"
+------------+--------------+-------------+
| product_id | product_name | expire_time |
+------------+--------------+-------------+
| 1 | Package A | 2015-12-31 |
| 1 | Package B | 2022-07-31 |
| 1 | Package C | 2025-12-31 |
| 1 | Package D | 2028-05-31 |
+------------+--------------+-------------+
[root@mysql5 ~]# mysql -ubuddy -h192.168.56.181 -P3306 -p123456 -D db_1 -e "select * from orders"
+----------+------------+----------------+------------+--------------+
| order_id | product_id | effective_time | cust_name | cust_address |
+----------+------------+----------------+------------+--------------+
| 2 | 1 | 2019-12-25 | liubei | sz |
| 4 | 2 | 2019-12-25 | zhugeliang | nj |
| 6 | 4 | 2019-12-25 | caopi | wh |
| 8 | 1 | 2019-12-25 | zhengfei | bj |
| 10 | 1 | 2019-12-25 | weiyan | cq |
| 12 | 1 | 2019-12-25 | lvbu | cd |
| 14 | 2 | 2019-12-25 | huangzhong | hz |
+----------+------------+----------------+------------+--------------+验证db_2分片上的数据。
[root@mysql5 ~]# mysql -ubuddy -h192.168.56.182 -P3306 -p123456 -D db_2 -e "select * from products"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+--------------+-------------+
| product_id | product_name | expire_time |
+------------+--------------+-------------+
| 1 | Package A | 2015-12-31 |
| 1 | Package B | 2022-07-31 |
| 1 | Package C | 2025-12-31 |
| 1 | Package D | 2028-05-31 |
+------------+--------------+-------------+
[root@mysql5 ~]# mysql -ubuddy -h192.168.56.182 -P3306 -p123456 -D db_2 -e "select * from orders"
+----------+------------+----------------+-----------+--------------+
| order_id | product_id | effective_time | cust_name | cust_address |
+----------+------------+----------------+-----------+--------------+
| 1 | 1 | 2019-12-25 | caocao | gz |
| 3 | 2 | 2019-12-25 | sunquan | sh |
| 5 | 3 | 2019-12-25 | simayi | hz |
| 7 | 1 | 2019-12-25 | guanyu | cd |
| 9 | 3 | 2019-12-25 | zhaoyun | cd |
| 11 | 4 | 2019-12-25 | machao | cs |
| 13 | 1 | 2019-12-25 | diaochan | sh |
+----------+------------+----------------+-----------+--------------+总结
这篇我们学习了基本的schema.xml的配置方法，目前还是比较皮毛的。下一篇我们来介绍分片的一些算法。