Demo's Mycat read-write separation

Foreword
Database read-write separation is an essential and important function for large-scale systems or Internet applications with high access volume; for MySQL, the standard read-write separation is the master-slave mode, and a write node Master is followed by multiple A read node consists of two steps, one is the master-slave synchronization of the data source, and the other is the read-write distribution of SQL; Mycat is not responsible for any data synchronization, and the specific data synchronization still depends on the function of the Mysql database itself.

Mysql master-slave replication
Prepare two hosts and install the same version of Mysql database. Next, prepare to configure the master-slave replication configuration of Mysql:
1. Configure the Master
to configure my.ini or my.conf as follows:

server-id=1
binlog_format=STATEMENT
log_bin=D:/mysql/bin-log.log

server-id : generally set to IP, pay attention to be unique; binlog_format : set the format of binlog; log_bin : enable log_bin

2. Configure Slave
to configure my.ini or my.conf as follows:

server-id=2
binlog_format=STATEMENT
relay_log=mysql-relay-bin

relay_log : The I/O thread of the slave server reads the binary log of the master server and records it to the local file of the slave server, and then the SQL thread reads the contents of the relay-log log and applies it to the slave server;

More binlog configuration: https://my.oschina.net/OutOfMemory/blog/1571107

3. Restart Master and Slave respectively

service mysqld restart;

4. Query the status of the Master

mysql> show master status;
+----------------+----------+--------------+------------------+
| File           | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+----------------+----------+--------------+------------------+
| bin-log.000001 |      107 |              |                  |
+----------------+----------+--------------+------------------+

5.Slave set Master

change master to master_host='192.168.110.1', master_user='root', master_password='root', master_port=3306, master_log_file='bin-log.000001', master_log_pos=107, master_connect_retry=10; 

master_host : MasterIp; master_user : the user used to synchronize data; master_password : password used to synchronize data; master_log_file : which log file Slave starts to synchronize data from; master_log_pos : where to start synchronizing data;

6.Slave starts synchronization

start slave; 

7. View Slave synchronization status

show slave status;

Both Slave_IO_Running and Slave_SQL_Running are Yes to indicate that the work has started;

8. Test master-slave synchronization
Create database db1 on the Master:

create database db1;

Check whether the slave can be synchronized to the database;

Mycat read-write separation
Mycat read-write separation is configured in the conf/schema.xml file:

<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
    <heartbeat>select user()</heartbeat>
    <writeHost host="hostM1" url="localhost:3306" user="root" password="root">
            <readHost host="hostS2" url="192.168.237.128:3306" user="root" password="root" />
    </writeHost>
</dataHost>

maxCon : the maximum connection of the connection pool, minCon : the minimum connection of the example connection pool; dbType : specify the database type of the backend connection; dbDriver : specify the Driver used to connect to the backend database, the current optional values ​​are native and JDBC;
balance : Load balancing type
0: Do not enable read-write separation mechanism,
1: All readHost and stand by writeHost participate in load balancing of select statement,
2: All read operations are randomly distributed on writeHost and readhost,
3: All read requests are randomlyDistributed to the readhost corresponding to the writerHost for execution, the writerHost does not bear the read pressure
writeType : load balancing type
0: all write operations are sent to the first writeHost configured, the first one hangs and switches to the surviving second writeHost, after restarting The one that has been switched shall prevail. The switch is recorded in the configuration file: dnindex.properties
1: All write operations are randomly sent to the configured writeHost, which is deprecated after 1.5.
switchType : switch type
-1: means no automatic switching,
1:default value is automatically switched,
2: Determine whether to switch based on the status of MySQL master-slave synchronization (the heartbeat statement is show slave status),
3: Based on the switching mechanism of MySQL galary cluster (heartbeat statement is show status like 'wsrep%')
slaveThreshold : Slave database delay threshold
1.4 began to support the read-write separation mechanism of MySQL master-slave replication status binding, making reading more secure and reliable; switchType=”2” and slaveThreshold=”100”, which means to enable the read-write separation and switching mechanism of MySQL master-slave replication status binding. Mycat heartbeat mechanism detects "Seconds_Behind_Master", "Slave_IO_Running" in show slave status, "Slave_SQL_Running" three fields are used to determine the current master-slave synchronization status and Seconds_Behind_Master master-slave replication delay. When Seconds_Behind_Master>slaveThreshold, the read-write separation filter will filter out this slave machine to prevent reading old data from a long time ago, and When the master node is down, the switching logic will check whether the Seconds_Behind_Master on the slave is 0. If it is 0, it means that the master-slave is synchronized, and it can be safely switched, otherwise it will not switch;
heartbeat label : It is used to check the heartbeat with the back-end database. Statement;
writeHost tag, readHost tag : writeHost specifies a write instance, readHost specifies a read instance, and multiple writeHost and readHost can be defined in one dataHost. If the back-end database specified by writeHost is down, then all readHost bound to this writeHost will be unavailable. Use; because the writeHost is down, the system will automatically detect it and switch to the standby writeHost.

Demo show
1. Prepare test data
and execute insert statement:

insert into travelrecord (id,name) values(1,'hehe');

Query log:

ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=root,txIsolation=3, autocommit=true, schema=TESTDB]insert into travelrecord (id,name) values(1,'hehe'), route={
   1 -> dn1{insert into travelrecord (id,name) values(1,'hehe')}
} rrs 

Insert into dn1 through sharding strategy

2. Do not open the read-write separation mechanism, set balance=0
to execute the query:

SELECT * FROM travelrecord;

Query log:

select read source hostM1 for dataHost:localhost1

It can be found that the queried data still comes from hostM1, without read-write separation

3. readHost and stand by writeHost participate in the load balancing of the select statement, set balance=1
to execute the query:

SELECT * FROM travelrecord;

Query log:

select read source hostS2 for dataHost:localhost1

Because there is no stand by writeHost here, all read operations go to hostS2

4. All read operations are randomly distributed on writeHost and readhost, set balance=2
to execute the query multiple times:

SELECT * FROM travelrecord;

Query log:

select read source hostS2 for dataHost:localhost1
select read source hostM1 for dataHost:localhost1

Because it is distributed randomly, the query statement will switch between hostS2 and hostM1

5. Distribute to the readhost corresponding to the writerHost for execution, set balance=3
to execute the query multiple times:

SELECT * FROM travelrecord;

Query log:

select read source hostS2 for dataHost:localhost1

Execute multiple times to find that the query statement is only on hostS2

6. Automatically switch, switchType=1
stops hostM1, and executes the query:

SELECT * FROM travelrecord;
ERROR 1105 (HY000): backend connect: java.net.ConnectException: Connection refus
ed: no further information

If the backend database specified by writeHost is down, all readHost bound by this writeHost will be unavailable

7. Automatically switch, set balance=3, switchType=1,
first change readHost to writeHost from the library:

<writeHost host="hostM1" url="localhost:3306" user="root" password="root"/>
<writeHost host="hostM2" url="192.168.237.128:3306" user="root" password="root" />

Execute the query:

SELECT * FROM travelrecord;

Query log:

select read source hostM1 for dataHost:localhost1

At this time, there is no slave library in hostM1, so even if balance=3, it will be queried from hostM1

Stop at hostM1 and execute the query and insert statement:

SELECT * FROM travelrecord;

select read source hostM2 for dataHost:localhost1

Execute the insert:

insert into travelrecord (id,name) values(3,'hehe');

node=dn1{insert into travelrecord (id,name) values(3,'hehe')}, packetId=1], host=192.168.237.128, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=true

Check that conf/dnindex.properties has been updated as follows:

#update
#Thu Mar 08 15:20:31 CST 2018
localhost1=1

hostM1 is the 0th, hostM2 is the 1st, so the subsequent main libraries are all hostM2

Restart Mycat and execute the query and insert statement:

SELECT * FROM travelrecord;

select read source hostM2 for dataHost:localhost1

Execute the insert:

insert into travelrecord (id,name) values(4,'hehe');

node=dn1{insert into travelrecord (id,name) values(4,'hehe')}, packetId=1], host=192.168.237.128, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=true

It can be found that because dnindex.properties has recorded the main host, the subsequent queries and insertions are all located to hostM2

8. Do not switch automatically, set balance=1, switchType=-1

Note: For convenience, set localhost1 in conf/dnindex.properties to 0

SELECT * FROM travelrecord;

select read source hostM2 for dataHost:localhost1

balance=1, all query statements are in hostM2, stop hostM1, and execute the query:

SELECT * FROM travelrecord;

select read source hostM2 for dataHost:localhost1

Execute the insert statement:

insert into travelrecord (id,name) values(5,'hehe');
ERROR 1184 (HY000): Connection refused: no further information

You can query the data that cannot be inserted, and the value in conf/dnindex.properties has not been updated

9. For safe switching, set balance=1, switchType=2
to switch normally, and execute the insert statement:

insert into travelrecord (id,name) values(6,'hehe');

Stop hostM1, at this time dnindex.properties is updated, localhost1=1, indicating that it has been switched to hostM2

Delayed switching, in order to simulate the synchronization delay, lock the table of hostM2:

LOCK TABLES travelrecord write;

Execute the insert statement:

insert into travelrecord (id,name) values(7,'hehe');

Stop hostM1, at this time dnindex.properties is not updated, localhost1=0, indicating no switching

unlock table

UNLOCK TABLES;

10. To avoid read delay, set balance=3, switchType=2, slaveThreshold=1, and configure writeHost at the same time

<writeHost host="hostM1" url="localhost:3306" user="root"
    <readHost host="hostS2" url="192.168.237.128:3306" user="root" password="root" />
</writeHost>

first lock the table

LOCK TABLES travelrecord write;

Then execute insert data:

insert into travelrecord (id,name) values(8,'hehe');

Execute the query statement at this point:

SELECT * FROM travelrecord;

select read source hostM1 for dataHost:localhost1

Unlock the table and then query the table travelrecord again. The query log is as follows:

select read source hostS2 url for dataHost:localhost1

In the case of delay in synchronizing data from the table, query data to hostM1, and when synchronization is completed, query data to hostS2 again

Summary
This article mainly introduces Mycat's read-write separation and related configurations, and then shows Mycat's read-write separation mechanism in the form of Demo, in which various configurations are shown as much as possible, and strive to understand Mycat's read-write separation by full name.