Mycat installation and configuration read and write separation

Today, I bring you an article about how Mycat implements read-write separation based on MySQL master-slave replication. What you need to note here is that MySQL’s master-slave replication relies on MySQL’s own master-slave replication mechanism. Mycat is not responsible for MySQL’s master-slave replication. Slave replication, for the master-slave replication configuration of MySQL, you can refer to the blog post "MySQL - Master-slave replication configuration". Okay, let's get into today's topic.

MyCat read and write separation

Mycat is an open source distributed database system, but since a real database requires a storage engine, and Mycat does not have a storage engine, it is not a fully distributed database system.

Mycat is a database middleware, which can also be understood as a database agent. In the architecture system, it is a component located between the database and the application layer, and is transparent to the application layer, that is, the database cannot feel the existence of mycat and thinks it is a directly connected mysql database (actually a connected mycat, mycat implementation MySQL’s native protocol)

Mycat’s three major functions: table splitting, read-write separation, and master-slave switching

• Splitting tables
  For tables with a large amount of data (above tens of millions), the performance of MySQL will be greatly reduced, so try to control the size of each table to one million levels. For a table with a large amount of data, you can consider placing these records in different databases according to certain rules. In this way, the data volume of each database is not too large, and the performance will not be greatly lost. Implementation of mycat sub-table: First configure the logical table in
  mycat , and indicate in the configuration which physical libraries this table is on. scheme.xmlThe name of this logical table is consistent with the name in the real database! Then you need to configure the sharding rules, that is, according to what logic the database should be divided!

• Separation of reading and writing.
  Statistics show that a large number of operations on the database are read operations, which generally account for more than 70% of all operations. Therefore, it is necessary to separate reading and writing. If we do not separate reading and writing, then the slave library will also be a big waste.

• .MyCat basic elements
  1. Logical library, exists in mycat, which is equivalent to mysql database for applications. The backend may correspond to multiple physical databases, and data is not saved in the logical library.
  2. Logical tables, tables in the logical library, are In terms of application, it is equivalent to a data table of MySQL. The backend may correspond to tables in multiple physical databases and does not save data.
  Logical table classification
  1. Sharded table, a table that is horizontally split, has the same table structure but is stored in Tables in different databases, the collection of all sharded tables is a complete table
  2. Non-sharded table, a vertically split table, a complete table is saved in one database
  3. Global table, all sharded tables Tables that exist in the tablet database, such as dictionary tables, are small in number and are maintained and updated by mycat. 4.
  ER relational table is unique to mycat. The child table depends on the parent table and is guaranteed to be in the same database.

1 Introduction to Mycat

2 What is read-write separation?

Note: Master-slave replication needs to be implemented before realizing read-write separation.
You can refer to my previous article "MySQL Master-Slave Synchronization Configuration"

In an actual production environment, if the data read and write operations are all performed in the same database server, when encountering a large number of concurrent read or write operations, there is no way to meet the actual demand, and the database throughput will face a huge problem. bottleneck pressure.

• Master-slave replication
  builds a master-slave architecture and splits the database into a master database and a slave database. The master database is responsible for processing transactional additions, deletions, and modifications, and the slave database is responsible for processing query operations, which can effectively avoid row locks caused by data updates. The query performance of the entire system is greatly improved.

• Read-write separation
  Read-write separation means that the main database handles transactional operations and the slave database handles select queries. Database replication is used to synchronize data changes caused by transactional queries to the slave database, and the master database can also select queries.

The data content in data nodes with separate reading and writing is consistent.

3 Environment installation/configuration

Because MyCat is developed in Java, JDK needs to be installed to run MyCat (JRE is enough to be precise), and JDK1.7 or above is required.

Download MyCat from the official website: http://dl.mycat.org.cn/1.6.7.4/Mycat-server-1.6.7.4-release/

1. Unzip the file and copy it to linux/usr/local/

   tar -zxvf Mycat-server-1.6.7.4-release-20200105164103-linux.tar.gz -C /usr/local/
   

2. After unzipping, enter the conf directory and modify server.xml
   the account information configured in server.xml as follows (mainly configuring the default account, database, read-only account, and password):

   <?xml version="1.0" encoding="UTF-8"?>
   <!-- - - Licensed under the Apache License, Version 2.0 (the "License"); 
   	- you may not use this file except in compliance with the License. - You 
   	may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 
   	- - Unless required by applicable law or agreed to in writing, software - 
   	distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT 
   	WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the 
   	License for the specific language governing permissions and - limitations 
   	under the License. -->
   <!DOCTYPE mycat:server SYSTEM "server.dtd">
   <mycat:server xmlns:mycat="http://org.opencloudb/">
   	<system>
   	<!-- 这里配置的都是一些系统属性，可以自己查看mycat文档 -->
   	<property name="defaultSqlParser">druidparser</property>
   	<property name="charset">utf8mb4</property>
   	</system>
    
   							
   	<!-- 用户1，对应的MyCat逻辑库连接到的数据节点对应的主机为主从复制集群 -->
   	<user name="user1">
   		<property name="password">123456</property>
   		<property name="schemas">pcx_schema</property>
   	</user>
   							
   	<!-- 用户2，只读权限-->
   	<user name="user2">
   		<property name="password">123456</property>
   		<property name="schemas">pcx_schema</property>
   		<property name="readOnly">true</property>
   	</user>
    
   </mycat:server>
   

3. Open the file in the conf folder schema.xmland configure it as follows:

   <?xml version="1.0"?>
   <!DOCTYPE mycat:schema SYSTEM "schema.dtd">
   <mycat:schema xmlns:mycat="http://org.opencloudb/">
   	
   	<!-- 定义MyCat的逻辑库 -->
   	<schema name="pcx_schema" checkSQLschema="false" sqlMaxLimit="100" dataNode="pcxNode"></schema>
    
   	<!-- 定义MyCat的数据节点 -->
   	<dataNode name="pcxNode" dataHost="dtHost" database="pcx" />
    
   	
   	<!-- 定义数据主机dtHost，连接到MySQL读写分离集群 ,schema中的每一个dataHost中的host属性值必须唯一-->
   	<!-- dataHost实际上配置就是后台的数据库集群，一个datahost代表一个数据库集群 -->
   	<!-- balance="1"，全部的readHost与stand by writeHost参与select语句的负载均衡-->
   	<!-- writeType="0"，所有写操作发送到配置的第一个writeHost，这里就是我们的hostmaster，第一个挂了切到还生存的第二个writeHost-->
   	<!--方式二-->
   	<dataHost name="dtHost" maxCon="500" minCon="20" balance="1"
   		writeType="0" dbType="mysql" dbDriver="native" switchType="2" slaveThreshold="100">
   		<!--心跳检测 -->
   		<heartbeat>select user()</heartbeat>
   		
   		<!--配置后台数据库的IP地址和端口号，还有账号密码 -->
   		<writeHost host="hostMaster" url="192.168.1.50:3306" user="root" password="123456" />
   		<writeHost host="hostSlave" url="192.168.1.51:3306" user="root" password="123456" />
   	</dataHost>
   	
   </mycat:schema>
   

   There is a second configuration dataHost

   <dataHost name="dtHost" maxCon="500" minCon="20" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="2"  slaveThreshold="100">
       <heartbeat>select user()</heartbeat>
       <writeHost host="hostMaster" url="192.168.1.50:3306" user="root" password="123456">
           <readHost host="hostSlave" url="192.168.1.51:3306" user="root" password="123456" />
       </writeHost>
   </dataHost>
   

   Return to the bin path and execute the command to start: ./mycat start
   The second configuration of the above two configurations is unavailable when writing fails and reading is disabled. The first configuration can continue to be used, so the first configuration is recommended.

4 Detailed explanation of key attributes

The following only introduces common configuration tags. For other information, please refer to the official website.

balance：

Notes on the balance attribute of the dataHost tag (write operations are all on the host):

1. When Balance="0", the read-write separation operation is not enabled, and all read operations will be sent to the first writeHost (readHost will not be sent) 2.
When Balance="1", all readHost and stand by writeHost participate in the selection Statement load balancing, simply put, when the dual-master dual-slave mode (M1->S1, M2->S2, and M1 and M2 are mutually active and backup), under normal circumstances, M2, S1, and S2 all participate in the select statement. Load balancing
3. When Balance="2", read operations will be randomly sent to writeHost and readHost. Theoretically, load balancing
4. When Balance="3", all read requests will be randomly distributed to the readhost corresponding to writerHost for execution. writerHost does not bear the reading pressure. Note that balance=3 is only available in 1.4 and later versions, but not in 1.3.

Generally set to "1": dual master and dual slave

writeType：

1. When writeType="0", all write operations are sent to the first configured writeHost. If the first one hangs, it will be switched to the second surviving writeHost. The one that has been switched after restarting shall prevail. The switching record is in the configuration In the file: dnindex.properties
2. When writeType="1", all write operations are randomly sent to the configured writeHost. It will be abandoned after 1.5 and is not recommended.

switchType：

1. When switchType="-1", no automatic switching.
2. When writeType="1", default value, automatic switching.
3. When writeType="2", whether to switch is determined based on the status of MySQL master-slave synchronization.

5 Test mycat read and write separation

MyCat default port is 8066

1. Use Navicat to connect to MyCat. The password is configured in server.xml. The default is 123456.
   
2. Create user databases in the main database and slave database, with character set utf8mb4, default sorting rules, and create a new user table.
   

After that, you can test it. Because the master and backup have been configured before, when inserting data, the data is first inserted into the master and then synchronized to the slave.
To verify the separation of reading and writing, we only need to change the slave data and see if MyCat reads the slave data.
I manually modified the data from the database here

At this point, the entire "Mycat installation and configuration reading and writing separation" has been completed~