15. MyCat database fragmentation

        MyCat is an open source database middleware, mainly used to route and distribute database operation requests to multiple database nodes in the backend.

1. Mycat environment construction

        Create the same table in two different databases 

        download mycat

https://github.com/MyCATApache/Mycat-Server https://github.com/MyCATApache/Mycat-Server         Upload the downloaded mycat to linux and decompress it, enter the conf directory and open the server.xml file to configure the account and password

        Open the schema.xml file configuration        

        Configure virtual table

        configure node

        Configure mysql for each node

        Enter the bin directory, start mycat and check the running status

./mycat start

ps -ef|grep myca

        Open port 8066 of mycat

firewall-cmd --zone=public --add-port=8066/tcp --permanent

systemctl restart firewalld.servic

        Connect the three databases with tools, add data, and you will find that every five million IDs are placed in a table, which is determined by the method configured in the schema.xml file of mycat

2. Mycat read and write separation

        Note: It is better to have the same version of mysql master and slave, otherwise it will cause the slave configuration to fail (even if the server-id is set)

Slave is not configured or failed to initialize properly. You must at least set --server-id to enable either a master or a slave. Additional error messages can be found in the MySQL error log.

2.1 master-slave separation of mysql

        Database read-write separation is an essential and important function for large-scale systems or Internet applications with high traffic volume. Insert SQL takes tens of milliseconds, and Select SQL takes several seconds to several minutes to produce results. Many complex SQLs consume the server's CPU very powerfully, no less than the power of an infinite loop. Some complex SQL queries during peak hours will cause the CPU of the database server to explode, and the system will be paralyzed. In severe cases, the database may crash. We should try to avoid single-node databases without a master-slave replication mechanism.

        For MySQL, the standard read-write separation is the master-slave mode. One write node Master is followed by multiple read nodes. The number of read nodes depends on the pressure of the system, usually 1-3 read node configurations.

2.2 The principle of MySQL master-slave replication

1. Record the binary log on the main library (how to set it will be described later). Before each ready-to-commit transaction completes the data update, the main library records the data update event in the binary log, and MySQL records the binary log in the order in which the transactions are submitted rather than in the order in which each statement is executed.

2. The standby library copies the binary log of the main library to its local relay log. The standby library will start a worker thread, called the IO thread. The I/O thread establishes an ordinary client connection with the main library, and then in Start a special binary dump on the main library (binhg dump, the thread does not have a corresponding SQL command), this binary dump thread will read the events in the binary log on the main library

3. The SQL thread of the standby database executes the last step. This thread reads events from the relay log and executes them in the standby database, so as to realize the update of the standby database data.

2.3 MySQL master-slave replication

After the master-slave replication of mysql is completed, mycat is introduced next, and the standard MySQL master-slave replication high-availability configuration is adopted and delivered to Mycat to complete the master-slave automatic switching of the back-end MySQL nodes.

The workflow is:

1. The MySQL node starts the master-slave replication configuration scheme, and configures the master node as the writeNode in Mycat's dataHost, and configures the slave node as readNode.

2. Mycat regularly initiates heartbeat detection for all writeHost and readHost nodes in a dataHost. Under normal circumstances, Mycat will use the first writeHost as a write node, and all DML SQL will be sent to this node, and readHost will be used as a read node.

3. When two or more writeHosts are configured in a dataHost, if the first writeHost fails, Mycat will automatically switch to the next available writeHost to execute DML SQL after the default 3 heartbeat checks fail statement.

4.dataNodeHeartbeatPeriod=10000 The frequency of node heartbeat detection is once every 10s. Mycat will send select users() to the database by default

2.3.1 Mysql master-slave replication operation steps

1. Host operation

1.1. Modify the /etc/my.cnf file of the host MySQL. Pay special attention to the fact that my.cnf is the main configuration file of the MySQL database. This path is the main configuration file specified when I installed MySQL

<span style="background-color:#f8f8f8"><span style="color:#333333">#主机配置，同步db1中的数据，不同步mysql自带的数据库
binlog-do-db=db1
binlog-ignore-db=mysql
#启用日志
log-bin=mysql-bin
#服务器唯一ID，所有MySQL的id不能一致
server-id=<span style="color:#116644">1</span></span></span>

1.2, restart mysql

service mysqld restart

1.3. Log in to MySQL to create an account for the slave to connect to the host and authorize it

mysql -u root -p

grant file on . to 'tiger'@'%' identified by '123456';

Create a user and specify that the user can operate all files ( . ) on the host , and can log in to the host through the tiger account on any ip (@'%') device

GRANT REPLICATION SLAVE,REPLICATION CLIENT ON . TO 'tiger'@'%' IDENTIFIED BY "123456";

The specified slave can log in to the host through the tiger user to read and write files

Generally, the root account is not used, and "%" indicates that all clients may be connected, as long as the account and password are correct. It can be replaced by specific client IP here, such as 10.211.55.5 to enhance security.

1.4. Refresh permissions to make the authorization operation in the previous step take effect must be done

FLUSH PRIVILEGES;

1.5. Check which users are currently on MySQL

select user,host from mysql.user;

1.6. View master status

show master status;

File: The name of the log file that records successful transactions

position: The offset in the log file, which is used to record the position of the content that has not been synchronized, so as to avoid repeated synchronization of the data that has been synchronized

2. Configure from MySQL

2.1. Create a database and table db1 with the same name as the host

2.2. Modify the my.cnf file from MySQL, specify the server-id, and restart the slave

(The MySQL configuration file installed by docker is: cd)

2.3. Log in to the slave server mysql, execute the following command to configure the slave server slaveof host port

docker exec -it mysql bash

mysql -u root -p

change master to master_host='192.168.137.129',master_port=3306,master_user='tiger',master_password='root',master_log_file='mysql-bin.000003',master_log_pos=474; 

If the message should stop slave is reported here, it is because the master-slave structure has been set before and it is still in effect. You should stop this state first and then enter the above command

Be careful not to disconnect in the middle of the statement, masteruser is the database account that performs the synchronization operation, "992" has no single quotation marks (449 here is the value of position seen in show master status, and mysql-bin.000002 here is the corresponding file value).

2.4. Restart the slave mysql

exit exit mysql

exit exit docker

service mysqld restart

2.5. Log in to the slave machine, enter the following command to start the copy function from the server

docker exec -it mysql bash

mysql -u root -p

start slave;

2.6. Check the status of the replication function from the server

show slave status \G;

If "Slave_IO_Running" is NO, it is likely that the UUIDs of the two databases are exactly the same. The solution:

• Switch to the root user on the slave machine and switch the current working path to the system root directory

  cd /

• Enter the following command to find the file where the database UUID is located

  find -name auto.cnf

• Open the found file and modify the id

• Restart the slave

2.7. Insert a piece of data into the corresponding table of the corresponding database in the host, and then query the corresponding table of the slave to see if there is the same data