mysql read and write separation

Continued from the previous blog,
Mysql read-write separation
mainly uses mysql-proxy. What I understand is that only the host can write, but the slave cannot. If the slave can write, other slaves cannot be synchronized. The master writes, and other slaves can use it to synchronize the data of the master.

Online query:
MySQL-Proxy is a middle-tier proxy. Simply put, MySQL-Proxy is a connection pool, which is responsible for forwarding connection requests from foreground applications to background databases. By using lua scripts, complex connection control and Filtering to achieve read-write separation and load balancing. For applications, MySQL-Proxy is completely transparent, and applications only need to connect to the listening port of MySQL-Proxy. Of course, this proxy machine may become a single point of failure, but multiple proxy machines can be used for redundancy, and the connection parameters of multiple proxy machines can be configured in the connection pool configuration of the application server.
One of the more powerful functions of MySQL-Proxy is to achieve "read-write separation". The basic principle is to let the main database process transactional queries and let the slave database process SELECT queries. Database replication is used to synchronize changes caused by transactional queries to slaves in the cluster.

Experimental process:
1. Add a proxy user:
useradd -r mysql-proxy

2.安装mysql-proxy
tar xvf mysql-proxy-0.8.5-linux-el6-x86-64bit.tar.gz -C /usr/local/
cd /usr/local/
ln -s mysql-proxy-0.8.5-linux-el6-x86-64bit/ mysql-proxy

3. Changes in environment variables (so that the system can use mysl directly)
vim /root/.bash_profile

 PATH=$PATH:$HOME/bin:/usr/local/mysql-proxy/bin

source /root/.bash_profile

Note: During this experiment, download and install mysql in the same way as the above master, and start the service

4. Start the mysql-proxy service
cd /usr/local/mysql-proxy

 mysql-proxy --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.25.78.3:3306" --proxy-read-only-backend-addresses="172.25.78.2:3306"

The meaning of the connected parameters:

--proxy-address=host:port ———— 代理服务监听的地址和端口；
--admin-address=host:port ———— 管理模块监听的地址和端口；
--proxy-backend-addresses=host:port ———— 后端mysql服务器的地址和端口；
--proxy-read-only-backend-addresses=host:port——后端只读mysql服务器的地址和端口；
--proxy-lua-script=file_name ———— 完成mysql代理功能的Lua脚本；
--daemon ———— 以守护进程模式启动mysql-proxy；
--keepalive ———— 在mysql-proxy崩溃时尝试重启之；
--log-file=/path/to/log_file_name ———— 日志文件名称；
--log-level=level ———— 日志级别；
--log-use-syslog ———— 基于syslog记录日志；
--plugins=plugin,.. ———— 在mysql-proxy启动时加载的插件；
--user=user_name ———— 运行mysql-proxy进程的用户；
--defaults-file=/path/to/conf_file_name ———— 默认使用的配置文件路径；其配置段使用[mysql-proxy]标识；
--proxy-skip-profiling ———— 禁用profile；
--pid-file=/path/to/pid_file_name ———— 进程文件名；

5. Create an account and password for testing on the master master

mysql> grant all on *.* to root@'172.25.78.%' identified by 'westos';（）
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

6. Test whether you can connect to mysql-proxy through proxy port 4040

[root@master profile.d]# mysql -uroot -pwestos -h172.25.78.4 --port=4040（连接mysql-proxy）
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 19
Server version: 5.1.71-log Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye
综上显示已经成功通过读写分离代理端口4040连接进mysql-proxy

7. mysql-proxy itself does not realize read-write separation, it mainly relies on lua script to realize, add read-write separation script, restart service (there is a rw-splitting.lua script in the installation directory of mysql-proxy, which is specially used to realize read-write separation Write split, the path is /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua)

 mysql-proxy --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.25.78.3:3306" --proxy-read-only-backend-addresses="172.25.78.2:3306" --proxy-lua-script="/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua "

8. Check whether it is successful after adding the lua read-write separation script.

[root@proxy mysql-proxy]# tail /var/log/mysql-proxy.log 
2017-08-02 15:07:40: (message) proxy listening on port :4040
2017-08-02 15:07:40: (message) added read/write backend: 172.25.78.3:3306
2017-08-02 15:07:40: (message) added read-only backend: 172.25.78.2:3306
2017-08-02 15:24:48: (critical) plugin proxy 0.8.5 started
2017-08-02 15:24:48: (debug) max open file-descriptors = 1024
2017-08-02 15:24:48: (critical) network-socket.c:492: bind(0.0.0.0:4040) failed: Address already in use (98)
2017-08-02 15:24:48: (critical) chassis-mainloop.c:270: applying config of plugin proxy failed
2017-08-02 15:24:48: (critical) mysql-proxy-cli.c:599: Failure from chassis_mainloop. Shutting down.
2017-08-02 15:24:48: (message) Initiating shutdown, requested from mysql-proxy-cli.c:600
2017-08-02 15:24:48: (message) shutting down normally, exit code is: 1
[root@proxy mysql-proxy]# netstat -tunlp | grep 4040（添加脚本之后，代理端口启动成功）
tcp        0      0 0.0.0.0:4040                0.0.0.0:*                   LISTEN      1418/mysql-proxy    

9. (from the network) Provide a management interface for mysql-proxy, which is convenient to check the status and access type of the back-end mysql server at any time in the future, and realize the management function.
The following provides a script for the management interface, which is also a LUA script. It is recommended to follow The read-write separation scripts are placed in the same directory.

# vim /usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua
function set_error(errmsg)
proxy.response = {
type = proxy.MYSQLD_PACKET_ERR,
errmsg = errmsg or "error"
}
end
function read_query(packet)
if packet:byte() ~= proxy.COM_QUERY then
set_error("[admin] we only handle text-based queries (COM_QUERY)")
return proxy.PROXY_SEND_RESULT
end
local query = packet:sub(2)
local rows = { }
local fields = { }
if query:lower() == "select * from backends" then
fields = {
{ name = "backend_ndx",
type = proxy.MYSQL_TYPE_LONG },
{ name = "address",
type = proxy.MYSQL_TYPE_STRING },
{ name = "state",
type = proxy.MYSQL_TYPE_STRING },
{ name = "type",
type = proxy.MYSQL_TYPE_STRING },
{ name = "uuid",
type = proxy.MYSQL_TYPE_STRING },
{ name = "connected_clients",
type = proxy.MYSQL_TYPE_LONG },
}
for i = 1, #proxy.global.backends do
local states = {
"unknown",
"up",
"down"
}
local types = {
"unknown",
"rw",
"ro"
}
local b = proxy.global.backends[i]
rows[#rows + 1] = {
i,
b.dst.name,          -- configured backend address
states[b.state + 1], -- the C-id is pushed down starting at 0
types[b.type + 1],   -- the C-id is pushed down starting at 0
b.uuid,              -- the MySQL Server's UUID if it is managed
b.connected_clients  -- currently connected clients
}
end
elseif query:lower() == "select * from help" then
fields = {
{ name = "command",
type = proxy.MYSQL_TYPE_STRING },
{ name = "description",
type = proxy.MYSQL_TYPE_STRING },
}
rows[#rows + 1] = { "SELECT * FROM help", "shows this help" }
rows[#rows + 1] = { "SELECT * FROM backends", "lists the backends and their state" }
else
set_error("use 'SELECT * FROM help' to see the supported commands")
return proxy.PROXY_SEND_RESULT
end
proxy.response = {
type = proxy.MYSQLD_PACKET_OK,
resultset = {
fields = fields,
rows = rows
}
}
return proxy.PROXY_SEND_RESULT
end

10. Restart mysql-proxy and add the following startup options
–plugins=admin ———— The plugin loaded when mysql-proxy starts;
–admin-username=”admin” The user who runs the mysql-proxy process management;
–admin- password=”admin” Password
–admin-lua-script=”/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua” The path of the configuration file used by the plugin;
at this time, it is found that there are many more logs and ports There is a port 4041 which is just added for management.

 mysql-proxy --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.25.78.3:3306" --proxy-lua-script="/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua 
--plugins=admin --admin-username="admin"  --admin-password="admin"
--admin-lua-script="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"

11. Find a client to verify the management interface function, the client can be any client except mysql-proxy. Remember to use the management port 4041 to log in, the password is admin
mysql -uadmin -p -h172.25.78.4 –port=4041

12. Execute the following command after logging in to the interface. This interface can only execute this command to check the read-write separation status.
At this time, because there are no read and write operations, the status is unknown.

 mysql> select * from backends;

Test read-write separation status:
Find any server that provides mysql client connection to test. We will use the main mysql test for now.
Under normal circumstances, no matter which server you operate on, it will first pass through mysql-proxy and then distribute it to each read-write mysql.
mysql>mysql -uroot -p -h172.25.78.4 –port=4040 -e 'select user from mysql.user;'
mysql>mysql -uroot -p -h172.25.78.4 –port=4040 -e 'create database abc;'

14. At this point you will see the status change from unknown to up.

mysql -uadmin -p -h172.25.78.4 –port=4041
mysql> select * from backends;