【MySql】ProxySQL

ProxySQL is a flexible and powerful MySQL proxy layer. It is a MySQL middleware that can be used in the production environment. It can realize the separation of reading and writing, supports query routing function, supports dynamic designation of a certain SQL for caching, and supports dynamic loading of configuration and faults. Toggle and some SQL filtering functions. There are also similar products such as DBproxy, MyCAT, OneProxy, etc. However, after repeated comparisons and tests, I still feel that ProxySQL is a reliable and stable MySQL middleware product with unknown performance. It has two official versions and percona versions. The percona version is modified on the basis of the official version, adding Several more practical tools have been introduced. It is recommended to use the official version for the production environment. ProxySQL is developed in C++ language, although it is also a lightweight product. The

main functions include:

1. read-write separation
2. load balancing
3. connection pool
4. Automatically remove the downed DB
5. Load configuration dynamically
6. Access control
7. query cache
8. ProxySQL cluster

ProxySQL operation mechanism

• runtime: the configuration file used in running
• memory: Provide users with dynamic modification of configuration files
• disk: Save the modified configuration to the disk SQLit table (ie: proxysql.db)

The RUNTIME layer
represents the current effective configuration of ProxySQL, including global_variables, mysql_servers, mysql_users, and mysql_query_rules. The configuration here cannot be modified directly, it must be loaded from the next layer.

The MEMORY layer
is configured in the main that is usually modified on the mysql command line, and can be considered as a mirror image of the SQLite database in memory. We modify the configuration through Set to modify the content of this layer first.

The configuration of the persistent storage of the DISK layer
, generally in $(DATADIR)/proxysql.db, will be loaded from the hard disk when restarting. The /etc/proxysql.cnf file is only used during the first initialization. After that, if you want to modify the listening port, etc., you still need to modify it in the management command line, and then save it to the hard disk.

ProxySQL installation

## 配置proxysql源
[root@localhost yum.repos.d]# vim ProxySQL.repo
 
[proxysql_repo]
name=ProxySQL
baseurl=http://repo.proxysql.com/ProxySQL/proxysql-2.0.x/centos/8
gpgcheck=1
gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
 
[root@localhost ~]# rpm -ivh proxysql-2.0.15-1-centos8.x86_64.rpm 
warning: proxysql-2.0.15-1-centos8.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 79953b49: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:proxysql-2.0.15-1                warning: group proxysql does not exist - using root
warning: group proxysql does not exist - using root
################################# [100%]
Created symlink /etc/systemd/system/multi-user.target.wants/proxysql.service → /etc/systemd/system/proxysql.service.
 
## 启动proxysql服务
[root@localhost ~]# systemctl enable --now proxysql
[root@localhost yum.repos.d]# ss -antl
State          Recv-Q         Send-Q                   Local Address:Port                   Peer Address:Port         
LISTEN         0              128                            0.0.0.0:111                         0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:6032                        0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:6033                        0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:6033                        0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:6033                        0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:6033                        0.0.0.0:*            
LISTEN         0              128                            0.0.0.0:22                          0.0.0.0:*            
LISTEN         0              128                               [::]:111                            [::]:*            
LISTEN         0              128                               [::]:22                             [::]:* 

Admin management interface of ProxySQL

When Proxysql starts, it will listen to two ports:

• admin management interface, the default port is 6032. This port is used to view and configure ProxySQL
• The interface for receiving SQL statements. The default port is 6033. This interface is similar to MySQL's port 3306.
  
  Locally use admin to manage ProxySQL (admin is the default management user and only allows local login)

[root@localhost ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.30 (ProxySQL Admin Module)
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MySQL [(none)]> 
 
 
 
## 查看sqlite库中数据库
MySQL [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main          |                                     |
| 2   | disk          | /var/lib/proxysql/proxysql.db       |
| 3   | stats         |                                     |
| 4   | monitor       |                                     |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.001 sec)

Since the configuration of ProxySQL is stored in several built-in libraries, it is very convenient to modify the configuration of ProxySQL by sending some SQL commands through the management interface. ProxySQL will parse certain specific commands valid for ProxySQL sent through this interface, and convert them reasonably and send them to the embedded SQLite3 database engine to run

The configuration of ProxySQL is almost all operated through the management interface. Through the Admin management interface, almost all configurations can be modified online and made effective. Only the configuration of two variables must restart ProxySQL to take effect, they are:
mysql-threads and mysql-stacksize

Variables related to the admin management interface

admin-admin_credentials

The admin-admin_credentials variable controls the administrator account for the admin management interface. The default administrator account and password are admin:admin, but this default user can only be used locally. If you want to connect to ProxySQL remotely, for example, use navicat on Windows to connect to the ProxySQL management interface on Linux, you must customize an administrator account.

添加管理员帐户

mysql> select @@admin-admin_credentials;
+---------------------------+
| @@admin-admin_credentials |
+---------------------------+
| admin:admin               |
+---------------------------+
1 row in set (0.00 sec)

//设置管理员帐号myadmin,密码yei123!
mysql> set admin-admin_credentials='admin:admin;myadmin:yei123!';
Query OK, 1 row affected (0.00 sec)

mysql> select @@admin-admin_credentials;
+-----------------------------+
| @@admin-admin_credentials   |
+-----------------------------+
| admin:admin;myadmin:yei123! |
+-----------------------------+
1 row in set (0.00 sec)

mysql> load admin variables to runtime;
Query OK, 0 rows affected (0.00 sec)

mysql> save admin variables to disk;   
Query OK, 35 rows affected (0.00 sec)

修改后，就可以使用该用户名和密码连接管理接口

[root@xian ~]# mysql -umyadmin -pyei123! -h192.168.116.180 -P6032
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

所有的配置操作都是在修改main库中对应的表

mysql> select * from global_variables where variable_name='admin-admin_credentials'; 
+-------------------------+-----------------------------+
| variable_name           | variable_value              |
+-------------------------+-----------------------------+
| admin-admin_credentials | admin:admin;myadmin:yei123! |
+-------------------------+-----------------------------+
1 row in set (0.01 sec)

It is necessary to distinguish between the user name of the admin management interface and the user name in mysql_users

The user of the admin management interface is connected to the management interface (default port 6032) to manage and configure ProxySQL. The user
name in the mysql_users table is the user used by the application to connect to ProxySQL (default port 6033) and ProxySQL to connect to the backend MySQL Servers . Its role is to send and route SQL statements, similar to port 3306 of MySQL Server. Therefore, the users in this table must already exist on the backend MySQL Server and the users who have authorized the
admin management interface must not exist in mysql_users. user

admin-stats_credentials

The admin-stats_credentials variable controls ordinary users of the admin management interface. Users in this variable do not have super administrator privileges and can only view statistical data in the monitor library and the main library. Other libraries are invisible and do not have any write permissions.

The default common user name and password are both stats. Like admin, it can only be used for local login by default. If you want people to view remotely, you need to add a dedicated user for viewing

mysql> select @@admin-stats_credentials;
+---------------------------+
| @@admin-stats_credentials |
+---------------------------+
| stats:stats               |
+---------------------------+
1 row in set (0.00 sec)

//添加专有的查看用户
mysql> set admin-stats_credentials='stats:stats;mystats:yei123!';
Query OK, 1 row affected (0.00 sec)

mysql> select @@admin-stats_credentials;
+-----------------------------+
| @@admin-stats_credentials   |
+-----------------------------+
| stats:stats;mystats:yei123! |
+-----------------------------+
1 row in set (0.00 sec)

mysql> load admin variables to runtime;
Query OK, 0 rows affected (0.00 sec)

mysql> save admin variables to disk;  
Query OK, 35 rows affected (0.00 sec)

同样，这个变量中的用户必须不能存在于mysql_users表中
使用mystats用户远程连接查看

[root@yeqixian ~]# mysql -umystats -pyei123! -h192.168.116.180 -P6032
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

mysql> show tables from main;
+----------------------------------------------------+
| tables                                             |
+----------------------------------------------------+
| global_variables                                   |
| mysql_aws_aurora_hostgroups                        |
| mysql_collations                                   |
| mysql_firewall_whitelist_rules                     |
| mysql_firewall_whitelist_sqli_fingerprints         |
| mysql_firewall_whitelist_users                     |
| mysql_galera_hostgroups                            |
| mysql_group_replication_hostgroups                 |
| mysql_query_rules                                  |
| mysql_query_rules_fast_routing                     |
| mysql_replication_hostgroups                       |
| mysql_servers                                      |
| mysql_users                                        |
| proxysql_servers                                   |
| restapi_routes                                     |
| runtime_checksums_values                           |
| runtime_global_variables                           |
| runtime_mysql_aws_aurora_hostgroups                |
| runtime_mysql_firewall_whitelist_rules             |
| runtime_mysql_firewall_whitelist_sqli_fingerprints |
| runtime_mysql_firewall_whitelist_users             |
| runtime_mysql_galera_hostgroups                    |
| runtime_mysql_group_replication_hostgroups         |
| runtime_mysql_query_rules                          |
| runtime_mysql_query_rules_fast_routing             |
| runtime_mysql_replication_hostgroups               |
| runtime_mysql_servers                              |
| runtime_mysql_users                                |
| runtime_proxysql_servers                           |
| runtime_restapi_routes                             |
| runtime_scheduler                                  |
| scheduler                                          |
+----------------------------------------------------+
32 rows in set (0.00 sec)

admin-mysql_ifaces

The admin-mysql_ifaces variable specifies the listening address of the admin interface in the form of a colon-separated hostname:port list. The default listener is at 0.0.0.0:6032

Note that it is allowed to use the UNIX domain socket for monitoring, so that the applications in the host can be processed directly.
For example:

mysql> SET admin-mysql_ifaces='0.0.0.0:6032;/tmp/proxysql_admin.sock';
Query OK, 1 row affected (0.00 sec)

mysql> load admin variables to runtime;
Query OK, 0 rows affected (0.00 sec)

mysql> load admin variables to runtime;
Query OK, 0 rows affected (0.00 sec)

read-write separation

The most basic function of database middleware is to achieve read-write separation, and of course ProxySQL also supports it. ProxySQL can selectively route various queries very efficiently and flexibly. Routing configuration rules are as follows:

1. Configure port-based read/write separation

Port-based separation of read and write has great limitations. Applications need built-in read and write capabilities to distinguish between read and write, which is basically the same as the current business situation.

2. Use regular expressions to achieve basic read/write separation

Create corresponding routing rules in the routing table to realize the separation of reading and writing that you want. For example, all selects go to the slave database, select for update goes to the master database, all other queries go to the master database, and so on.

3. Use regular expressions and digest to configure read-write separation

Send only those queries that you want to send to slaves/readers, not just by SELECT statement. All requests go to the main database, find the most expensive select statement through the stats_mysql_query_digest table in ProxySQL, and then decide to route these expensive statements to the responsible read node, and start configuring the mysql_query_rules rule table.

example:

environmental information

host IP	Role	hosting service
192.168.197.141	Database Proxy Host	proxysql
192.168.197.150	Master database master	mysql
192.168.197.154	from database slave1	mysql
192.168.197.155	from database slave2	mysql

deployment steps

• Configure database master-slave

## 全部安装mysql或者mairadb服务
 
[root@localhost ~]# yum install -y mariadb*
......
 
[root@localhost ~]# systemctl start mariadb
[root@localhost ~]# ss -antl
State  Recv-Q Send-Q   Local Address:Port   Peer Address:Port 
LISTEN 0      80             0.0.0.0:3306        0.0.0.0:*    
LISTEN 0      128            0.0.0.0:22          0.0.0.0:*    
LISTEN 0      128               [::]:22             [::]:* 
 
 
## master端创建slave主从用户
MariaDB [mysql]> grant replication slave ON *.* TO 'repli'@'192.168.197.154' identified by '123456';
Query OK, 0 rows affected (0.000 sec)
 
MariaDB [mysql]> grant replication slave ON *.* TO 'repli'@'192.168.197.155' identified by '123456';
Query OK, 0 rows affected (0.000 sec)
 
MariaDB [mysql]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
 
 
## 两个slave测试登录数据库
[root@slave1 ~]# mysql -urepli -p123456 -h192.168.197.150
......
MariaDB [(none)]>
 
[root@slave2 ~]# mysql -urepli -p123456 -h192.168.197.150
......
MariaDB [(none)]>
 
 
## 开启master端二进制日志
[root@master ~]#vim /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=10
#
......
 
 
## 查看master节点位置
MariaDB [(none)]> show master status;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 |      328 |              |                  |
+------------------+----------+--------------+------------------+
1 row in set (0.000 sec)
 
 
 
## 开启slave中继日志
[root@slave1 ~]# vim /etc/my.cnf
[mysqld]
server-id=20
relay-log=mysql-relay-bin
......
 
[root@slave2 ~]# vim /etc/my.cnf
[mysqld]
server-id=30                           
relay-log=mysql-relay-bin
 
 
## 两个slave重启服务
[root@slave1 ~]# systemctl restart mariadb
[root@slave2 ~]# systemctl restart mariadb

Note: Regardless of the master or slave, the server-id must be unique, and the id on the slave server must be greater than the id on the master server

## 配置并启动主从复制
[root@slave1 ~]# mysql
......
MariaDB [(none)]> change master to
    -> master_host='192.168.197.150',
    -> master_user='repli',
    -> master_password='123456',
    -> master_log_file='mysql-bin.000001',
    -> master_log_pos=996;
 
MariaDB [(none)]> change master to master_host='192.168.197.150', master_user='repli', master_password='123456', master_log_file='mysql-bin.000001', master_log_pos=996;
Query OK, 0 rows affected (0.005 sec)
 
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.001 sec)
 
MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
                Slave_IO_State: Waiting for master to send event
                   Master_Host: 192.168.197.150
                   Master_User: repli
                   Master_Port: 3306
                 Connect_Retry: 60
               Master_Log_File: mysql-bin.000001
           Read_Master_Log_Pos: 996
                Relay_Log_File: mysql-relay-bin.000002
                 Relay_Log_Pos: 555
         Relay_Master_Log_File: mysql-bin.000001
              Slave_IO_Running: Yes
             Slave_SQL_Running: Yes
......
 
 
 
 
[root@slave2 ~]# mysql
......
MariaDB [(none)]> change master to
    -> master_host='192.168.197.150',
    -> master_user='repli',
    -> master_password='123456',
    -> master_log_file='mysql-bin.000001',
    -> master_log_pos=996;
 
MariaDB [(none)]> change master to master_host='192.168.197.150', master_user='repli', master_password='123456', master_log_file='mysql-bin.000001', master_log_pos=996;
Query OK, 0 rows affected (0.005 sec)
 
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.001 sec)
 
MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
                Slave_IO_State: Waiting for master to send event
                   Master_Host: 192.168.197.150
                   Master_User: repli
                   Master_Port: 3306
                 Connect_Retry: 60
               Master_Log_File: mysql-bin.000001
           Read_Master_Log_Pos: 996
                Relay_Log_File: mysql-relay-bin.000002
                 Relay_Log_Pos: 555
         Relay_Master_Log_File: mysql-bin.000001
              Slave_IO_Running: Yes
             Slave_SQL_Running: Yes
......

• test master-slave

## master创建新数据库
MariaDB [(none)]> create database test_db;
Query OK, 1 row affected (0.001 sec)
 
 
## slave1库查看
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test_db            |
+--------------------+
 
## slave2库查看
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test_db            |
+--------------------+
4 rows in set (0.001 sec)

• Create a read-write account for the proxysql host on the master

MariaDB [(none)]> grant all on *.* to 'proxysql'@'192.168.197.141' identified by '123456';
Query OK, 0 rows affected (0.001 sec)
 
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)

• proxysql login 6032 management terminal

[root@proxysql ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032;
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.30 (ProxySQL Admin Module)
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MySQL [(none)]> 
 

• Insert data into the mysql_servers table

 
## 首先查看mysql_servers表结构
MySQL [main]> pragma table_info("mysql_servers");
+-----+---------------------+--------------+---------+------------+----+
| cid | name                | type         | notnull | dflt_value | pk |
+-----+---------------------+--------------+---------+------------+----+
| 0   | hostgroup_id        | INT          | 1       | 0          | 1  |
| 1   | hostname            | VARCHAR      | 1       | NULL       | 2  |
| 2   | port                | INT          | 1       | 3306       | 3  |
| 3   | gtid_port           | INT          | 1       | 0          | 0  |
| 4   | status              | VARCHAR      | 1       | 'ONLINE'   | 0  |
| 5   | weight              | INT          | 1       | 1          | 0  |
| 6   | compression         | INT          | 1       | 0          | 0  |
| 7   | max_connections     | INT          | 1       | 1000       | 0  |
| 8   | max_replication_lag | INT          | 1       | 0          | 0  |
| 9   | use_ssl             | INT          | 1       | 0          | 0  |
| 10  | max_latency_ms      | INT UNSIGNED | 1       | 0          | 0  |
| 11  | comment             | VARCHAR      | 1       | ''         | 0  |
+-----+---------------------+--------------+---------+------------+----+
12 rows in set (0.000 sec)
 
## 插入数据
MySQL [main]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(10,'192.168.197.150',3306,1,'Write Group Master');
Query OK, 1 row affected (0.000 sec)
 
MySQL [main]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(20,'192.168.197.154',3306,1,'Read Group Slave1');
Query OK, 1 row affected (0.000 sec)
 
MySQL [main]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(30,'192.168.197.155',3306,1,'Read Group Slave2');
Query OK, 1 row affected (0.000 sec)
 
## 查看表内数据
MySQL [main]> select * from mysql_servers\G
*************************** 1. row ***************************
       hostgroup_id: 10
           hostname: 192.168.197.150
               port: 3306
          gtid_port: 0
             status: ONLINE
             weight: 1
        compression: 0
    max_connections: 1000
max_replication_lag: 0
            use_ssl: 0
     max_latency_ms: 0
            comment: Write Group Master
*************************** 2. row ***************************
       hostgroup_id: 20
           hostname: 192.168.197.154
               port: 3306
          gtid_port: 0
             status: ONLINE
             weight: 1
        compression: 0
    max_connections: 1000
max_replication_lag: 0
            use_ssl: 0
     max_latency_ms: 0
            comment: Read Group Slave1
*************************** 3. row ***************************
       hostgroup_id: 30
           hostname: 192.168.197.155
               port: 3306
          gtid_port: 0
             status: ONLINE
             weight: 1
        compression: 0
    max_connections: 1000
max_replication_lag: 0
            use_ssl: 0
     max_latency_ms: 0
            comment: Read Group Slave2
3 rows in set (0.000 sec)
 
## 加载到runtime层，并持久保存到disk中
MySQL [main]> load mysql servers to run;
Query OK, 0 rows affected (0.004 sec)
 
MySQL [main]> save mysql servers to disk;
Query OK, 0 rows affected (0.121 sec)

• Add the administrator user previously created on the master to the mysql_users table

 
## 查看mysql_users表结构
MySQL [main]>  pragma table_info("mysql_users");
+-----+------------------------+---------+---------+------------+----+
| cid | name                   | type    | notnull | dflt_value | pk |
+-----+------------------------+---------+---------+------------+----+
| 0   | username               | VARCHAR | 1       | NULL       | 1  |
| 1   | password               | VARCHAR | 0       | NULL       | 0  |
| 2   | active                 | INT     | 1       | 1          | 0  |
| 3   | use_ssl                | INT     | 1       | 0          | 0  |
| 4   | default_hostgroup      | INT     | 1       | 0          | 0  |
| 5   | default_schema         | VARCHAR | 0       | NULL       | 0  |
| 6   | schema_locked          | INT     | 1       | 0          | 0  |
| 7   | transaction_persistent | INT     | 1       | 1          | 0  |
| 8   | fast_forward           | INT     | 1       | 0          | 0  |
| 9   | backend                | INT     | 1       | 1          | 2  |
| 10  | frontend               | INT     | 1       | 1          | 0  |
| 11  | max_connections        | INT     | 1       | 10000      | 0  |
| 12  | comment                | VARCHAR | 1       | ''         | 0  |
+-----+------------------------+---------+---------+------------+----+
 
## 添加用户
insert into mysql_users(username,password,default_hostgroup,transaction_persistent) values('proxysql','123456',10,1);
 
## 查看
MySQL [main]> select * from mysql_users\G
*************************** 1. row ***************************
              username: proxysql
              password: 123456
                active: 1
               use_ssl: 0
     default_hostgroup: 10
        default_schema: NULL
         schema_locked: 0
transaction_persistent: 1
          fast_forward: 0
               backend: 1
              frontend: 1
       max_connections: 10000
               comment: 
1 row in set (0.000 sec)

• Add read-only users for health checks on the master

MariaDB [(none)]> grant select on *.* to 'monitor'@'192.168.197.%' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.001 sec)
 
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

• Modify this user variable value on proxysql

MySQL [main]> select @@mysql-monitor_username;
+--------------------------+
| @@mysql-monitor_username |
+--------------------------+
| monitor                  |
+--------------------------+
1 row in set (0.001 sec)
 
MySQL [main]> select @@mysql-monitor_password;
+--------------------------+
| @@mysql-monitor_password |
+--------------------------+
| monitor                  |
+--------------------------+
1 row in set (0.001 sec)
 
## 修改monitor密码
MySQL [main]> set mysql-monitor_password='123456';
Query OK, 1 row affected (0.000 sec)
 
MySQL [main]> load mysql variables to run;
Query OK, 0 rows affected (0.001 sec)
 
## 加载，保存并查看
MySQL [main]> save mysql variables to disk;
Query OK, 137 rows affected (0.004 sec)
 
MySQL [main]> select @@mysql-monitor_password;
+--------------------------+
| @@mysql-monitor_password |
+--------------------------+
| 123456                   |
+--------------------------+
1 row in set (0.001 sec)

• Proxysql adds routing rules for read-write separation

MySQL [main]> pragma table_info("mysql_query_rules");
+-----+-----------------------+--------------+---------+------------+----+
| cid | name                  | type         | notnull | dflt_value | pk |
+-----+-----------------------+--------------+---------+------------+----+
| 0   | rule_id               | INTEGER      | 1       | NULL       | 1  |
| 1   | active                | INT          | 1       | 0          | 0  |
| 2   | username              | VARCHAR      | 0       | NULL       | 0  |
| 3   | schemaname            | VARCHAR      | 0       | NULL       | 0  |
| 4   | flagIN                | INT          | 1       | 0          | 0  |
| 5   | client_addr           | VARCHAR      | 0       | NULL       | 0  |
| 6   | proxy_addr            | VARCHAR      | 0       | NULL       | 0  |
| 7   | proxy_port            | INT          | 0       | NULL       | 0  |
| 8   | digest                | VARCHAR      | 0       | NULL       | 0  |
| 9   | match_digest          | VARCHAR      | 0       | NULL       | 0  |
| 10  | match_pattern         | VARCHAR      | 0       | NULL       | 0  |
| 11  | negate_match_pattern  | INT          | 1       | 0          | 0  |
| 12  | re_modifiers          | VARCHAR      | 0       | 'CASELESS' | 0  |
| 13  | flagOUT               | INT          | 0       | NULL       | 0  |
| 14  | replace_pattern       | VARCHAR      | 0       | NULL       | 0  |
| 15  | destination_hostgroup | INT          | 0       | NULL       | 0  |
| 16  | cache_ttl             | INT          | 0       | NULL       | 0  |
| 17  | cache_empty_result    | INT          | 0       | NULL       | 0  |
| 18  | cache_timeout         | INT          | 0       | NULL       | 0  |
| 19  | reconnect             | INT          | 0       | NULL       | 0  |
| 20  | timeout               | INT UNSIGNED | 0       | NULL       | 0  |
| 21  | retries               | INT          | 0       | NULL       | 0  |
| 22  | delay                 | INT UNSIGNED | 0       | NULL       | 0  |
| 23  | next_query_flagIN     | INT UNSIGNED | 0       | NULL       | 0  |
| 24  | mirror_flagOUT        | INT UNSIGNED | 0       | NULL       | 0  |
| 25  | mirror_hostgroup      | INT UNSIGNED | 0       | NULL       | 0  |
| 26  | error_msg             | VARCHAR      | 0       | NULL       | 0  |
| 27  | OK_msg                | VARCHAR      | 0       | NULL       | 0  |
| 28  | sticky_conn           | INT          | 0       | NULL       | 0  |
| 29  | multiplex             | INT          | 0       | NULL       | 0  |
| 30  | gtid_from_hostgroup   | INT UNSIGNED | 0       | NULL       | 0  |
| 31  | log                   | INT          | 0       | NULL       | 0  |
| 32  | apply                 | INT          | 1       | 0          | 0  |
| 33  | comment               | VARCHAR      | 0       | NULL       | 0  |
+-----+-----------------------+--------------+---------+------------+----+
34 rows in set (0.000 sec)

• Add routing rules (based on SQL statements)

MySQL [main]>  insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(10,1,'^(update|insert|delete)',10,1);
Query OK, 1 row affected (0.000 sec)
 
MySQL [main]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(15,1,'^select.*from.*test_table',20,1);
Query OK, 1 row affected (0.000 sec)
 
MySQL [main]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(20,1,'^(select|show|desc)',30,1);
Query OK, 1 row affected (0.000 sec)
 
 
 
## 加载并保存
MySQL [main]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.001 sec)
 
MySQL [main]> save mysql query rules to disk;
Query OK, 0 rows affected (0.357 sec)
 
## 查看表数据
MySQL [main]> select * from mysql_query_rules\G
*************************** 1. row ***************************
              rule_id: 10
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 0
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: ^(update|insert|delete)
        match_pattern: NULL
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: NULL
destination_hostgroup: 10
 ......
                apply: 1
              comment: NULL
*************************** 2. row ***************************
              rule_id: 15
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 0
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: ^select.*from.*test_table
        match_pattern: NULL
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: NULL
destination_hostgroup: 20
......
                apply: 1
              comment: NULL
*************************** 3. row ***************************
              rule_id: 20
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 0
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: ^(select|show|desc)
        match_pattern: NULL
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: NULL
destination_hostgroup: 30
......
                apply: 1
              comment: NULL
3 rows in set (0.000 sec)

• proxysql local login

[root@proxysql ~]# mysql -uproxysql -p123456 -h127.0.0.1 -P3306;
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 232
Server version: 10.3.17-MariaDB-log MariaDB Server
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MariaDB [(none)]> 

• Query the table test_table

MySQL [(none)]> select * from test_db.test_table;
Empty set (0.010 sec)

• The proxysql host enters port 6032 for management

[root@proxysql ~]# mysql -padmin -uadmin -h127.0.0.1 -P6032;
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.5.30 (ProxySQL Admin Module)
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MySQL [(none)]> select * from stats_mysql_query_digest\G      ## 此表用来统计路由信息
 
MySQL [(none)]> select * from stats_mysql_query_digest\G
*************************** 1. row ***************************
        hostgroup: 20                      ## hostgroup组为20，也就是slave1主机，正好对应了select规则
       schemaname: information_schema
         username: proxysql
   client_address: 
           digest: 0x23E7E13566F1C0BF
      digest_text: select * from test_db.test_table
       count_star: 1
       first_seen: 1609244618
        last_seen: 1609244618
         sum_time: 8958
         min_time: 8958
         max_time: 8958
sum_rows_affected: 0
    sum_rows_sent: 0
 
*************************** 3. row ***************************
        hostgroup: 30                        ## show 命令对应id为30的group，也就是slave2主机
       schemaname: information_schema
         username: proxysql
   client_address: 
           digest: 0x02033E45904D3DF0
      digest_text: show databases          
       count_star: 1
       first_seen: 1609244445
        last_seen: 1609244445
         sum_time: 7690
         min_time: 7690
         max_time: 7690
sum_rows_affected: 0
    sum_rows_sent: 4

Since the query test_table table must be executed by slave1, the first query is run by the host with hostgroup=20.
The three lookup commands show, select, and desc are uniformly handed over to slave2 for execution, so here hostgroup=30

• Create a new table, add data, and delete a data

[root@proxysql ~]# mysql -uproxysql -p123456 -h127.0.0.1 -P3306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.5.30 (ProxySQL)
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MySQL [(none)]> use test_db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
MySQL [test_db]> create table student (id int not null,name varchar(100) not null,age tinyint);
Query OK, 0 rows affected (0.020 sec)
 
 
## 添加一条数据
MySQL [test_db]> insert into student (id,name,age) value(1,'tom',20);
Query OK, 1 row affected (0.007 sec)
 
 
## 删除这条数据
## MySQL [test_db]> delete from student where id=1;
Query OK, 1 row affected (0.005 sec)

• Enter the admin control port to view the records

MySQL [(none)]> select * from stats_mysql_query_digest\G;
*************************** 9. row ***************************
        hostgroup: 10
       schemaname: test_db
         username: proxysql
   client_address: 
           digest: 0x25502B06F2A2F4A4
      digest_text: create table student (id int not null,name varchar(?) not null,age tinyint)
       count_star: 1
       first_seen: 1609245630
        last_seen: 1609245630
         sum_time: 18922
         min_time: 18922
         max_time: 18922
sum_rows_affected: 0
    sum_rows_sent: 0
 
*************************** 1. row ***************************
        hostgroup: 10
       schemaname: test_db
         username: proxysql
   client_address: 
           digest: 0x728E984EDA5CFAE6
      digest_text: insert into student (id,name,age) value(?,?,?)
       count_star: 1
       first_seen: 1609245834
        last_seen: 1609245834
         sum_time: 5892
         min_time: 5892
         max_time: 5892
sum_rows_affected: 1
    sum_rows_sent: 0
 
        hostgroup: 10
       schemaname: test_db
         username: proxysql
   client_address: 
           digest: 0xED5AABA3AB8F6C15
      digest_text: delete from student where id=?
       count_star: 1
       first_seen: 1609245917
        last_seen: 1609245917
         sum_time: 4328
         min_time: 4328
         max_time: 4328
sum_rows_affected: 1
    sum_rows_sent: 0
 

Here, since create, insert, and delete are all write operations, and the rule belongs to hostgroup=10, they all run on the master host, so as to achieve the effect of read-write separation

load balancing

The main function of load balancing is to make the load of each DB server roughly equal.

Proxy can select the connection selected by the current request through the current connection number of each DB, request time, reconnection times, DB system status (status, variables, etc.), DB machine status, etc.

Figure 1-2 Load balancing

As shown in the figure above, the current load balancing strategy is:

1. The basic principle is to select the one with the smallest current connection number/weight coefficient among all candidate DB servers for connection.

2. If the total weight is greater than 0 and the weight of the current DB server is equal to 0, skip this DB server.

3. If the current DB server reaches the maximum number of connections, skip this DB server

connection pool

When the client connection is closed, ProxySQL checks whether the connection to the DB server is reusable. If it is not reusable, the connection is disconnected. If it is reusable, the connection to the DB server is not disconnected, but the connection is put into the connection pool. , the next time the client connects to ProxySQL, ProxySQL obtains a connection from the connection pool for the client to use. If no connection is available in the connection pool, a new connection is established.

ProxySQL supports multiplexing, which is used to divide statements into multiple routes. Only when the multiplexing switch is turned on, read/write separation and routing according to rules can be carried out. But sometimes, some statements require routing to the same host group, or even the same node in the same host group. Multiplexing is automatically disabled in this case. With multiplexing disabled, statements are routed over the same connection. Such as: when activating a transaction, locking a table, etc.

Automatically remove the downed DB

The Monitor module is used to periodically check various indicators of the backend. Currently supports 4 types of indicators to check connect, ping, replication lag, and read only; it can update the mysql topology by itself, and perform fault handling and perception. ProxySQL automatically detects fault slaves and removes them in the following situations:

1. The delay exceeds the threshold

2. The ping delay exceeds the threshold

3. Network failure or downtime

4. The number of connections is full

Load configuration dynamically

ProxySQL has a management port, hot modification configuration (memory modification), that is, online modification. Only the modification of the three parameters mysql-interfaces, mysql-threads and mysql-stacksize needs to restart the instance.

Access control

Password encryption: For security reasons, the password is encrypted by the encryption function in the local configuration (sqlite), and the plaintext password is not displayed.

SQL filtering: It can be automatically added according to the execution frequency and execution time, and the frequency and time can be dynamically modified according to your own needs; it supports query routing, and can be assigned to which instance to execute for a certain statement .

Figure 1.3 Security authentication process

IP whitelist: user IP restriction, which limits the user's host address. This function provides the user with the function of restricting the ip of the user connected to ProxySQL; a global whitelist can be provided in the ProxySQL routing table, and each user can also configure their own user private whitelist.

Independent account: slave database traffic configuration, you can specify that a certain user can only access certain slave databases, or conversely, certain slave databases only allow certain users to access, which may affect the database in a more detailed manner resources are allocated.

query cache

A very common scenario is to identify those non-optimized SELECT statements that cause the database load to increase. The results of these non-optimized queries should be cached for a few seconds. Create matching rules in the routing table and set the cache duration.

ProxySQL cluster

The native cluster function is still in the experimental stage, that is, the isomorphism of multiple nodes that can be extended without external tools.

A few points to note in the above picture:

The load software layer can also use ProxySQL to load balance the ProxySQL cluster, because ProxySQL itself is a proxy, and it is a proxy specifically responsible for the MySQL protocol.
In the load balancing layer, to ensure the durability of the transaction, different statements in a transaction cannot be routed to different targets, that is to say, the target of the load is "stateful". Therefore, when using lvs/haproxy, you must find a way to ensure transaction persistence. The general method is to take one of multiple ProxySQL nodes as a logical write node (backup is possible), and haproxy/lvs will force all write statements to be routed to This is fixed on the ProxySQL node.
The ProxySQL cluster is proxyed through ProxySQL, because ProxySQL supports transaction persistence internally (the field transaction_persistent in the mysql_users table controls persistence).

source

Proxysql Introduction and Configuration
ProxySQL User Manual - Main Function Introduction
Mysql Middleware - ProxySQL