【SQL】sqladvisor

Enterprise 2023-07-11 18:32:11 views: null

Article Directory

overview

SQLAdvisor is a SQL optimization tool developed and maintained by the DBA team (Beijing) of the Technical Engineering Department of Meituan Dianping Company to analyze SQL and give index optimization suggestions. It is based on MySQL's native lexical analysis, combined with the analysis of where conditions, aggregation conditions, and multi-table Join relationships in SQL to give index optimization suggestions.

Main function: output SQL index optimization suggestions

The SQLAdvisor SQL optimization suggestion tool open sourced by Meituan only has a command line, so some netizens have developed a web version for this, saying goodbye to the command line.

GitHub address: https://github.com/zyw/sqladvisor-web

The Meituan SQL analysis tool used in the project is compiled on CentOS, so it is recommended to deploy it on CentOS.
The project is developed using Python's Flask framework.
Use the Python version that comes with CentOS, the version number is 2.7.5.

Architecture process

insert image description here

Features

  • Based on MySQL native lexical analysis, fully guarantee the performance, accuracy and stability of lexical analysis;
  • Support common SQL (Insert/Delete/Update/Select);
  • Support multi-table Join and automatically select the driving table logically;
  • Support aggregation conditions Order by and Group by;
  • Filter existing indexes on the table.

Installation and deployment

[admin@git-server opt]# pwd
/opt
[admin@git-server opt]#  yum install git
[admin@git-server opt]# git clone https://github.com/Meituan-Dianping/SQLAdvisor.git
[admin@git-server opt]# yum install cmake libaio-devel libffi-devel glib2 glib2-devel
[admin@git-server opt]# yum install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm
[admin@git-server opt]# yum install Percona-Server-shared-56
[admin@git-server opt]# cd /usr/lib64/
[admin@git-server opt]# ln -s libperconaserverclient_r.so.18 libperconaserverclient_r.so

[admin@git-server opt]# cmake -DBUILD_CONFIG=mysql_release -DCMAKE_BUILD_TYPE=debug -DCMAKE_INSTALL_PREFIX=/usr/local/sqlparser ./
[admin@git-server opt]# make && make install
[admin@git-server opt]# cd SQLAdvisor/sqladvisor
[admin@git-server sqladvisor]# cmake -DCMAKE_BUILD_TYPE=debug ./
[admin@git-server sqladvisor]# make

Generate a sqladvisor executable file under this path, which is what we want.

The command parameters are as follows:

[admin@git-server sqladvisor]# ./sqladvisor --help
Usage:
  sqladvisor [OPTION...] sqladvisor

SQL Advisor Summary

Help Options:
  -?, --help              Show help options

Application Options:
  -f, --defaults-file     sqls file
  -u, --username          username
  -p, --password          password
  -P, --port              port
  -h, --host              host
  -d, --dbname            database name
  -q, --sqls              sqls
  -v, --verbose           1:output logs 0:output nothing

use

help output

$ sqladvisor -- help
Usage:
sqladvisor [OPTION . . . ] sqladvisor
SQL Advisor Summary
Help Options:
- ? , -- help               Show help options
Application Options:
-f , --defaults- file     sqls file
-u , -- username          username
-p , -- password          password
-P , -- port              port
-h , -- host              host
-d , -- dbname             database name
-q , -- sqls              sqls
-v , -- verbose           1: output logs 0: output nothing

command line parameter call

$ sqladvisor - h xx  - P xx  - u xx - p 'xx' - d xx - q "sql" - v 1

Configuration file parameter call

$ cat sql .cnf
[sqladvisor ]
username= xx
password= xx
host= xx
port= xx
dbname= xx
sqls=sql1 ;sql2 ;sql3 . . . .

$ sqladvisor - f sql .cnf  - v 1

Test 1: Test the small table

[admin@git-server sqladvisor]#  ./sqladvisor -h 127.0.0.1 -P 3306 -u admin -p "test@6666" -d zixun3 -q "select * from  zx_addonarticle;" -v 1
2018-06-14 17:57:33 1050 [Note] 第1步: 对SQL解析优化之后得到的SQL:select `*` AS `*` from `zixun3`.`zx_addonarticle` 

2018-06-14 17:57:33 1050 [Note] 第2步：表zx_addonarticle 的SQL太逆天,没有优化建议 

2018-06-14 17:57:33 1050 [Note] 第3步: SQLAdvisor结束!

Of course, the above command can also be written into the configuration file, which can be operated by specifying the file through the -f parameter

[admin@git-server sqladvisor]# cat test01.cnf 
[sqladvisor]
username=admin
password=test@6666
host=127.0.0.1
port=3306
dbname=zixun3
sqls=select * from  zx_addonarticle;

[admin@git-server sqladvisor]# ./sqladvisor -f test01.cnf -v 1
2018-06-14 18:01:42 1198 [Note] 第1步: 对SQL解析优化之后得到的SQL:select `*` AS `*` from `zixun3`.`zx_addonarticle` 
2018-06-14 18:01:42 1198 [Note] 第2步：表zx_addonarticle 的SQL太逆天,没有优化建议 
2018-06-14 18:01:42 1198 [Note] 第3步: SQLAdvisor结束!

Test 2: There is an index test for large tables

MySQL [zhangyou]> show index from test_table01;;
+--------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table        | Non_unique | Key_name    | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| test_table01 |          0 | PRIMARY     |            1 | id          | A         |      120201 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          0 | day         |            1 | day         | A         |          10 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          0 | day         |            2 | planid      | A         |        1521 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          0 | day         |            3 | uid         | A         |      120201 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          0 | day         |            4 | siteid      | A         |      120201 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          0 | day         |            5 | zoneid      | A         |      120201 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          1 | idx_day_uid |            1 | day         | A         |         112 |     NULL | NULL   |      | BTREE      |         |               |
| test_table01 |          1 | idx_day_uid |            2 | uid         | A         |       10927 |     NULL | NULL   |      | BTREE      |         |               |
+--------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
11 rows in set (0.00 sec)

[admin@git-server sqladvisor]# vim test.cnf 
[sqladvisor]
username=admin
password=test@6666
host=127.0.0.1
port=3306
dbname=zhangyou
sqls=select * from test_table01 where day='2016-04-22' and uid=26';

[admin@git-server sqladvisor]# ./sqladvisor -f test.cnf -v 1
2018-06-14 18:08:24 1281 [Note] 第1步: 对SQL解析优化之后得到的SQL:select had some error 
2018-06-14 18:08:24 1281 [Note] 第2步：开始解析where中的条件:(`day` = '2016-04-22') 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 
2018-06-14 18:08:24 1281 [Note] show table status like 'test_table01' 
2018-06-14 18:08:24 1281 [Note] select count(*) from ( select `day` from `test_table01` FORCE INDEX( day ) order by day DESC,planid DESC,uid DESC,siteid DESC,zoneid DESC,adstypeid DESC limit 10000) `test_table01` where (`day` = '2016-04-22')  
2018-06-14 18:08:24 1281 [Note] 第3步：表test_table01的行数:120201,limit行数:10000,得到where条件中(`day` = '2016-04-22')的选择度:10000 
2018-06-14 18:08:24 1281 [Note] 第4步：开始解析where中的条件:(`uid` = 26) 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 
2018-06-14 18:08:24 1281 [Note] show table status like 'test_table01' 
2018-06-14 18:08:24 1281 [Note] select count(*) from ( select `uid` from `test_table01` FORCE INDEX( day ) order by day DESC,planid DESC,uid DESC,siteid DESC,zoneid DESC,adstypeid DESC limit 10000) `test_table01` where (`uid` = 26)  
2018-06-14 18:08:24 1281 [Note] 第5步：表test_table01的行数:120201,limit行数:10000,得到where条件中(`uid` = 26)的选择度:10000 
2018-06-14 18:08:24 1281 [Note] 第6步：开始验证 字段day是不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='day' and Seq_in_index = 1 
2018-06-14 18:08:24 1281 [Note] 第7步：字段day不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] 第8步：开始验证 字段day是不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='day' and Seq_in_index = 1 
2018-06-14 18:08:24 1281 [Note] 第9步：字段day不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] 第10步：开始验证表中是否已存在相关索引。表名:test_table01, 字段名:day, 在索引中的位置:1 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 where Column_name ='day' and Seq_in_index =1 
2018-06-14 18:08:24 1281 [Note] 第11步：开始验证 字段uid是不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='uid' and Seq_in_index = 1 
2018-06-14 18:08:24 1281 [Note] 第12步：字段uid不是主键。表名:test_table01 
2018-06-14 18:08:24 1281 [Note] 第13步：开始验证表中是否已存在相关索引。表名:test_table01, 字段名:uid, 在索引中的位置:2 
2018-06-14 18:08:24 1281 [Note] show index from test_table01 where Column_name ='uid' and Seq_in_index =2 
2018-06-14 18:08:24 1281 [Note] 第14步：索引(day,uid)已存在 
2018-06-14 18:08:24 1281 [Note] 第15步: SQLAdvisor结束!

Test 3: Test for large tables without indexes

[admin@git-server sqladvisor]# vim test.cnf 
[sqladvisor]
username=admin
password=test@6666
host=127.0.0.1
port=3306
dbname=zhangyou
sqls=select * from test_table01 where  zoneid=42 and views=198;

[admin@git-server sqladvisor]# ./sqladvisor -f test.cnf -v 1
2018-06-14 18:15:09 1348 [Note] 第1步: 对SQL解析优化之后得到的SQL:select `*` AS `*` from `zhangyou`.`test_table01` where ((`zoneid` = 42) and (`views` = 198)) 
2018-06-14 18:15:09 1348 [Note] 第2步：开始解析where中的条件:(`zoneid` = 42) 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 
2018-06-14 18:15:09 1348 [Note] show table status like 'test_table01' 
2018-06-14 18:15:09 1348 [Note] select count(*) from ( select `zoneid` from `test_table01` FORCE INDEX( day ) order by day DESC,planid DESC,uid DESC,siteid DESC,zoneid DESC,adstypeid DESC limit 10000) `test_table01` where (`zoneid` = 42)  
2018-06-14 18:15:09 1348 [Note] 第3步：表test_table01的行数:120201,limit行数:10000,得到where条件中(`zoneid` = 42)的选择度:10000 
2018-06-14 18:15:09 1348 [Note] 第4步：开始解析where中的条件:(`views` = 198) 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 
2018-06-14 18:15:09 1348 [Note] show table status like 'test_table01' 
2018-06-14 18:15:09 1348 [Note] select count(*) from ( select `views` from `test_table01` FORCE INDEX( day ) order by day DESC,planid DESC,uid DESC,siteid DESC,zoneid DESC,adstypeid DESC limit 10000) `test_table01` where (`views` = 198)  
2018-06-14 18:15:09 1348 [Note] 第5步：表test_table01的行数:120201,limit行数:10000,得到where条件中(`views` = 198)的选择度:2000 
2018-06-14 18:15:09 1348 [Note] 第6步：开始验证 字段zoneid是不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='zoneid' and Seq_in_index = 1 
2018-06-14 18:15:09 1348 [Note] 第7步：字段zoneid不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] 第8步：开始验证 字段zoneid是不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='zoneid' and Seq_in_index = 1 
2018-06-14 18:15:09 1348 [Note] 第9步：字段zoneid不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] 第10步：开始验证表中是否已存在相关索引。表名:test_table01, 字段名:zoneid, 在索引中的位置:1 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 where Column_name ='zoneid' and Seq_in_index =1 
2018-06-14 18:15:09 1348 [Note] 第11步：开始验证 字段views是不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 where Key_name = 'PRIMARY' and Column_name ='views' and Seq_in_index = 1 
2018-06-14 18:15:09 1348 [Note] 第12步：字段views不是主键。表名:test_table01 
2018-06-14 18:15:09 1348 [Note] 第13步：开始验证表中是否已存在相关索引。表名:test_table01, 字段名:views, 在索引中的位置:2 
2018-06-14 18:15:09 1348 [Note] show index from test_table01 where Column_name ='views' and Seq_in_index =2 
2018-06-14 18:15:09 1348 [Note] 第14步：开始输出表test_table01索引优化建议: 
~~*2018-06-14 18:15:09 1348 [Note] CreateIndexSQL：alter table testtable01 add index idxzoneidviews(zoneid,views) *~~
2018-06-14 18:15:09 1348 [Note] 第15步: SQLAdvisor结束!

As you can see, the suggestion to create an index is given at the end

Test 4: Simultaneous analysis of multiple SQLs:

[admin@git-server sqladvisor]# cat /opt/SQLAdvisor/sqladvisor/test.cnf 
[sqladvisor]
username=admin
password=test@6666
host=127.0.0.1
port=3306
dbname=zhangyou
sqls=select * from test_table01 where  zoneid=42 and views=198;select * from test_table01 where day='2016-04-22' and uid=26';
[admin@git-server sqladvisor]#

source

SQL index optimization tool SQLAdvisor introduces the original
SQL optimization tool SQLAdvisor uses
SQLAdvisor

Guess you like

Origin blog.csdn.net/weixin_44231544/article/details/131170617
SQL
Recommended
Ranking
Jcseg 2.5.0 release, Java open source lightweight natural language processing package
[Principle] Process scheduling algorithm
React routing lazy loading lazy, Suspense and Vite's Glob import file
Create a Django project under pycharm Windows virtual environment [interpreter]
flink pass parameter to function
What should I do if the mac mouse cursor disappears?
gym 101810 M. Greedy Pirate (LCA)
Springboot inherits log4j2 perfect practice
[vue] 路由器的两种工作模式
Richard Matthew Stallman
Daily
More
2024-06-16(0)
2024-06-15(0)
2024-06-14(0)
2024-06-13(0)
2024-06-12(0)
2024-06-11(0)
2024-06-10(0)
2024-06-09(0)
2024-06-08(0)
2024-06-07(0)

Code World

© 2018 codetd.com