SQL Audit Tool SQLE 2.2308.0 was released today. The following is a detailed interpretation of the new version of Release Notes.
The article is mainly divided into the following three parts:
1. SQLE project introduction
2. Introduction of the main functions of the new version
3. Complete Release information
1. SQLE project introduction
The SQLE of the Ecoson open source community is a SQL auditing tool for database users and managers, which supports multi-scenario auditing, supports standardized online processes, natively supports MySQL auditing, and has scalable database types. Currently supports various database rules 700+.
SQLE get
| type | address |
|---|---|
| Repository | https://github.com/actiontech/sqle |
| document | https://actiontech.github.io/sqle-docs/ |
| release news | https://github.com/actiontech/sqle/releases |
| Data audit plug-in development documentation | https://actiontech.github.io/sqle-docs/docs/dev-manual/plugins/howtodev |
| Comparison of Community Edition and Enterprise Edition Features | https://actiontech.github.io/sqle-docs/docs/support/compare |
| Online Experience - Community Edition | http://demo.sqle.actionsky.com super administrator: admin password: admin |
| Online Experience-Enterprise Edition | http://demo.sqle.actionsky.com:8889 User: admin Password: admin |
2. Introduction of the main functions of the new version
# Community Edition
1. Smart scan supports SQL file scanning
The platform has added scanning tasks, and currently supports scanning SQL files. With this scan task, users can discover potential security vulnerabilities before executing SQL in batches, improve database performance and database development quality. Here is a simple function trial:
1. Create a scan task, select SQL file as the task type.
2. Enter the terminal and execute the Scanner file. The execution statement is as follows:
./scannerd sqlfile -H127.0.0.1 -N"SQLfile" -P"10000" -J"default" -D/root/sqle/sqle/cmd/scannerd/scanners/mybatis/testdata/ -A"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcG4iOiJ0ZXN0MDcyNCIsImV4cCI6MTcyMTcyODU0NywibmFtZSI6ImFkbWluIn0.CojdTvBkr-iN_26tWI-Kr6mMnJt1NIDr-Y3v7m9ySuM"
-J, --project：说明扫描任务所在项目，例如“default”；
-H, --host string：指定SQLE主机所在地址；
-P, --port string：指定SQLE所在端口；
-N, --name string：指定扫描任务名称，scannerd会将获得的SQL传至指定的任务池中审核；
-D, --dir string：指定要扫描的XML文件路径
-A, --token string：输入扫描任务凭证token；
-K, --skip-sql-file-audit：只上传sql，不审核sql
-S, --skip-error-sql-file; 跳过无法解析的sql文件
3. Obtain the execution result.
4. Users can also obtain scan results and view audit suggestions in the scan task details.
2. Work order informants support matching by instance
The platform currently supports matching work order informants (members who perform the online operation) by instance, and optimizes the problem that all data sources in a work order can only be launched by the same person, and realizes more fine-grained authority control in the online link. If the informant is set to "match members with data source permissions" in the process template, in the ticket details, the members who have the online operation on the corresponding instance will be displayed in the place of the person to be informant.
Here is a simple function trial:
1. Add members u1 and u2, bind u1 with the DBA authority on the MySQL instance, and bind u2 with the DBA authority on the Oracle instance.
2. Modify the rule template and set the informant as "Match members with data source permissions".
3. Create a work order, which includes SQL operations on the MySQL instance and the Oracle instance. After the review is passed, check that the persons to be operated are members with the online permission on the instance.
4. Log in to the account u1 of the person to be operated, and you can perform online operations on the MySQL instance bound to u1, and have no right to perform online operations on the Oracle data source.
3. The IntelliJ IDEA plugin supports reviewing multiple MyBatis files at the same time
In the previously provided audit plug-in, when auditing MyBatis files that contain cross-namespace references, an error that the ID does not exist will be reported. In this issue, the IntelliJ IDEA plug-in has been improved. Now it supports all XML files in a one-time audit folder. As long as the definition and reference of variables are in the same folder, they can be successfully parsed. Here is a simple trial of the functionality:
As shown in the figure, there are two XML files under the com folder .
Right-click the com folder and select MyBatis Audit.
Get the audit results of the file.
# Enterprise Edition
1. Java applications can access SQLE at zero cost
In this issue, support for the Java probe method has been added to the scanning task, and Java applications can be connected to SQLE at zero cost. Developers and DBAs can grasp the SQL in the application and the corresponding audit results through task details and audit reports, and control the quality of SQL in the Java process. Let's take the SurveyKing application as an example to conduct a simple function trial:
1. Create an Application SQL Crawl Scan task.
Note: The added data source is the data source used by the application.
2. Start the Java application, enter the application directory, and execute the following commands (the following commands need to be edited and replaced locally, and then copied to the environment for execution).
SQLE_COLLECT_ENABLE=true \
SQLE_BASE_URL=http://10.186.64.175:10000 \
SQLE_TASK_NAME=probe \
SQLE_TASK_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcG4iOiJwcm9iZSIsImV4cCI6MTcyMzI2MTE4OSwibmFtZSI6ImFkbWluIn0.pkYBOYsQ_Z6KhaweWdE44hkkDG8Ka73AWaM9mdXthNc \
SQLE_EXCLUDE_JAVA_FILE=UserServiceImpl.java \
SQLE_LOG_FILE=/tmp/collect.log \
java -javaagent:/opt/surveyking/sql-agent-1.0-SNAPSHOT.jar \
-jar surveyking-v0.3.0-beta.4.jar \
--server.port=1991 \
--spring.datasource.url=jdbc:mysql://10.186.64.175:3306/jw \
--spring.datasource.username=root \
--spring.datasource.password=mysqlpass \
& >>/opt/surveyking/std.log
Note: The Java Agent package is customized for SQLE.
3. Enter the platform, you can see the SQL that has been executed by the current application.
4. Check the audit report. Through the audit report of the audit task, the user can know whether the applied SQL complies with the preset audit rules, so as to make adjustments in time.
2. Added support for the termination of the enterprise version plug-in
In this issue , the function of suspending online has been added to four types of data sources: Ocean Base For MySQL, TDSQL, TiDB, and Oracle. When the SQL on this type of data source is blocked online, the user can stop the online operation with one click on the work order interface, reducing the cost of manual intervention.
3. Support OceanBase For Oracle data source
In this issue, the OceanBase For Oracle data source plug-in is added. Users can supervise OceanBase For Oracle data sources and perform SQL auditing according to the provided distributed rules.
4. Add TDSQL distributed audit rules
In this issue, TDSQL distributed audit rules are added to meet distributed application scenarios. The rules are as follows:
-
Disable DELETE/UPDATE...LIMIT statements on sharded tables
-
JOIN between fragmented tables and non-fragmented tables is prohibited
-
Disallow UPDATE operations on shard keys
-
It is forbidden to reference and manipulate variables when querying sharded tables
-
Fragmented table INSERT operation, the inserted field must contain the fragmentation key
-
Fragmented table SELECT, DELETE, UPDATE operations, the condition field must contain the fragmentation key
-
Fragmented table SELECT, INSERT, DELETE, UPDATE operations, prohibit the use of functions on the fragmentation key
-
Fragmented tables are associated, and cross-database JOIN is prohibited
-
When sharding tables are associated, the associated field must be a SHARDKEY, and non-SHARDKEY associated fields are not allowed to do associated queries
3. Complete Release information
# Community Edition
hint:
-
[#1754] Upgrading the old version to 2.2308.0 requires structural changes to the SQLE metadata table
New features:
-
[#1690] Smart scan supports SQL file scanning
-
[#1746] IntelliJ IDEA plugin supports reviewing multiple MyBatis files at the same time
optimization:
-
[#1605] Work order informant is refined to match by data source
-
[#1729] When scannerd scans MyBatis and SQL files, it is recommended to output all audit results
-
[#1731] Relax the length limit of work order name and other resources
Bug fixes:
-
[#1725] Fix the problem of program panic when exporting work orders with deleted data sources
# Enterprise Edition
New features:
-
[#1718] Various types of data source plug-ins support abort online function
-
[#884] Support JAVA probe to collect JAVA application SQL
-
[#868] Add TDSQL distributed audit rules
optimization:
-
[#875] Some plugins are upgraded to V2 version
Bug fixes:
-
[#888] Fix the problem that the user cannot jump to the home page after logging in through OAuth 2.0
-
[#883] Fix the wrong definition of Swagger interface for custom interface
-
[#879] Repair DB2 library table metadata scanning, the problem of wrong review of table creation statement
-
[#875] Fix the problem that DB2 database table metadata scan fails