DBA team 360 cloud computing
Heroine declaration
The audit function of the database is mainly to record all kinds of operation behaviors of users on the database for future query, analysis and tracking of problems. This article mainly investigates some open source audit plug-ins, and introduces their installation and usage methods.
PS: rich first-line technology, a wide range of forms, all in " 3 60 cloud computing " point of concern Oh!
Audit overview
1.1
general log
MySQL itself has actually provided information about all the statements executed by the server, which is what we usually call the general log, which can be used as an audit log, but it has the following disadvantages:
a. Regardless of whether the executed sentence is executed correctly, it will be recorded, which will result in a large amount of invalid information, and subsequent filtering is not easy to operate;
b. When the concurrent access of the server is very large, the log record will have a certain impact on the IO production, so as to affect the performance of the server;
c. The log files are easy to quickly add, and there is no regular rotation function according to the date and size, which is not convenient for maintenance;
1.2
binlog
Another log is a binary log, which is commonly referred to as binlog. Since binlog records all modifications to server data, in theory, it is possible to audit all data records and records related to table structure changes. However, because binlog itself does not record connection usage information, if you pass the binlog audit, you need to pass an additional section to record the connection information, and use the assigned thread id to compare it with the thread id in the binlog. Perform correlation analysis to obtain information about the connection used, such as:
The process id of root@localhost is 18494655, and a table is created after connecting:
Then, we can parse the binlog and get the following information:
It can be seen that the thread id can be matched with the process id, and you can get the usage information, but how to save it, you need to keep the connection information for each initial connection. You can configure the init_connect parameters and create additional To record the connection information table, do this:
There is a problem:
a. All users need to have write permission on the access_log table, otherwise, there is no way to use the database;
b. The access_log table needs regular manual maintenance;
c. The connection information of the super user will not be recorded;
d. Due to the limitation of binlog's own log, it is impossible to audit some content that is not recorded in binlog.
In response to the insufficiency of the above cases, some open source audit plug-ins have appeared, which will be explained separately below, and can be used as appropriate according to audit needs! The following are some benchmark notes for this test:
a. Does not involve performance comparison, only for functional testing;
b. MariaDB and McAfee audit plug-in, the test operation is based on MySQL 5.7.25 version, MySQL 8.0 as the counterpart version is 8.0.18;
c. The Percona audit plug-in uses Percona MySQL as the basic MySQL environment, and the version is 8.0.19
MariaDB Audit Plugin
2.1
Supported versionMySQL 5.7 and below
2.2
installation
a. Install the plug-in:
b. Configuration file
c. MySQL 5.7 load plugin
d. MySQL 8.0 loading problem
2.3
Related parameters
2.4
Audit event
2.5
Example
a. Only audit connect ddl
b. Only audit connect dml
c. Only audit connect dcl
Percona Audit Plugin
3.1
Supported version
Percona MySQL branch
3.2
installation
a. Install the plugin
Comes with it, no additional installation is required
b. Configuration file
3.3
Related parameters
a. audit_log_format
OLD
NEW
JSON
{"audit_record":{"name":"Query","record":"4082_2020-05-07T03:28:39","timestamp":"2020-05-07T03:30:42Z","command_class":"insert","connection_id":"8","status":1136,"sqltext":"insert into t1 values(0)","user":"root[root]@localhost []","host":"localhost","os_user":"","ip":"","db":""}}
{"audit_record":{"name":"Query","record":"4083_2020-05-07T03:28:39","timestamp":"2020-05-07T03:30:50Z","command_class":"insert","connection_id":"8","status":0,"sqltext":"insert into t1 values(0,1)","user":"root[root]@localhost []","host":"localhost","os_user":"","ip":"","db":""}}
{"audit_record":{"name":"Query","record":"4084_2020-05-07T03:28:39","timestamp":"2020-05-07T03:30:52Z","command_class":"insert","connection_id":"8","status":0,"sqltext":"insert into t1 values(0, 2)","user":"root[root]@localhost []","host":"localhost","os_user":"","ip":"","db":""}}
CSV
"Query","49284_2014-08-27T10:47:11","2014-08-27T10:47:23 UTC","show_databases","37",0,"show databases","root[root]@localhost []","localhost","",""
b. audit_log_exclude(include)_commands
c. audit_log_strategy
ASYNCHRONOUS - (default) log using memory buffer, do not drop messages if buffer is full
PERFORMANCE - log using memory buffer, drop messages if buffer is full
SEMISYNCHRONOUS - log directly to file, do not flush and sync every event
SYNCHRONOUS - log directly to file, flush and sync every event
3.4
Example
Mcafee Audit Plugin
4.1
Supported version
MySQL 5.7 and below
4.2
installation
a. Install the plugin
b. Configuration file
c. MySQL 5.7 load plugin
d. MySQL 8.0 loading plugin
4.3
Related parameters
4.4
Example
{"msg-type":"activity","date":"1588835829430","thread-id":"7","query-id":"30","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"rows":"5","status":"0","cmd":"show_databases","objects":[{"db":"information_schema","name":"/home/vicigel/sandboxes/msb_5_7_25_1/tmp/#sql_3b49_0","obj_type":"TABLE"}],"query":"show databases"}
{"msg-type":"activity","date":"1588835829430","thread-id":"7","query-id":"31","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"status":"0","cmd":"show_tables","objects":[{"db":"information_schema","name":"/home/vicigel/sandboxes/msb_5_7_25_1/tmp/#sql_3b49_0","obj_type":"TABLE"}],"query":"show tables"}
{"msg-type":"activity","date":"1588835843868","thread-id":"7","query-id":"32","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"status":"0","cmd":"create_table","objects":[{"db":"test","name":"t1","obj_type":"TABLE"}],"query":"create table t1(id int primary key auto_increment, b int)"}
{"msg-type":"activity","date":"1588835856549","thread-id":"7","query-id":"33","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"rows":"2","status":"0","cmd":"insert","objects":[{"db":"test","name":"t1","obj_type":"TABLE"}],"query":"insert into t1 values(0,1),(2,3)"}
{"msg-type":"activity","date":"1588835864360","thread-id":"7","query-id":"34","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"rows":"1","status":"0","cmd":"delete","objects":[{"db":"test","name":"t1","obj_type":"TABLE"}],"query":"delete from t1 where id = 1"}
{"msg-type":"activity","date":"1588835865148","thread-id":"7","query-id":"35","user":"msandbox","priv_user":"msandbox","ip":"127.0.0.1","host":"localhost","connect_attrs":{"_os":"linux-glibc2.12","_client_name":"libmysql","_pid":"18280","_client_version":"5.7.25","_platform":"x86_64","program_name":"mysql"},"cmd":"Quit","query":"Quit"}
Summary and comparison
The following is a comparison of the above audit plugins from the following dimensions
5.1
Audit granularity
The Percona audit plugin can control multiple audits of different granularities through the parameter audit_log_exclude(include)_commands. The McAfee audit plugin can control the audit granularity through the parameter audit_record_cmds(audit_whitelist_cmds). The MariaDB audit plugin can only be controlled by audit events, so:
Percona audit plugin = McAfee audit plugin > MariaDB audit plugin
5.2
Audit format log
Percona audit plugin can control different log output formats through the parameter audit_log_format. Neither McAfee audit plugin nor MariaDB audit plugin have related parameter control, so:
Percona audit plugin > McAfee audit plugin = MariaDB audit plugin
5.3
Controllable performance impact
The Percona audit plugin can control different log flushing strategies through the audit_log_strategy parameter, which can be a good trade-off and compromise between performance and log integrity. The McAfee audit plugin can control each audit_json_file_sync log flushing through the audit_json_file_sync parameter. The MariaDB audit plugin has nothing to do with it. Parameter control, therefore:
Percona audit plugin > McAfee audit plugin > MariaDB audit plugin
From the comparison results, the Percona audit plugin is better than the McAfee audit plugin. If MySQL chooses the Percona branch, then the built-in audit program is undoubtedly the best, and the other McAfee audit plugin and MariaDB audit plugin Before actual use on the line, adequate testing should also be done.
related articles
https://mariadb.com/kb/en/mariadb-audit-plugin/
https://www.percona.com/doc/percona-server/LATEST/management/audit_log_plugin.html
https://github.com/mcafee/mysql-audit/wiki/Configuration