mysql database audit (1)

1. Introduction and selection of database audit tools

1.1. Introduction to database audit tools

• The audit function of the MySQL branch is included in the enterprise edition, and the community edition can use the tools provided by other branches.
• Currently known audit tools, the community version includes Percona Server Audit Log of Percona , MariaDB Audit Plugin of MariaDB and mysql-audit of McAfee .     
• However, the compatibility of  McAfee 's plug-ins is poor. When auditing different versions of MySQL , it is necessary to execute scripts to obtain and specify offsets. It is released earlier and unmaintained, so it is not recommended.
• The MariaDB Audit  Plugin of the MariaDB version has better compatibility  with the MySQL branch, and the output is clear, and it can also record wrong operation commands for users. 
• Percona 's plug-in  Percona Server Audit Log is distributed with each version of Percona Server . The log can be specified in 4 formats and has many filtering rules.

1.2.  Selection of database audit tools

• MariaDB and  Percona plug-ins are released with the version, and both have plug-ins of version 5.7 . To choose which plug-in to choose, you need to compare the functional characteristics and pressure test performance.

2. Percona Audit Log

2.1. Download Percona Audit Log

• Percona Audit Log is distributed with Percona Server , so you need to download Percona Server MySQL .

2.2. Percona Audit Log installation

• Unzip Percona Server and upload the plugin

cp Percona-Server-5.7.28-31-Linux.x86_64.ssl1:111/lib/mysql/plugin/audit_log.so /usr/local/mysql/lib/plugin/


INSTALL PLUGIN audit_log SONAME 'audit_log.so';
ERROR 1126 (HY000): Can't open shared library '/usr/local/mysql/lib/plugin/percona_audit_log.so' (errno: 11 /usr/local/mysql/lib/plugin/percona_audit_log.so: undefined symbol: plugin_thdvar_safe_update)

• Error: undefined symbol: plugin_thdvar_safe_update
• Google it, there is a related topic: Percona Audit on Oracle MySQL EE: "undefined symbol: plugin_thdvar_safe_update" , there is a reply:
  • Plugin .so files from a specific packages are not supposed to work with other packages or third-party software.
    And in general our plugins are working only with Percona Server, not with Oracle MySQL.
  • Percona's official technical staff reply does not apply to third-party branches.

Another article gives a method to determine whether the current database is available ( Trying to install Audit_log Plugin on MySQL )

• nm /usr/local/mysql/bin/mysqld | grep plugin_thdvar_safe_update
• If the output is empty, the parameter is missing at compile time and cannot be used.  If you want to use it, you need to add the option to compile the plugin_thdvar_safe_update  parameter at compile time .

• The MySQL output is as follows

• The Percona Server output is as follows

2.3. The Percona Audit plug-in is not available in the Oracle MySQL branch, so choose the MariaDB branch audit plug-in.

3. MariaDB Audit Plugin

3.1. Introduction to MariaDB Audit Plugin

• The server_audit plugin records server activity. For each client session, it records who connected to the server (i.e. username and host), which queries were performed, which tables were accessed, and server variables were changed. This information is stored in a circular log file, or can be sent to the local syslogd .

3.2.  Installation of MariaDB Audit Plugin

3.2.1. Plug-in download

• The plug-in server_audit.so is included in the MariaDB branch program, download the 5.x version of MariaDB server , and you can find it after decompression.

3.2.2. Plug-in installation

• Execute the command to get the plugin directory path

SHOW GLOBAL VARIABLES LIKE 'plugin_dir'; +---------------+--------------------------+ | Variable_name | Value                    | +---------------+--------------------------+ | plugin_dir    | /usr/lib64/mysql/plugin/ | +---------------+--------------------------+

• Copy the plugin to the plugin directory

cp server_audit.so /usr/lib64/mysql/plugin/

• install plugin

INSTALL PLUGIN server_audit SONAME 'server_audit.so'; show plugins; +----------------------------+----------+--------------------+-----------------+---------+ | Name                       | Status   | Type               | Library         | License | +----------------------------+----------+--------------------+-----------------+---------+ |...                         |          |                    |                 |         | | SERVER_AUDIT               | ACTIVE   | AUDIT              | server_audit.so | GPL     | +----------------------------+----------+--------------------+-----------------+---------+

3.2.3. Plug-in configuration

• After the audit plugin is installed and loaded, there will be some new global variables in MariaDB .
• These can be used to configure many audit server related components, limits and methods. You can set these variables related to logs, such as their location, size limit, rotation parameters and method of log information.
• It is also possible to set the information to be logged, such as connection, disconnection and failed connection attempts.
• You can also use the audit plugin to log queries, read and write access to tables. In order not to overload your logs, audit plugins can be configured based on a list of users.
• Specific user activity can be included or excluded from the log.

SHOW GLOBAL VARIABLES LIKE 'server_audit%'; +-------------------------------+-----------------------+ | Variable_name                 | Value                 | +-------------------------------+-----------------------+ | server_audit_events           | CONNECT,QUERY,TABLE   | | server_audit_excl_users       |                       | | server_audit_file_path        | server_audit.log      | | server_audit_file_rotate_now  | OFF                   | | server_audit_file_rotate_size | 1000000               | | server_audit_file_rotations   | 9                     | | server_audit_incl_users       |                       | | server_audit_logging          | ON                    | | server_audit_mode             | 0                     | | server_audit_output_type      | file                  | | server_audit_query_log_limit  | 1024                  | | server_audit_syslog_facility  | LOG_USER              | | server_audit_syslog_ident     | mysql-server_auditing | | server_audit_syslog_info      |                       | | server_audit_syslog_priority  | LOG_INFO              | +-------------------------------+-----------------------+

•  开启插件，需要用户具有 SUPER 权限。

SET GLOBAL server_audit_logging=ON;

• 创建审计日志文件存放路径

# 在 mysql datadir 中创建 auditlog 目录 mkdir -p auditlog chown -R mysql:mysql auditlog chmod 750 auditlog

• 设置审计日志文件存放路径

set global server_audit_file_path='/home/mysql/mysql57_3306/auditlog/server_audit.log';

• 设置审计日志的相关参数

set global server_audit_events='CONNECT,QUERY,TABLE,QUERY_DDL,QUERY_DML,QUERY_DCL';
set global server_audit_output_type= file;
set global server_audit_logging = 1;
set global server_audit_file_rotate_size = 500000000;
set global server_audit_file_rotations = 3;
set global server_audit_incl_users = '';
set global server_audit_excl_users = '';
set global server_audit_query_log_limit = 4096;

3.2.4. 卸载

UNINSTALL PLUGIN server_audit;

• 防止被卸载，如果设置如下选项，需要注释配置文件后重启 MySQL 才可以卸载审计插件。

[mysqld] server_audit=FORCE_PLUS_PERMANENT

3.2.4. 配置文件参数

loose-server_audit_file_path='/data/mysql/mysql57_3306/auditlog/server_audit.log'
loose-server_audit_events='CONNECT,QUERY,TABLE,QUERY_DDL,QUERY_DML,QUERY_DCL'
loose-server_audit_output_type= file
loose-server_audit_logging = 1
loose-server_audit_file_rotate_size = 500000000
loose-server_audit_file_rotations = 3
loose-server_audit_incl_users = ''
loose-server_audit_excl_users = ''
loose-server_audit_query_log_limit = 4096

3.3. MariaDB Audit Plugin 的日志格式

• 审计插件记录用户对MariaDB及其对象的访问。审核跟踪（即审核日志）是一组记录，以纯文本格式写为文件的字段列表。
• 日志中的字段用逗号分隔。插件自己的日志文件使用的格式与它记录到系统日志时使用的格式略有不同，因为它具有自己的标准格式。日志记录到插件自己文件的一般格式如下：

[timestamp],[serverhost],[username],[host],[connectionid],[queryid],[operation],[database],[object],[retcode]

• 如果将 server_audit_output_type 变量设置为 syslog 而不是默认值 file，则审核日志文件格式将如下所示：

	日志项	描述
[timestamp][syslog_host][syslog_ident]:[syslog_info][serverhost],[username],[host],[connectionid],[queryid],[operation],[database],[object],[retcode]
	timestamp	事件发生的时间。如果使用syslog，则格式由 syslogd 定义
	syslog_host	接收系统日志条目的主机
	syslog_ident	用于标识系统日志条目，包括MariaDB服务器
	syslog_info	用于提供标识系统日志条目的信息
	serverhost	MariaDB服务器主机名
	username	连接的用户
	host	用户连接的主机
	connectionid	相关操作的连接标识号
	queryid	查询ID号，可用于查找关系表事件和相关查询。对于TABLE事件，将添加多行
	operation	Recorded action types: CONNECT, QUERY, READ, WRITE, CREATE, ALTER, RENAME, DROP
	database	active database (set by USE)
	object	Execute a query on the table name of the QUERY event or TABLE event
	retcode	Returns the code for the logged operation

• Various events will result in different audit records. Some events will not return a value for some fields (for example, when connecting to a server without an active database set).
• The following is a generic example of connection event output, where placeholders represent data. These are events where a user connects, disconnects, or attempts to connect to the server fail.