Due to the different security needs, database audit, but also many customers concerned about the topic. For RDS Mysql, we can audit login for each user, and operation information it? The answer is yes.
MySQL Enterprise Edition has this feature, is charged components. Community Edition Mysql, common audit plug-in is "MariaDB audit plug-ins" and Percona audit plug.
1. AWS community version of RDS Mysql, audit approach, using "MariaDB audit plug-in", using the method detailed in the official documentation. This article will not go into the setting method. Note that the following parameters are set.
https://docs.amazonaws.cn/AmazonRDS/latest/UserGuide/Appendix.MySQL.Options.AuditPlugin.html
Setting Options |
Valid values |
Defaults |
description |
SERVER_AUDIT_FILE_PATH |
/rdsdbdata/log/audit/ |
/rdsdbdata/log/audit/ |
Location of the log file. The log file contains SERVER_AUDIT_EVENTS activities specified in the record. For more information, see View and list database log files and MySQL database log files . |
SERVER_AUDIT_FILE_ROTATE_SIZE |
1–1000000000 |
1000000 |
Upon reaching this size, in bytes, it can lead to file rotation. For more information, see the log file size . |
SERVER_AUDIT_FILE_ROTATIONS |
0–100 |
9 |
To save the log rotation number. For more information, see the log file size and download the database log files . |
SERVER_AUDIT_EVENTS |
CONNECT、QUERY |
CONNECT、QUERY |
To type of activity recorded in the log. Audit plug-in installed MariaDB can log on their own. The CONNECT : record of success and failure of database connections and disconnect from the database. The QUERY : text database records for all queries running. TABLE : Table records affected by a query against the database to run queries. For MariaDB, support the CONNECT , the QUERY and TABLE . For MySQL, support CONNECT and the QUERY . |
SERVER_AUDIT_INCL_USERS |
A plurality of comma-separated values |
no |
It includes only the specified user activity. By default, it records all user activity. If SERVER_AUDIT_EXCL_USERS and SERVER_AUDIT_INCL_USERS both specified by a user, the user's activity is logged. |
SERVER_AUDIT_EXCL_USERS |
A plurality of comma-separated values |
no |
Excludes the specified user activity. By default, it records all user activity. If SERVER_AUDIT_EXCL_USERS and SERVER_AUDIT_INCL_USERS both specified by a user, the user's activity is logged. rdsadmin user queries the database once per second to check the health of the database. Depending on your other settings, this activity may cause your log file size to grow very quickly. If you do not record this activity, please rdsadmin user to the SERVER_AUDIT_EXCL_USERS list. note It will always be recorded for all users CONNECT activities, even if this option sets the specified user. |
SERVER_AUDIT_LOGGING |
ON |
ON |
Logging is active. The only valid value for the ON . Amazon RDS does not support disabling logging. If you want to disable logging, delete MariaDB audit plugin. For more information, see Delete MariaDB audit plugin . |
2. open the audit function, we can see in the console in the audit log.
3. Below, we look at, audit log in the end what is recorded information.
3.1 I record foreground operation is as follows:
3.1.1 using the administrator user login database
# mysql -hmysql-rds.cq7qaukj3smd.rds.cn-northwest-1.amazonaws.com.cn -uadmin -pxxxxx
3.1.2 Creating a user test2, and give some of the new
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES ON *.* TO 'test2'@'ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn' identified by 'xxxxx' WITH GRANT OPTION; Query OK, 0 rows affected, 1 warning (0.02 sec)
3.1.3 使用新建的用户连接数据库
# mysql -hmysql-rds.cq7qaukj3smd.rds.cn-northwest-1.amazonaws.com.cn -utest2 -pxxxxx
3.1.4 执行数据库切换，创建表等命令
mysql> use tests; mysql> create table xx as select * from liang; mysql> commit; -----注释：create 是DDL语句，默认提交，所以这里commit没有意义。
3.2 audit log如下，可以看到以下几点信息
3.2.1. 谁访问和执行的，来至于哪里？
--test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn
3.2.2. 执行的全部操作信息记录
20190726 15:10:01,ip-10-4-2-104,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,13,883,QUERY,,'GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES ON *.* TO \'test2\'@\'ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn\' IDENTIFIED WITH \'mysql_native_password\' AS \'*01A6717B58FF5C7EAFFF6CB7C96F7428EA65FE4C\' WITH GRANT OPTION',0 20190726 15:10:03,ip-10-4-2-104,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,13,0,DISCONNECT,,,0 ...... 20190726 15:10:10,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,0,CONNECT,,,0 20190726 15:10:10,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,893,QUERY,,'select @@version_comment limit 1',0 20190726 15:10:19,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,894,QUERY,,'SELECT DATABASE()',0 20190726 15:10:19,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,896,QUERY,tests,'show databases',0 20190726 15:10:19,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,897,QUERY,tests,'show tables',0 20190726 15:10:31,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,913,QUERY,tests,'create table as select * from liang',1064 ...... 20190726 15:10:39,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,922,QUERY,tests,'create table xx as select * from liang',0 20190726 15:10:43,ip-10-4-2-104,test2,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,14,923,QUERY,tests,'commit',0
4. 当然审计日志会记录数据库的全部操作，自然也就有一些我们不需要的信息参杂其中。例如RDS后台操作用户rdsadmin的全部操作记录。
突然意识到，audit log，也可以用来研究AWS RDS后面实现的一些自动化运维的原理。
20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,900,QUERY,,'SELECT 1',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,901,QUERY,,'SELECT 1',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,902,QUERY,,'SELECT 1',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,903,QUERY,,'SELECT count(*) from information_schema.TABLES WHERE TABLE_SCHEMA = \'mysql\' AND TABLE_NAME = \'rds_heartbeat2\'',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,904,QUERY,,'SELECT 1',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,905,QUERY,,'SELECT value FROM mysql.rds_heartbeat2',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,906,QUERY,,'SELECT 1',0 20190726 15:10:20,ip-10-4-2-104,rdsadmin,localhost,2,907,QUERY,,'SELECT @@GLOBAL.read_only',0 20190726 15:10:25,ip-10-4-2-104,rdsadmin,localhost,2,908,QUERY,,'SELECT 1',0 20190726 15:10:25,ip-10-4-2-104,rdsadmin,localhost,2,909,QUERY,,'SELECT NAME, VALUE FROM mysql.rds_configuration',0 20190726 15:10:25,ip-10-4-2-104,rdsadmin,localhost,2,910,QUERY,,'SELECT 1',0 20190726 15:10:25,ip-10-4-2-104,rdsadmin,localhost,2,911,QUERY,,'SELECT @@session.transaction_read_only',0 20190726 15:10:25,ip-10-4-2-104,rdsadmin,localhost,2,912,QUERY,,'PURGE BINARY LOGS TO \'mysql-bin-changelog.008941\'',0
5. 但是真的rdsadmin的操作信息，是不需要做审计的。我们可以考虑不记录rdsadmin用户的审计信息，只需修改“选项组”的参数SERVER_AUDIT_EXCL_USERS=rdsadmin即可
5.1 关闭之后的audit log输出：
20190727 14:24:54,ip-10-4-0-205,rdsadmin,localhost,19,0,CONNECT,,,0 20190727 14:24:54,ip-10-4-0-205,rdsadmin,localhost,19,0,DISCONNECT,,,0
---正常情况下，rdsadmin 用户会每秒查询一次数据库，以检查数据库的运行状况。不记录rdsadmin的信息之后，世界立即清净了许多
20190727 14:25:04,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2689,QUERY,,'create database user',0 20190727 14:25:06,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2690,QUERY,,'SELECT DATABASE()',0 20190727 14:25:06,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2692,QUERY,user,'show databases',0 20190727 14:25:06,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2693,QUERY,user,'show tables',0 20190727 14:25:26,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2715,QUERY,user,'create table xx (id int)',0 20190727 14:25:29,ip-10-4-0-205,admin,ec2-52-80-168-237.cn-north-1.compute.amazonaws.com.cn,18,2716,QUERY,user,'commit',0
6. 针对数据库的审计，加密等操作，在增加安全性的同时，也要付出性能损耗的成本。
下文是AWS官方blog，其中讨论到RDS Mysql在开启审计之后，还是有比较大的性能损耗的。
7. 开启audit之后，会有大量的audit log产生，但是RDS的日志会滚动删除，不能长久的保存。如果为了长时间审计的需求，将audit保存下来，还可以做更多的统计和分析。
通过RDS的Modify页面，将audit log输出到cloudwatch。
至此，RDS Mysql数据库审计功能如何使用已经了解了。后续，我会继续分享，如何利用audit作为数据源。
通过数据处理，汇总到数据集/湖 中，以进一步分析使用。