postgres csv log and view user permissions
Recently two problems encountered when using postgres, what better way recording of information found.
Suspect postgres SQL error in the implementation of the program clear whether the log exception information. Csv be determined by looking at the log whether SQL is really an error when executed.
Transfer from the following: https://www.cnblogs.com/kuang17/p/6902122.html?utm_source=itdadao&utm_medium=referral
Audit log
Audit is the value of recording the user's login and exit after landing conduct operations in the database, you can set different levels according to audit the security level is not the same,
Here parameter files involved are:
logging_collector - whether to open the log collection switch, default off, turn on to restart DB
log_destination - logging type, the default is stderr, log only error output
log_directory - log path, the default is $ PGDATA / pg_log, this directory is best not to catalog and data files together, the directory needs to start the operating system user postgres write permissions.
log_filename - log name, default postgresql-% Y-% m-% d_% H% M% S.log
log_file_mode - Log file type, the default is 0600
log_truncate_on_rotation - defaults to off, set to on, and if a new log file of the same name, the original file will be empty, and then write the log, rather than appended.
log_rotation_age - retain the maximum duration of a single file, the default is 1d, there 1h, 1min, 1s, personally feel that is not practical
log_rotation_size - retain the maximum size of a single file, the default is 10MB
log_error_verbosity - defaults to default, verbose representation lengthy
log_connections - user session if written to the log landing, off by default
log_disconnections - whether written to the log when the user session exit, default off
[Recommended log configuration]
logging_collector = on
log_destination = 'csvlog'
log_truncate_on_rotation = on
log_connections = on
log_disconnections = on
log_error_verbosity = verbose
log_statement = ddl
log_min_duration_statement = 60s
log_checkpoints = on
log_lock_waits ＝ on
deadlock_timeout ＝ 1s
1. Record Level :
After the various operations of a database for recording user login, log Postgres divided into 3 categories, pg_statement controlled by parameters, the default parameter values pg_statement is none, that is not recorded, it may be provided DDL (recording create, drop and alter), mod ( recording ddl + insert, delete, update, and TRUNCATE) and all (mod + select).
log_statement = ddl
General OLTP system audit level is set to ddl enough, because the record output of the impact of various SQL performance is still quite large, a little higher level of security can also be set mod mode, conditions can also do not in the database level, but purchase of equipment on the network layer monitor resolution.
Configure the audit level in different situations:
1) audit all users connected to the SQL database hm hm's.
alter role hm in database hm set log_statement='all';
2) audit of all SQL users of hm.
alter role hm set log_statement='all';
3) any audit of all SQL users to connect to the database hm.
alter database hm set log_statement='all';
2. Positioning slow query SQL
You can set a certain duration parameters (log_min_duration_statement), to record all SQL exceeds the length of time, it is effective to identify the slow query the current database. For example log_min_duration_statement = 2s, recording of more than 2 seconds SQL, the need to change the complete reload
log_min_duration_statement = 2s
3. Monitoring database checkpoint
When the database is a large update operation, if the parameter is set properly, it will leave in the log a lot of alarm information, do checkpoint frequent cause system slow down, there will not set.
But the system does not record the normal checkpoint, if you want to see how many times the checkpoint system one day, some of the details of the class and every checkpoint occurred, such as buffer, sync, etc., you can set log_checkpoints, the parameter default is off .
log_checkpoints = on
4. Lock monitoring database
Lock the database can usually find a table in pg_locks this system, but this is only the current lock table / row information, if you have a day to see how many times the deadlock over the lock occurs, you can set up and view the log , log_lock_waits The default is off, you can set to open. This distinction can be slow SQL resource constraints or lock wait problem.
log_lock_waits＝on
deadlock_timeout＝1
Some Debug feature, modify the source code for debugging, it does not require a general system of being more concerned about these.
The table may be provided on the trigger to audit
6. You can use the plug pg_log_userqueries do not necessary
【Precautions】
1. Super Users can modify these configuration items, so the user is best to be audited ordinary users. Otherwise, users can connect up to modify these audit items.
2. program for database accounts and personal accounts separate database for the database account with the program can only audit DDL operations, and database account for personal use, it is recommended to audit all SQL.
3. Variable priority (Transaction> Session> database / role> startup parameter> configuration file)
Some global variables can be dynamically modified, e.g. log_statement mentioned herein. After modifying the reload, all sessions will be reading the latest variable
Another problem is the wrong delete recreate shcema Times no superuser privileges. Pg used to determine a user's privileges \ du command to see.
Require superuser privileges operations have created extension, create shcema and so on.
Some commands to see. Is not actually used, the first record.
COPY weather FROM '/home/user/weather.txt';
bulk contents of the text file into the table wether
Modify user belonging group
Alter Group name Group add user username
Assign permissions for the group
to group group name grant operating On the table name
SCHEMA permissions for users to copy
grant all on SCHEMA scope name to the user name
View client connections
SELECT client_addr, client_port, waiting, query_start , current_query FROM pg_stat_activity;
Executing SQL queries
SELECT
procpid,
start,
now() - start AS lap,
current_query
FROM
(SELECT
backendid,
pg_stat_get_backend_pid(S.backendid) AS procpid,
pg_stat_get_backend_activity_start(S.backendid) AS start,
pg_stat_get_backend_activity(S.backendid) AS current_query
FROM
(SELECT pg_stat_get_backend_idset() AS backendid) AS S
) AS S
WHERE
current_query <> ''
ORDER BY
lap DESC;
Recently two problems encountered when using postgres, what better way recording of information found.
Suspect postgres SQL error in the implementation of the program clear whether the log exception information. Csv be determined by looking at the log whether SQL is really an error when executed.
Transfer from the following: https://www.cnblogs.com/kuang17/p/6902122.html?utm_source=itdadao&utm_medium=referral
Audit log
Audit is the value of recording the user's login and exit after landing conduct operations in the database, you can set different levels according to audit the security level is not the same,
Here parameter files involved are:
logging_collector - whether to open the log collection switch, default off, turn on to restart DB
log_destination - logging type, the default is stderr, log only error output
log_directory - log path, the default is $ PGDATA / pg_log, this directory is best not to catalog and data files together, the directory needs to start the operating system user postgres write permissions.
log_filename - log name, default postgresql-% Y-% m-% d_% H% M% S.log
log_file_mode - Log file type, the default is 0600
log_truncate_on_rotation - defaults to off, set to on, and if a new log file of the same name, the original file will be empty, and then write the log, rather than appended.
log_rotation_age - retain the maximum duration of a single file, the default is 1d, there 1h, 1min, 1s, personally feel that is not practical
log_rotation_size - retain the maximum size of a single file, the default is 10MB
log_error_verbosity - defaults to default, verbose representation lengthy
log_connections - user session if written to the log landing, off by default
log_disconnections - whether written to the log when the user session exit, default off
[Recommended log configuration]
logging_collector = on
log_destination = 'csvlog'
log_truncate_on_rotation = on
log_connections = on
log_disconnections = on
log_error_verbosity = verbose
log_statement = ddl
log_min_duration_statement = 60s
log_checkpoints = on
log_lock_waits ＝ on
deadlock_timeout ＝ 1s
1. Record Level :
After the various operations of a database for recording user login, log Postgres divided into 3 categories, pg_statement controlled by parameters, the default parameter values pg_statement is none, that is not recorded, it may be provided DDL (recording create, drop and alter), mod ( recording ddl + insert, delete, update, and TRUNCATE) and all (mod + select).
log_statement = ddl
General OLTP system audit level is set to ddl enough, because the record output of the impact of various SQL performance is still quite large, a little higher level of security can also be set mod mode, conditions can also do not in the database level, but purchase of equipment on the network layer monitor resolution.
Configure the audit level in different situations:
1) audit all users connected to the SQL database hm hm's.
alter role hm in database hm set log_statement='all';
2) audit of all SQL users of hm.
alter role hm set log_statement='all';
3) any audit of all SQL users to connect to the database hm.
alter database hm set log_statement='all';
2. Positioning slow query SQL
You can set a certain duration parameters (log_min_duration_statement), to record all SQL exceeds the length of time, it is effective to identify the slow query the current database. For example log_min_duration_statement = 2s, recording of more than 2 seconds SQL, the need to change the complete reload
log_min_duration_statement = 2s
3. Monitoring database checkpoint
When the database is a large update operation, if the parameter is set properly, it will leave in the log a lot of alarm information, do checkpoint frequent cause system slow down, there will not set.
But the system does not record the normal checkpoint, if you want to see how many times the checkpoint system one day, some of the details of the class and every checkpoint occurred, such as buffer, sync, etc., you can set log_checkpoints, the parameter default is off .
log_checkpoints = on
4. Lock monitoring database
Lock the database can usually find a table in pg_locks this system, but this is only the current lock table / row information, if you have a day to see how many times the deadlock over the lock occurs, you can set up and view the log , log_lock_waits The default is off, you can set to open. This distinction can be slow SQL resource constraints or lock wait problem.
log_lock_waits＝on
deadlock_timeout＝1
Some Debug feature, modify the source code for debugging, it does not require a general system of being more concerned about these.
The table may be provided on the trigger to audit
6. You can use the plug pg_log_userqueries do not necessary
【Precautions】
1. Super Users can modify these configuration items, so the user is best to be audited ordinary users. Otherwise, users can connect up to modify these audit items.
2. program for database accounts and personal accounts separate database for the database account with the program can only audit DDL operations, and database account for personal use, it is recommended to audit all SQL.
3. Variable priority (Transaction> Session> database / role> startup parameter> configuration file)
Some global variables can be dynamically modified, e.g. log_statement mentioned herein. After modifying the reload, all sessions will be reading the latest variable
Another problem is the wrong delete recreate shcema Times no superuser privileges. Pg used to determine a user's privileges \ du command to see.
Require superuser privileges operations have created extension, create shcema and so on.
Some commands to see. Is not actually used, the first record.
COPY weather FROM '/home/user/weather.txt';
bulk contents of the text file into the table wether
Modify user belonging group
Alter Group name Group add user username
Assign permissions for the group
to group group name grant operating On the table name
SCHEMA permissions for users to copy
grant all on SCHEMA scope name to the user name
View client connections
SELECT client_addr, client_port, waiting, query_start , current_query FROM pg_stat_activity;
Executing SQL queries
SELECT
procpid,
start,
now() - start AS lap,
current_query
FROM
(SELECT
backendid,
pg_stat_get_backend_pid(S.backendid) AS procpid,
pg_stat_get_backend_activity_start(S.backendid) AS start,
pg_stat_get_backend_activity(S.backendid) AS current_query
FROM
(SELECT pg_stat_get_backend_idset() AS backendid) AS S
) AS S
WHERE
current_query <> ''
ORDER BY
lap DESC;