Transaction features
1. Atomicity: After the transaction starts, all operations are either done or not done, and it is impossible to stagnate in the middle.
2. Consistency: Before and after the transaction starts, the integrity constraints of the database are not destroyed. For example, if A transfers money to B, it is impossible for A to deduct the money, but B does not receive it.
3. Isolation: At the same time, only one transaction is allowed to request the same data, and different transactions do not interfere with each other. For example, A is withdrawing money from a bank card, and B cannot transfer money to this card until the process of withdrawing money from A is over.
4. Durability: After the transaction is completed, all updates to the database by the transaction will be saved to the database and cannot be rolled back.
Transaction Concurrency Issues
1. Dirty read: transaction A reads the data updated by transaction B, and then B rolls back the operation, then the data read by A is dirty data
2. Non-repeatable read: Transaction A reads the same data multiple times, and transaction B updates and commits the data during the multiple reads of transaction A, resulting in inconsistent results when transaction A reads the same data multiple times.
3. Phantom reading: System administrator A changed the grades of all students in the database from specific scores to ABCDE grades, but system administrator B inserted a record of specific scores at this time. When system administrator A finished the change, he found that There is also a record that has not been changed, as if a hallucination has occurred. This is called hallucination.
Summary: Non-repeatable reads and phantom reads are easily confused. Non-repeatable reads focus on modification, while phantom reads focus on additions or deletions. To solve the problem of non-repeatable read, you only need to lock the rows that meet the conditions, and to solve the phantom read, you need to lock the table
transaction isolation
Mysql default is "repeatable read", after serialization
| Transaction isolation level | dirty read | non-repeatable read | hallucinations |
| read-uncommitted | Yes | Yes | Yes |
| non-repeatable read (read-committed) | no | Yes | Yes |
| repeatable-read | no | no | Yes |
| serializable | no | no | no |
#查全局事务隔离级别
SELECT @@global.tx_isolation;
#查当前会话事务隔离级别
SELECT @@session.tx_isolation;
#查当前事务隔离级别
SELECT @@tx_isolation;
#设置全局隔离级别
set global transaction isolation level read committed;
#设置当前会话隔离级别
set session transaction isolation level read committed;Serialization is the highest isolation level and solves the problem of phantom reads by forcing transaction ordering so that they cannot conflict with each other. In short, it adds a shared lock on each read data row, at this level, can cause a lot of timeouts and lock contention.
Shared lock (Share) : The code name of the shared lock is S
MySQL log file system composition
1. The composition of the MySQL log file system
a. Error log: record the problems that occur when starting, running or stopping mysqld.
b. General log: record the established client connection and the executed statement.
c. Update log: a statement that records the changed data. This log is deprecated in MySQL 5.1.
d. Binary log: records all statements that change data. Also used for replication.
e. Slow query log: record all queries whose execution time exceeds long_query_time seconds or queries that do not use indexes.
f. Innodb log: innodb redo log
Binary log (binlog):
Contains all data that has updated data or has potentially updated data (such as a DELETE that does not match any rows)
Contains execution time information about each statement that updates the database (DML)
Does not contain statements that do not modify any data, enable this if required Option, you need to turn on the general log function
The main purpose is to restore the database as far as possible to the point of failure of the database, because the binary log contains all the updates made after the backup
Used to log all statements on the master replication server that will be sent to the slave server
Enable this option The database performance is reduced by 1%, but the integrity of the database is guaranteed. For important databases, it is worth trading performance for integrity. Some are similar to oracle turning on archive mode.
show variables like '%version%';
show variables like '%log_bin%'; //是否启用 binlog
show variables like '%binlog%'; //binlog 相关参数
show variables like '%datadir%'; //数据文件目录，默认日志存在该目录
#编辑my.cnf来设定binary log日志位置(注，配置二进制日志路径及文件名后，系统变量log_bin被自动置为on)
log_bin=/var/lib/mysql/binarylog/binlog
#如果在my.cnf里面只设置log_bin，但是不指定file_name，然后重启数据库。你会发现二进制日志文件名称为${hostname}-bin 这样的格式
#切换日志
show master status;
flush logs;
show master status;Every time the MySQL service is restarted, a new binary log file will be generated, which is equivalent to binary log switching. When switching the binary log, you will see these numbers keep incrementing. In addition, in addition to these binary log files, you will see that a DB-Server-bin.index file is also generated. This file stores a list of all binary log files, also known as the index of the binary files.
Binary log deletion can be manually deleted through commands, or automatic cleanup can be set.
show binary logs;
mysql> purge binary logs to 'DB-Server-bin.000002';
purge binary logs to xxx; 表示删除某个日志之前的所有二进制日志文件。这个命令会修改index中相关数据
purge binary logs before '2017-03-10 10:10:00'; 清除某个时间点以前的二进制日志文件。
purge master logs before date_sub( now( ), interval 7 day);清除7天前的二进制日志文件
reset master;清除所有的二进制日志文件（当前不存在主从复制关系）
show variables like 'expire_logs_days';我们也可以设置expire_logs_days参数，设置自动清理，其默认值为0,表示不启用过期自动删除功能，如果启用了自动清理功能，表示超出此天数的二进制日志文件将被自动删除，自动删除工作通常发生在MySQL启动时或FLUSH日志时。
set expire_logs_days=7;Binary log related parameters
1. The system variable log_bin_trust_function_creators, the default is OFF, this parameter will limit the creation of stored procedures, functions, and triggers.
2: The system variable sql_log_bin is used to control the opening or closing of the session-level binary log function. The default value is ON, which means that the binary log function is enabled.
3. The system variable binlog_cache_size indicates that a cache of binlog_cache_size is allocated for each client, and the default value is 32768. The prerequisite for the use of binary log cache is that the server uses an engine that supports transactions and the bin log function is enabled. It is a memory area designed by MySQL to improve the efficiency of binlog to temporarily cache binlog data in a short period of time. Generally speaking, if there are no large transactions in our database, and writes are not particularly frequent, 2MB to 4MB is a suitable choice. However, if our database has many large transactions or multi-transaction statements, and the write volume is relatively large, binlog_cache_size can be appropriately increased. At the same time, we can use binlog_cache_use and binlog_cache_disk_use to analyze whether the set binlog_cache_size is sufficient, and whether a large number of binlog_cache use temporary files (binlog_cache_disk_use) to cache due to insufficient memory size.
You can judge whether binlog_cache_size needs to be adjusted by checking Binlog_cache_disk_use and Binlog_cache_use.
4. The system variable max_binlog_cache_size is the maximum cache memory size that the binary log can use. When executing a multi-statement transaction, if max_binlog_cache_size is not large enough, the system may report the error "Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage".
5. System variable max_binlog_stmt_cache_size
max_binlog_cache_size is for transactional statements, and max_binlog_stmt_cache_size is for non-transactional statements. When we find that Binlog_cache_disk_use or Binlog_stmt_cache_disk_use is relatively large, we need to consider increasing the size of the cache
6. The system variable max_binlog_size represents the maximum value of the binary log, which is generally set to 512M or 1GB, but cannot exceed 1GB. This setting does not strictly control the size of the binary log, especially when the binary log is relatively close to each other and a relatively large transaction is encountered, in order to ensure the integrity of the transaction, it is impossible to switch the log. All SQL is logged into the current log until the end of the transaction.
7. The system variable binlog_checksum is used as the master-slave check of replication. NONE means no checksum is generated, and CRC-32 means that this algorithm is used for verification.
8. The system variable sync_binlog, this parameter is very important for the Mysql system. It not only affects the performance loss caused by the binary log file to MySQL, but also affects the integrity of the data in MySQL.
sync_binlog=0, when the transaction is committed, Mysql just writes the data in binlog_cache to the binlog file, but does not execute disk synchronization instructions such as fsync to notify the file system to flush the cache to disk, but let Filesystem decide when to come do sync. The default setting in MySQL is sync_binlog=0, that is, without any mandatory disk flushing instructions, this setting has the best performance, but also the greatest risk. Once the system crashes (Crash), all binary log information in the file system cache will be lost. This leads to the problem of incomplete data.
sync_binlog=n, after n transaction submissions, Mysql will execute a disk synchronization command such as fsync, and the file system will flush the Binlog file cache to disk.
Sync_binlog can be adjusted appropriately to obtain higher concurrency and performance at the expense of certain consistency.
9. The system variable binlog_format specifies the type of binary log. There are three values: STATEMENT, ROW, and MIXED. MySQL 5.7.6 defaulted to STATEMENT mode. After MySQL 5.7.7, the default is ROW mode. This parameter mainly affects master-slave replication.
SQL statement-based replication (statement-based replication, SBR),
row-based replication (RBR),
Mixed-based replication (MBR).
View binary log content
Method 1: Use the show binlog events method to obtain the current and specified binlog logs, which is not suitable for extracting a large number of logs.
SHOW BINLOG EVENTS[IN 'log_name'] [FROM pos] [LIMIT [offset,] row_count]
SHOW BINLOG EVENTS IN 'mysql-bin.000005' \G
Method 2: Use the mysqlbinlog command line to view the log content (suitable for batch extraction of logs).
system mysqlbinlog /var/lib/mysql/DB-Server-bin.000013;
mysqlbinlog /var/lib/mysql/DB-Server-bin.000013 > test.sql;
Type of binary log
Segment-based log format
binlog_format=STATEMENT
The sql statement that records the operation.
advantage:
The amount of log records is relatively small, saving disk and network I/O, and only the daily amount generated by modifying or inserting a record in ROW format is smaller than the log amount generated by the segment.
shortcoming:
Context information must be recorded to ensure that the execution of the statement on the slave server is the same as on the master server.
Certain functions such as UUID, non-deterministic functions such as USER() cannot be copied.
It may cause data inconsistency between the active and standby servers replicated by MySQL, thereby interrupting the replication link.
Display binlog format
show variables like 'binlog_format';
set session binlog_format=statement;
line-based log format
Modify my.ini binary format to binlog_format=ROW
Advantages of row: The row format can avoid the problem of master-slave inconsistency in MYSQL replication, and this format is officially recommended. The same sql statement modifies 10,000 pieces of data. Segment-based logging will only log this SQL statement. The row-based log will have 10,000 records, which record the modification of each row of data separately.
1. MySQL master-slave replication is more secure.
2. Data modification per row is more efficient than segment-based replication. If the wrong operation modifies the data in the database, and there is no backup to restore, we can analyze the binary log and reverse the data modification operation recorded in the log to achieve the purpose of restoring the data.
Disadvantage of row: large amount of logging
binlog_row_image=[full,minimal,noblob]
full : all modifications of the column are recorded; minimal : only the modified columns are recorded. noblob : If it is a text type or clob field, do not record these logs.
Use mysqlbinlog -vv ../data/mysql-bin.000005 to view detailed logs.
set session binlog_row_image=minimal
Hybrid log format:
binlog_format=MIXED
Features: According to the sql statement, the system decides to choose between the log segment and the row-based log format. The amount of data is determined by the executed SQL.
How to choose a binary format
建议binlog_formart =mixed or binlog_format=row; binlog_row_image=minimal;
Copy method:
1. SQL statement-based replication (SBR)
Advantages: generate fewer logs and save IDs for network transmission. It is not required that the table definitions of the master and slave databases are exactly the same.
More flexible than row-based replication.
Disadvantage: For non-deterministic events, the consistency of master-slave replication data cannot be guaranteed. For stored procedures, triggers
2. Row-Based Replication (RBR)
Advantages: Replication that can be applied to any SQL including non-deterministic functions, stored procedures, etc. Can reduce the use of database locks.
Disadvantages: The table structure of the master and slave databases is required to be the same, otherwise replication will be interrupted.
3. Copy the way it works
1. The master writes the changes to the binary log.
2. Read the master's binary log changes and write them to relay_log.
Logpoint-based replication, GTID-based replication.
3. Replay the log in relay_log from above.
SQL segment-based logging is re-execution of logged SQL on the slave.
Row-based logging applies modifications to data rows directly on the slave.