What is the big affairs?
In short, those who run a long time, more data manipulation transactions
How to query large transaction?
To query execution time over 10 seconds of a transaction as an example:
select * from information_schema.innodb_trx where TIME_TO_SEC(timediff(now(),trx_started))>10Large transaction database will generally cause any problems?
Locking too much data, likely to cause a lot of deadlocks and lock out
When the cycle of resource dependence occurs between different transaction systems, transactions involved are waiting for the other transaction releases resources, which will lead to several transactions have entered an infinite wait state, such as the following scenario:
At this time, the transaction A waiting transaction B releases the lock id = row 2, and row transaction B releases the lock id = 1 in the waiting transaction a. Transaction A and Transaction B in each other's resources to wait for the release, is to enter deadlock state
First of all we know, there are two strategies can deal with the deadlock:
- Wait deadlock timeout. Timeout (innodb_lock_wait_timeout) default is 50s, this time can really be too long, but if it piecemeal, and this may affect the normal can eliminate deadlock
- Deadlock detection. The default configuration deadlock detection is turned on. Deadlock detection is whenever a transaction is locked, it is necessary to see it depends thread has not locked the others, and so on, and finally determine if there is a deadlock
But the deadlock detection there may be a problem:
if all transactions have to update the same row of the time. Each new thread is blocked, and will not be judged because of their added led to a deadlock, this is a time complexity is O (n) operation. Suppose there are 1000 concurrent threads to simultaneously update the same row, then the deadlock detection operation is one million this magnitude. Although the results of the final test is no deadlock, but consumes a lot of CPU resources during this period. So, you will see the CPU utilization is high, but it can not perform several transactions per second.
Rollback records take up a lot of storage space, a long time transaction rollback
In MySQL, in fact, each record while recording a rollback in the update is going to be. The latest values recorded by the rollback operation, the value of the previous state can be obtained.
Assuming a value of 3, 4 are sequentially changed from 1, there will be similar to the following log records to roll back inside.
The current value is 4, but this record in a query when a transaction initiated at different times have different read-view. As shown in seen in the view A, B, C inside, which is a record of the values 2,4, the same record can be multiple versions of the system, version of the database is more concurrency control (MVCC ). For read-view A, to give 1, the current value must be sequentially executes all rollback obtained .
At the same time you will find that even now there is another transaction is changed to 5 to 4, the transaction with read-view A, B, C corresponding transaction will not conflict.
You will ask, the log can not roll back the remains of it, when to delete it? The answer is only removed when not needed. In other words, the system will determine when there is no longer need to use these to roll back a transaction log, rollback log will be deleted .
When is it not necessary? It is when the system does not roll back log earlier than the read-view time, another way is submitted after these things.
Performing a long time, likely to cause a delay from the master
Because the master database transaction must wait until the completion of the implementation of written binlog, then pass by the library. So, if the statement is executed on a main library for 10 minutes, then the transaction will most likely result in a delay from the library 10 minutes
solution
Based on two-phase locking protocol
Two-phase locking protocol is what?
In the InnoDB transaction, the lock is in line only when needed plus, but not not necessary to release immediately, but to wait until the end of the transaction release
Based on two-phase locking protocol optimizations we can do:
If your transaction requires more than one row lock, the lock should most likely to cause conflict, most likely to affect the degree of concurrency lock put back as much as possible
Suppose you responsible for the realization of a movie ticket online trading business, customer A to buy movie tickets in the theater B. We simplify it, this business need to involve the following:
- A customer ticket prices deducted from the account balance;
- B account balances to the theater this movie fare increase; 3. Record a transaction log.
In other words, to complete this transaction, we need to update two records, and insert a record. Of course, in order to ensure atomic transaction, we want these three operations in one transaction. So, how would you arrange the order of these three statements in a transaction it?
Imagine if there is another customer to buy a ticket at the theater while C B, then part of the conflict of these two transactions is a 2 statement. Because they want to update the same theater account balance, you need to modify the same rows of data.
According to the two-phase locking protocol, no matter how you arrange the order of statements, row lock all operations are needed when the transaction was submitted released. So, if you put the statement in the last 2 arrangements, such as 3,1,2 in this order, then the lock time theater account balances row would least. This minimizes the lock wait between transactions, enhance the degree of concurrency.
Based Deadlock Detection
Want to solve the problem of deadlock detection then it can only control the amount of O (n), the cost of the same line when a small number of concurrent time deadlock detection will be very low
But the number of concurrent pretty good control:
- The number of clients in a distributed system is uncertain, so do not limit the client
- MySQL to do so in the end you need to modify the source code, but this will not be for non Daniel
- Reference ConcurrentHashMap JDK1.7 segmented design of the lock, the row of data into the logical data on multiple lines to reduce lock conflicts
wherein the third embodiment is of little meaning, followed by theater account, for example, account information may be a on a plurality of records, such as record 10, equal to the sum total accounts theater value of the 10 records. So each theater to give account plus the amount of time, in which randomly selected to add to a record. So each collision probability becomes 1/10, can reduce the number of lock wait, it reduces the CPU consumption deadlock detection.
This program appears to be lossless, but in fact, such programs need to do the detailed design based on business logic. If the account balance may be reduced, such as refund logic, then this time you need to consider when part of rows becomes 0 when the code must be special treatment.
Based on transaction isolation level
We know that MySQL's default transaction isolation level is repeatable read, when writing data at this isolation level have these questions:
- If there is an index (primary key index comprises a) when the condition as an index update data, there will be a gap lock, line lock, the next key lock problem, so some rows lock
- It will lock the entire table if there is no index, update data
But if the isolation level is changed to read these two questions submitted does not exist, and each write data line will lock
But at the same time, you have to resolve data and log inconsistencies that may arise, we need to set the format for the binlog row
About why you want to set binlog can refer to this article why should the MySQL binlog format changed to row
other
- Some read-only operations there is no need to open things up
- By SETMAX_EXECUTION_TIME command, to control the maximum time for each statement executed, to avoid accidental execution of a single statement too long
- Monitoring information_schema.Innodb_trx table, provided long transaction threshold value, it exceeds the alarm / or kill
- In the business functional testing phase requires the output of all general_log, log analysis found that behavioral problems in advance
- Innodb_undo_tablespaces set value, the undo log isolated separate table space. If there really lead to large transaction rollback segment is too large, easier to clean up after this setting.