[The implementation mechanism of MySQL database transaction operations, master-slave replication, Redis database read-write separation, and master-slave synchronization]

The implementation mechanism of MySQL database transaction operation, master-slave replication, Redis database read-write separation, and master-slave synchronization

ACID and how to implement it

Atomicity, consistency, isolation and durability

• Transaction isolation is achieved through the database lock mechanism.

• The consistency of the transaction is guaranteed by the undo log: the undo log is a logical log that records the insert, update, and deltete operations of the transaction. During rollback, the reverse delete, update, and insert operations are performed to restore the data.

• The atomicity and durability of transactions are guaranteed by redo log: redo log is called redo log, which is a physical log. When a transaction is submitted, all logs of the transaction must first be written to the redo log for persistence. Considered complete.

Transaction isolation level:

Read uncommitted: When the current transaction reads, you can see uncommitted data from other transactions, which can easily cause dirty reads;

Read committed: When the current transaction is reading, you can see the data that has been submitted by other transactions. The data read multiple times in the current transaction may be different, which may easily cause non-repeatable reads;

Repeatable read: When the current transaction reads, a current read snapshot is generated. Each subsequent data read is read in the snapshot, but newly inserted data by other transactions can be read, which can easily cause phantom reads. .

Serialization: It has the best isolation effect among the four transaction isolation levels. It solves the problems of dirty reads, repeatable reads, and phantom reads. However, it has the worst effect. It changes the execution of transactions into sequential execution, which is different from the other three. Compared with the isolation level, it is equivalent to a single thread. The execution of the latter transaction must wait for the end of the previous transaction before execution.

MVCC multi-version concurrency control

By maintaining historical versions of data, the read consistency problem under concurrent access is solved.

READ COMMITTED generates a ReadView every time before reading data , so that you can ensure that you can read the data submitted by other transactions every time;

REPEATABLE READ generates a ReadView when reading data for the first time , thus ensuring that the results of subsequent reads are completely consistent.

MySQL database master-slave replication

1. Write master data and update binlog
2. The master creates a dump thread to push the binlog to the slave.
3. When the slave connects to the master, an IO thread will be created to receive the binlog and recorded in the relay log.
4. The slave then starts a sql thread to read the relay log event and execute it on the slave to complete the synchronization.
5. The slave records its own binglog

How to deal with master-slave synchronization delay

Reasons for master-slave synchronization delay

A server opens N links for clients to connect, so there will be large concurrent update operations. However, there is only one thread to read the binlog from the server. When a certain SQL is executed on the slave server, it takes a little longer. Or because a certain SQL needs to lock the table, there will be a large backlog of SQL on the master server and it will not be synchronized to the slave server. This leads to master-slave inconsistency, that is, master-slave delay.

Solution to master-slave synchronization delay

There are several common ways to solve master-slave replication delays:

1. The read operation after the write operation is designated to be sent to the database main server

For example, after the account registration is completed, the read operation of reading the account when logging in is also sent to the main database server. This method is strongly bound to the business and has a greater intrusion and impact on the business. If a new programmer does not know how to write code in this way, it will cause a bug.

2. Read the master again after failing to read from the slave.

This is what is usually called "secondary reading". Secondary reading is not bound to the business. It only needs to encapsulate the API accessed by the underlying database. The implementation cost is small. The disadvantage is that if there are many secondary readings, Reading will greatly increase the read operation pressure on the host. For example, if a hacker violently cracks an account, it will lead to a large number of secondary read operations. The host may not be able to withstand the pressure of read operations and collapse.

3. All read and write operations for key businesses are directed to the host, and read and write operations are separated for non-critical businesses.

For example, for a user management system, the registration + login business read and write operations all access the host. The user's introduction, love, level and other services can use read and write separation, because even if the user changes his self-introduction, the read and write operations can be separated. When I inquired, I saw that the self-introduction was still old. Compared with being unable to log in, the business impact was much smaller and tolerable.

besides:

1. Optimize the network environment: During master-slave replication, reduce the impact of network delay between the master and slave servers on database synchronization. You can consider optimizing the bandwidth of the connection between networks, increasing the hardware performance of the slave library, etc.

2. Increase the number of slave databases: Increasing the number of slave databases can increase the speed and reliability of data synchronization, while also reducing the burden on each slave database and improving the response speed of the slave database.

3. Adjust database-related parameters: You can adjust some related parameters in the MySQL database, such as adjusting binlog format, binlog buffer size, innodb_flush_log_at_trx_commit and other parameters, using semi-synchronous mode to speed up data synchronization.

4. Partitioned database: Divide the database into multiple areas, and each slave database only copies the data area it needs, which can effectively reduce queuing congestion, network transmission and other delay issues.

In summary, optimizing the network environment, increasing the number of slave databases, adjusting database-related parameters, partitioning the database and other methods can effectively reduce the delay of the MySQL master-slave replication mode.

Redis read and write separation

1.What is master-slave replication?

After the host data is updated, it is automatically synchronized to the Master/Slaver of the standby machine according to the configuration and policy. The Master is mainly for writing, and the Slave is mainly for reading.

Master-slave replication refers to copying data from one Redis server to other Redis servers. The former is called the master node (master) , and the latter is called the slave node (slave) . And data replication is one-way , only from the master node to the slave node. Redis master-slave replication supports master-slave synchronization and slave-slave synchronization . The latter is a new feature added in subsequent versions of Redis to reduce the synchronization burden on the master node.

What is the main function of copying from?

• Data redundancy: Master-slave replication implements hot backup of data, which is a data redundancy method in addition to persistence.
• Fault recovery: When a problem occurs on the master node, the slave node can provide services to achieve rapid fault recovery (actually a kind of service redundancy) .
• Load balancing: Based on master-slave replication, combined with read-write separation, the master node can provide write services and the slave nodes can provide read services (that is, when writing Redis data, the application connects to the master node, and when reading Redis data, the application connects to the slave node) , share the server load. Especially in the scenario of less writing and more reading, sharing the read load through multiple slave nodes can greatly increase the concurrency of the Redis server.
• Cornerstone of high availability: In addition to the above functions, master-slave replication is also the basis for the implementation of sentinels and clusters . Therefore, master-slave replication is the basis of Redis high availability.

2. Advantages of reading and writing separation

1. Separation of reading and writing, improved performance
2. Rapid disaster recovery

Why is Redis fast?

Redis is very fast. A single Redis can support hundreds of thousands of concurrencies per second. Compared with MySQL, the performance is ten times that of MySQL. There are several main reasons for the fast speed:

1. Completely based on memory operation
2. Using a single thread avoids the consumption caused by thread switching and race conditions.
3. Based on non-blocking IO multiplexing mechanism
4. C language implementation, optimized data structure, based on several basic data structures, redis has done a lot of optimization, and the performance is extremely high