Article Directory
This article is published by the official account [Developing Pigeon]! Welcome to follow! ! !
Old Rules-Sister Town House:
One. copy
(I. Overview
In Redis, the user can execute the SLAVE OF command or set the slaveof option to make one server replicate another server. We call the replicated server the master server and the other server as the slave server. The databases of both the master and slave servers in the process of replication will store the same data. This phenomenon is called consistent database status. Redis 2.8 version updated the replication function, here we will introduce the old version and the new version of the replication function.
(2) Old version copy function
1 Overview
The old version of the copy function is divided into two operations: synchronization and command propagation.
(1) Synchronization
Update the database status of the slave server to the current database status of the main server.
(2) Command propagation
When the database status of the master server is modified, causing the database status of the master and slave servers to be inconsistent, let the databases of the master and slave servers return to a consistent state.
2. Synchronization
The synchronization operation is completed by sending a SYNC command to the main server. When the main server receives the command, it executes the BGSAVE command, generates an RDB file in the background, and uses a buffer to record all write commands executed from now on. When the BGSAVE command is executed, the master server sends the RDB file to the slave server, receives the RDB file from the server, updates its database state to the state when the master server executes BGSAVE, and then the master server will write all the commands in the buffer Sent to the slave server, the slave server executes these commands to update the database status.
3. Command Propagation
After synchronization, the consistent state of the master and slave servers is not static. Whenever the master server executes the write command sent by the client, the state of the master and slave servers will be inconsistent. Therefore, the master server will perform command propagation operations on the slave server, and continuously send the write commands executed by itself to the slave server for execution, so that the states of the two will be restored to the same.
4. The old version of the copy defect
The old version of the copy is divided into two situations: the first copy and re-copy after disconnection.
(1) First copy
The slave server has not replicated any master server before, or the master server this time is different from the last time.
(2) Re-copy after disconnection
The master-slave server in the command propagation stage interrupted replication due to network reasons, and the slave server connected to the master server through automatic reconnection and continued to replicate. In this case, the efficiency is very low, because if you reconnect the user, the slave server will send the SYNC command to the master server again, and the master server will send the RDB file containing all its data to the slave server. All of the data is sent to it, because the slave server already has some data before, so this operation is very inefficient.
(3) Copy function of the new version
1 Overview
The new version of Redis uses the PSYNC command to replace the SYNC command to perform synchronization operations. There are two modes of complete resynchronization and partial resynchronization. Full resynchronization is used for initial replication, and partial resynchronization is used for recopying after disconnection.
2. Realization
Partial resynchronization consists of three parts: the replication offset of the master-slave server, the replication backlog buffer of the master server, and the server's running ID.
(1) Copy offset
Each time the master server transmits N bytes of data to the slave server, it adds N to its copy offset. Similarly, the slave server also adds N after receiving it. This attribute is used to compare the master-slave server Consistency.
(2) Copy the backlog buffer
A fixed-length FIFO queue maintained by the master server, with a default size of 1MB. When the master server performs command propagation, it not only sends the write command to all slave servers, but also enqueues it into the replication backlog buffer, saving a part of the most recently propagated Write command. When the slave server reconnects, the slave server sends its copy offset offset to the master server through the PSYNC command. If the data after the offset still exists in the copy buffer, it will perform partial resynchronization; otherwise, it will complete Resynchronization operation.
The size of this area can be adjusted, and it can be adjusted as large as possible according to the disconnection and reconnection.
(3) Server operation ID
Each Redis server has its own running ID. When the slave server first replicates the master server, the master server will send its running ID to the slave server, and the slave server will save it. Later, when the slave server reconnects, the running ID will be sent to the currently connected master server. If they are consistent, a partial resynchronization will be attempted, otherwise a complete resynchronization will be performed.
(Four) PSYNC command
There are two ways to call this command. One is to start a new replication and send a PSYNC? -1 command to the master server to actively request a complete resynchronization; the other is that the slave server has already replicated a certain server, and the The PSYNC command will be sent when copying, runID is the last saved master server ID, and offset is the copy offset.
If the primary server returns +FULLRESYNC and the server's running ID and replication offset, complete resynchronization is performed;
If it returns +CONTINUE, it means that partial resynchronization is performed;
If it returns -ERR, it means that the version of the master server is lower than 2.8 and PSYNC cannot be recognized, and the slave server will send a SYNC command to perform a complete resynchronization.
(5) Realization of replication
By sending the SLAVEOF command to the slave server, the slave server can replicate the master server.
1. Set the address and port of the main server
After the client sends the SLAVEOF command to the slave server, the SLAVEOF command sets the IP and port of the master server to the masterhost attribute and masterport attribute of the server status.
2. Establish a socket connection
The slave server creates a socket connection to the master server according to the IP and port set by the command. If the connection is successful, the slave server associates this socket with a file event handler dedicated to processing copy operations.
After the master server accepts the socket connection, it will create a corresponding client state for the socket, and treat the slave server as a client.
3. Send PING command
After the slave server becomes the client of the master server, the first thing is to send a PING command to the master server. This command has two functions. It checks whether the socket read and write status is normal and whether the master server can process the command request normally. If a PONG reply is received from the server, it means the connection is normal, otherwise it is disconnected and reconnected.
4. Identity Verification
After receiving the PONG reply, decide whether to perform authentication, and send an AUTH command to the master server according to the masterauth option set by the slave server.
5. Send port information
The slave server sends its own listening port number to the master server. After the master server receives the command, it will record the port number in the listening attribute of the client status corresponding to the slave server.
6. Synchronization
The slave server sends a PSYNC command to the master server to perform a synchronization operation. After this, both the master and the slave server will become each other's client.
7. Command Propagation
The master server always sends the write command executed by itself to the slave server.
(6) Heartbeat detection
In the command propagation phase, the slave server sends the command REPLCONF ACK to the master server once a second by default. The offset is the current replication offset of the slave server, detects the connection status, and detects command loss.
1. Check the connection
By sending the INFO replication command to the master server, you can list the information of the slave server. The lag attribute indicates the delay, which should be between 0-1 seconds, otherwise it is a connection error.
2. Loss of detection command
If the command propagation between the master and the slave is lost due to a network failure, when the slave server sends a REPLCONF ACK to the master server, if the carried offset is inconsistent with the master server, the data is lost, so the master server will backlog from the replication Find the missing data in the buffer and resend it.
The REPLCONF ACK command and the copy backlog buffer are both new in Redis 2.8. For data consistency, please use version 2.8 and above.