Article Directory:
1, 2 replication process, three synchronous data, the whole amount of 4 copy, copy portions 5, 6 heartbeat, asynchronous replication
Replication Works
1. Copy process
Copy the following process steps:
1, commands from the node to perform slaveof 2, the node simply holds information slaveof command in the master node, and does not initiate replication 3 Now, discovery information has the master node from the timing of the task in the node, start using socket connection master node 4, after the connection is established, send a ping command, hoping to get pong command response, otherwise it will be heavy even 5, if the master node set permissions, you need permission to carry out verification; if the validation fails, replication termination. 6, by the privilege verifier, data synchronization, which is the longest operation , the master node will send all data to all the slave nodes. 7, when the current master node to synchronize data from the node, will complete the process of establishing replication. Next, the master node will continue to send the write command to the slave node, ensure data consistency from the master .
2. The data synchronization between
The above said replication process, a step which is "synchronous data collection", "data synchronization between 'this is now talking about.
redis synchronization has two commands:
sync and psync, the former is a synchronous command before 2.8 redis, which is the command redis 2.8 In order to optimize the design of the new sync. We will focus on 2.8 psync command.
psync command requires three components support:
1, the master node of each copied from the offset 2, 3 buffer backlog replicate master node, master node ID
Primary copy offset from each node:
1 involved in replication master to maintain its own copy are offset from the node.
2, the master node after processing the write command, the byte length of the command will be accumulated records, statistics masterreploffset indicators in the info replication.
3, the node reports per second copy of itself offset to the master node, the master node will thus offset from the saved copy of the node.
4, the node receiving the command from the master node will itself offset accumulated statistical information in the info replication.
5, by comparing the replication master node offset can be determined from the master node are the same data.
The master node replication backlog buffer:
A replication backlog is stored in the FIFO buffer of the master node of a fixed length queue. The default size of 1MB. 2, the queue at the slave connection is created. At this time the master node in response to the write command, the command will only be sent to the slave node also written in the copy buffer. 3, part of his role is to copy and copy commands for lost data recovery. You can see the relevant information through info replication.
Master node ID:
Each redis start that they will generate a 40-bit operation ID. 2, the main function operation ID is used to identify Redis nodes. If ip + port way, if the master node then restarted modified RDB / AOF data, then copied from the node based on the offset would be unsafe. So, when running id change, the full amount from the node will be copied. In other words, redis after the restart, the default node from the full amount of copy.
If you do not change the ID run it on reboot?
RDB can be reloaded by debug reload command and keep running the same ID. So as to effectively avoid unnecessary replication of the full amount. 2, his shortcomings are: debug reload command will block the current node Redis main thread, so for a large amount of data to the master node or nodes can not tolerate obstruction, need to be used with caution. General can solve this problem by failover mechanism.
psync use the command:
Command format psync{runId}{offset}
runid: Run id offset master node copied from the node: the current data offset from the copied node
psync execution process:
Flow Description:
Psync command sent from node to the master node, the target ID is runid master node, if no default is -1, offset is the offset from the copy stored in the node, if it is the first copy or -1.
The master node will decide to return results based on runid and offset:
1, if the reply + FULLRESYNC {runId} {offset}, then the node will trigger the entire amount from the replication process.
2. If the reply + CONTINUE, the node will trigger a partial copy.
3. If the reply + ERR, explained the master node does not support psync command of 2.8 will be used to perform a full sync replication volume.
Here, between the data synchronization to talk about, and the length is quite long. Mainly for introduction between psync related commands.
3. Copy the full amount
Copy to copy the full amount of the earliest ways Redis support, but also the main establishment must go through when copying from the first stage. Command triggers the full amount of replication is sync and psync. Said before, these two commands watershed version is 2.8, using the sync before 2.8 redis can only perform different full amount, while supporting full amount of synchronization and partial synchronization after 2.8.
Process is as follows:
Tell us about the figure steps:
1, the transmission command psync (spync? -1) 2, the master node returns FULLRESYNC 3 according to a command from the master node ID and node records offset 4, the master node bgsave RDB and saved locally to 5, the master node sends the file to the node RBD 6 , received documents from the RDB node and loaded into memory 7, the master node while receiving data from node to save new data to "copy the client buffer", when loaded from the RDB node is completed, and then sent in the past. (If the node is taking too long, will cause a buffer overflow, the last full amount of synchronization failure) 8, load RDB files emptied from the node data, if RDB file is large, this step is still time-consuming, if this time the Client Access , will lead to inconsistent data, you can use the configuration slave-server-stale-data closed . 9, after the completion of the node successfully loaded RBD, if opened AOF, will do bgrewriteaof immediately .
More bold part is the whole amount of synchronous time-consuming entire place.
note:
1、如过 RDB 文件大于 6GB，并且是千兆网卡，Redis 的默认超时机制（60 秒），会导致全量复制失败。可以通过调大 repl-timeout 参数来解决此问题。 2、Redis 虽然支持无盘复制，即直接通过网络发送给从节点，但功能不是很完善，生产环境慎用。
4.部分复制
当从节点正在复制主节点时，如果出现网络闪断和其他异常，从节点会让主节点补发丢失的命令数据，主节点只需要将复制缓冲区的数据发送到从节点就能够保证数据的一致性，相比较全量复制，成本小很多。
步骤如下：
1、当从节点出现网络中断，超过了 repl-timeout 时间，主节点就会中断复制连接。 2、主节点会将请求的数据写入到“复制积压缓冲区”，默认 1MB。 3、当从节点恢复，重新连接上主节点，从节点会将 offset 和主节点 id 发送到主节点 4、主节点校验后，如果偏移量的数后的数据在缓冲区中，就发送 cuntinue 响应 —— 表示可以进行部分复制 5、主节点将缓冲区的数据发送到从节点，保证主从复制进行正常状态。
5.心跳
主从节点在建立复制后，他们之间维护着长连接并彼此发送心跳命令。
心跳的关键机制如下：
1、中从都有心跳检测机制，各自模拟成对方的客户端进行通信，通过 client list 命令查看复制相关客户端信息，主节点的连接状态为 flags = M，从节点的连接状态是 flags = S。 2、主节点默认每隔 10 秒对从节点发送 ping 命令，可修改配置 repl-ping-slave-period 控制发送频率。 3、从节点在主线程每隔一秒发送 replconf ack{offset} 命令，给主节点上报自身当前的复制偏移量。 4、主节点收到 replconf 信息后，判断从节点超时时间，如果超过 repl-timeout 60 秒，则判断节点下线。
注意：为了降低主从延迟，一般把 redis 主从节点部署在相同的机房/同城机房，避免网络延迟带来的网络分区造成的心跳中断等情况。
6.异步复制
主节点不但负责数据读写，还负责把写命令同步给从节点，写命令的发送过程是异步完成，也就是说主节点处理完写命令后立即返回客户度，并不等待从节点复制完成。
异步复制的步骤很简单，如下：
1、主节点接受处理命令 2、主节点处理完后返回响应结果 3、对于修改命令，异步发送给从节点，从节点在主线程中执行复制的命令。
总结
本文主要分析了 Redis 的复制原理，包括复制过程，数据之间的同步，全量复制的流程，部分复制的流程，心跳设计，异步复制流程。
其中，可以看出，RDB 数据之间的同步非常耗时。所以，Redis 在 2.8 版本退出了类似增量复制的 psync 命令，当 Redis 主从直接发生了网络中断，不会进行全量复制，而是将数据放到缓冲区（默认 1MB）里，在通过主从之间各自维护复制 offset 来判断缓存区的数据是否溢出，如果没有溢出，只需要发送缓冲区数据即可，成本很小，反之，则要进行全量复制，因此，控制缓冲区大小非常的重要。
引用
《Redis 开发与运维》
好了，关于redis 主从复制的原理就介绍到这里 ，篇幅有限，难免疏漏。
来源：http://t.cn/EJ24L7D