Redis Replication Service from the main building and the principle of

Existing stand-alone issue

Redis deploy a single node on a machine, what are the problems?

• Machine failure, data loss.
• Capacity bottlenecks, a single machine is limited memory.
• Performance bottlenecks
• Unable to availability

For existing single problem, Redis also offers a variety of clustering solutions:

• Master-slave replication
• Sentinel mode
• Clusters

This blog post is mainly for the "master-slave replication" make a brief record.

Master-slave replication

The role of nodes into Master and Slave, master and slave.
Master is responsible for writing data, and synchronize data to the Slave.
You slave responsible for doing persistent backup data, and is responsible for the client to read the data.

Master-slave replication can be done:

• Separate read and write
• Disaster Recovery

Environment to build

Host 8001

Host closed AOF, as little as possible triggers RDB persistence, improve performance, persistence task to do it from the machine.

# 任意IP可访问
bind 0.0.0.0
# 端口
port 8001
# 守护进程运行
daemonize yes
# 单机器运行多个Redis服务必须指定不同pidfile
pidfile /var/run/redis_8001.pid
# 日志文件，端口号区分
logfile "8001.log"
# rdb文件
dbfilename dump8001.rdb
# 基础目录
dir /app/redis/8001/
# 访问密码设置
requirepass 123

Slave 8002

Redis service started three stand-alone cluster to simulate the actual needs three physical machines, IP port can be configured.

# 任意IP可访问
bind 0.0.0.0
# 端口
port 8002
# 守护进程运行
daemonize yes
# 单机器运行多个Redis服务必须指定不同pidfile
pidfile /var/run/redis_8002.pid
# 日志文件，端口号区分
logfile "8002.log"
# rdb文件
dbfilename dump8002.rdb
# 基础目录
dir /app/redis/8002/
# 启用AOF持久化
appendonly yes
# 主机IP、端口
replicaof 127.0.0.1 8001
# 主机访问密码
masterauth 123

Slave 8003

和从机8002配置一致，端口号区分。

Redis service started three loaded 8001 ~ 8003 different profiles, see the process to see if the services are started successfully.

Service starts normally it means to build a successful cluster, then start the test.

Master-slave replication test

1, are connected to different services

redis-cli -p 8001 auth 123 //主机访问需要密码
redis-cli -p 8002
redis-cli -p 8003

2, see Replication information for each node.

8001 master

slave 8002

Similar to 8003, do not show.

Master, Slave information is not a problem, to write data to test.

3, the write data to the Master, the Slave read data correctly.

127.0.0.1:8001> set name hello
OK

127.0.0.1:8002> get name
"hello"

127.0.0.1:8003> get name
"hello"

4, Slave attempts to write data anomalies.

127.0.0.1:8002> set name world
(error) READONLY You can't write against a read only replica.

不能对只读副本进行写操作。
Forget what version from the start, Redis default Slave is read-only, can not write, if you need to write to Slave, you can modify the configuration file:

slave-read-only no

但是强烈建议不要这么做，因为主从复制数据流是单向的，从机写入的数据并不会同步到其他从机和主机，会出现数据不一致的问题。

Replication

Redis master-slave replication divided into two types: Full replication, incremental replication.

Copy the whole amount

Usually the total amount of replication occurs in the initialization phase Slave, Slave node data is empty, and for data synchronization Master, Slave need to have all the data on the Master copy, the steps outlined below:

1. Slave connection Master, send SYNC command, data synchronization data.
2. Master接收到SYNC命名后，执行BGSAVE命令异步生成RDB文件并使用缓冲区记录此后执行的所有写命令。
3. 异步执行BGSAVE的期间，Master将自身的runId标识和offset偏移量发送给Slave保存。
4. Master执行完BGSAVE后，向Slave发送快照文件，并在发送期间继续记录被执行的写命令。
5. Slave收到快照文件后丢弃所有旧数据，载入收到的快照。
6. Master快照发送完毕后开始向Slave发送缓冲区中的写命令。
7. Slave完成对快照的载入，开始接收命令请求，并执行来自Master缓冲区的写命令，后面就是增量复制了。

增量复制

Master每执行一次写操作，就将命令发送一份给所有Slave，Slave将增量数据写入，完成和Master的增量同步。

有盘复制和无盘复制

Master将数据保存到RDB文件中，然后发送给Slave，如果Master磁盘空间有限，可以采用无盘复制。
无盘复制将由Master开启一个Socket直接将RDB发送给Slave，无盘复制要求网络状况非常好。

runId和Offset

runId是由Redis生成的节点唯一标识，Offset是偏移量。
可以通过info server和info replication查看：

Offset的作用：
Master与Slave主从复制期间，总有意外情况发生，比如网络问题导致Slave掉线，掉线的这一段时间Slave就丢失了数据，需要和Master重新同步，但是如果每次同步都全量复制，Master压力就太大了，为了减少同步开销，Master通过Slave的Offset偏移量来判断是否需要进行全量复制。

Master在做主从复制的期间，会将数据写入一份到缓冲区repl_back_buffer，该缓冲区默认配置为1MB大小，可以把它看做是一个队列，达到1MB最先写入的数据会被丢弃。

If the Slave Offset is calculated by determining the missing data in the 1MB buffer size, then the transmitter only needs to buffer the data to Master Slave, to make an incremental replication, Offset calculated if the lost data is larger than the buffer 1MB , it means that the need for the full amount of copy.

To put it plainly, the role of the Offset Master is to determine whether or not to make the full amount for the Slave replication used.

From the master copy points

• Redis uses asynchronous replication, does not block the service, when Master sends data to the Slave, you can still answer a write operation.
• Slave upon receiving Master data, you can still read the response with the old data.
• In order to improve write performance Master, usually turns off persistent feature of the Master, the Master may lead to the restart data is empty, then the master-slave replication may cause Slave clear data.

Feature

• A Master can have multiple Slave
• Only one Master a Slave
• Unidirectional data flow, Master >> Slave
• Separate read and write data from the disaster recovery machine

Shortcoming

• Master can only have one, write performance bottlenecks.
• Master data can not be written down later.
• Synchronization between the Master and Slave delay, busy, too many Slave node problem is more serious.
• Multiple copies of data, can not do distributed storage.