(1) Introduction to master-slave replication
The redis operations mentioned above are all stand-alone operations. Although stand-alone operations are simple to operate, they have limited processing capabilities and cannot be highly available. The so-called high availability means that when a server is down, there is a backup server that can replace it. This is impossible in stand-alone operation, so master-slave replication occurs.
We regard one server as the master server, and the other servers as slave servers. Master-slave replication is to copy the data in the master to the slave immediately and effectively.
Master-slave replication characteristics:
A master can have multiple slaves, and a slave corresponds to only one master
The master is responsible for writing data, and automatically synchronizes the changed data to the slave
The slave is responsible for reading data, and writing data is prohibited
With master-slave replication, high availability can be achieved. When one slave goes down, there are still multiple slaves; when the master goes down, since the data of the slave is the same as the master, one can be selected Slave is a new master and can be highly available.
The role of master-slave replication:
Read and write separation: master write, slave read
Load balancing: The slaves share the load of the master, and the number of slaves can be changed according to specific needs
Failure recovery: When there is a problem with the master, the slave can be used to replace the master to achieve rapid recovery
Data redundancy: With multiple slaves, data backup becomes easier
(2) Master-slave replication environment configuration
Environment configuration
The master-slave configuration can be viewed through the info command
>info replication # 查看当前库的信息
# Replication
role:master #角色 master
connected_slaves:0 # 从机个数
master_replid:8ff09770a496e8472fffef9f35ebdd1bc0b15ecb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0Copy three configuration files, named redis6379.conf, redis6380.conf, redis6381.conf and then modify the corresponding information
#端口
port 6379 #复制多个改为6380、6381
#pid名字
pidfile /var/run/redis_6379.pid #复制多个改为6380、6381
#log文件名字
logfile "6379.log" #复制多个改为6380、6381
#dump.rdb名字
dbfilename dump6379.rdb #复制多个改为6380、6381Start redis-server through the configuration file respectively
./redis-server ../redis6379.conf
./redis-server ../redis6380.conf
./redis-server ../redis6381.conf
One-master two-slave configuration
By default, each redis server is the master node: under normal circumstances only need to configure the slave
Here we set 79 as the master, 80 and 81 as the slave
redis-cli -p 6380
127.0.0.1:6380> slaveof 127.0.0.1 6379 #设置6379为主机
#在从机中查看信息
127.0.0.1:6380> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:28
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:f669a0ed63ed7f23841baa0c708b3d757d7ccc54
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:28
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:28
在主机中查看信息
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6380,state=online,offset=112,lag=0
master_replid:f669a0ed63ed7f23841baa0c708b3d757d7ccc54
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:112
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:112In the same way, set the host of 6381 to 6379. What is configured in this way is the temporary master-slave configuration, and the permanent master-slave configuration is configured in the configuration file.
replicaof <masterip> <masterport> #设置主机的ip和端口
masterauth <master-password> #如果主机有密码则设置密码After setting the master-slave configuration, after the master node writes, it can be read at the slave node, but cannot be written at the slave node
127.0.0.1:6379> set k1 v1
OK
127.0.0.1:6380> get k1
"v1"
127.0.0.1:6380> set k2 v2
(error) READONLY You can't write against a read only replica. However, if the slave is still connected to the original master after the master is connected, it is necessary to introduce a sentinel to solve this problem.
It should be noted that when the slave connects to the master for the first time, a full copy will be triggered, and subsequent writes from the master will trigger incremental replication.
(3) Master-slave replication workflow
The master-slave replication workflow can be divided into three stages:
1. Establishing the connection phase
3. Command propagation phase (repeated synchronization)
2.1 Establishing the connection phase
1. Set the address and port number of the master
2. Establish a socket connection
3. Send ping command (timer task)
2.2 Data synchronization stage
1. Request to synchronize data
2. Create RDB synchronization data
3. Recover RDB synchronization data
4. Request partial synchronization data
5. Recover part of the synchronized data
Just looking at the five steps like this may be a bit abstract, and it can be represented by a picture:
Data synchronization is divided into full replication and partial replication. Full replication is the RDB operation performed after you start the synchronization command, and the data in the master is sent to the slave through the RDB. Because RDB uses the bgsave instruction, the operations done by the master during the full copy will enter a buffer, and when the full copy is over, it needs to be partially copied to restore the operation in the buffer. AOF is used here.
Notes on partial data synchronization:
Through the above introduction, we know that the operations performed during full replication will be put into the cache area, but if the cache area is set too small, it will cause the master to block. The cache area size can be set in the following way:
repl-backlog-size 1mbYou can temporarily turn off slave external services by configuring the following parameters
slave-server-stable-data yes|no2.3 Command Propagation Phase
The command propagation phase is when the status of the master database changes, it will be synchronized to the slave through the command propagation phase
There are three core elements in the command propagation phase:
The running id of the server, the replication backlog buffer of the master server, and the replication offset of the master-slave server
The server running id is the identification code of each server running each time , composed of 40 characters, used to identify the identity when transmitting between servers.
The replication backlog buffer of the primary server:
The copy backlog buffer, also known as the copy buffer, is a first-in first-out queue. When the master database changes, the master saves the commands to be propagated to the slave in the replication buffer, and the slave receives information from the replication buffer respectively.
The replication offset of the master and slave server:
Because the master first puts the data in the buffer during the command propagation phase, the master needs a copy offset to record the position corresponding to the instruction sent to the slave. The slave needs to synchronize the data in the buffer to itself, so it also needs a copy offset to record the received position. If the network is accidentally disconnected, it can continue directly from the copy offset position after the network is connected again. copy.
2.4 Heartbeat mechanism
After entering the command propagation phase, the master and slave need to maintain the connection between the two parties through the heartbeat mechanism
Query whether the slave is online through the ping command, the period can be set by repl-ping-slave-period, the default is 10 seconds
replconf ack {offset}Role: report the slave's own replication offset; determine whether the master is online
When most of the slaves are offline or the delay of the slave is too high, you can forcibly turn off the write function of the master to stop data synchronization:
min-slaves-to-write 2 当连接的slave小于等于2台时停止数据同步
min-slaves-max-lag 10 当连接的slave延迟大于10秒，停止数据同步