A: redis replication of core mechanics
(1) redis asynchronously copy data to the slave node, but starts redis 2.8, slave Node periodically acknowledged amount of data per own replication
(2) is a master node of the plurality of slave Node
(. 3) slave node can also be connected to other node of the Slave
(4) copy the Slave node do when it is working properly will not block master node of
(5) slave node when doing replication, it will not block query operations on their own, it will use old data sets to provide services; however, when copying is complete, you need to delete the old data set, load the new data set, this time will be the suspension of foreign service
(6) slave node is mainly used for horizontal expansion, do read and write separation, expansion slave node reads throughput can be improved
Two: master lasting significance for the master-slave architecture of security of
If a master-slave architecture, it is recommended to be turned master node persistence!
Not recommended for use as a data slave node hot standby master node, because in that case, if you turn off the persistent master may restart when downtime in the master data is empty, and then may go through a copy, salve node data also lose a.
master -> RDB and AOF are closed -> All in memory
master down, restart, there is no local data can be restored, then it will directly consider themselves IDE data is empty, the empty master data will be synchronized to the slave set up, all the data of all slave empty. 100% data loss
master node, you must use the persistence mechanism
Second, master of various backup programs, or else to do, if say to all local file is missing; the selection of a master rdb to restore from backup; so as to ensure that the master boot time, there is data.
Even with the high availability mechanisms, slave node can automatically take over the master node, it is also possible sentinal has not been detected master failure, master node automatically restart, or may result in a failure to empty all the slave node data above.
Three: The core principle of master-slave architecture
When you start a slave node when it sends a PSYNC command to the master node
If this is the slave node reconnect master node, then the master node to the slave copies only part missing data; otherwise, if the slave node is first connected master node, it will trigger a Full Resynchronization . Begin full resynchronization time, master starts a background thread to start generating an RDB snapshot file , but also all write commands from the client receives the cached in memory. After the RDB file generation is completed, master will be sent to the RDB Slave, SL ave will first write to local disk, and then loaded from the local disk into memory . Then the master will write memory cache command is sent to the slave, slave will synchronize the data.
slave node with the master node if there is a network failure, disconnected, it will automatically reconnect. master if there are multiple slave node to reconnect, just start a rdb save operation, all data services with a slave node.
Four: master-slave replication HTTP
From the beginning redis 2.8, it supports master-slave replication HTTP, if the master-slave replication, network connections cut off, you can then copy the last place, continue to replicate it, rather than starting from scratch copy. master node will be common in memory a backlog , Master and slave keeps a replica offset as well as a Master the above mentioned id , offset is stored in the backlog of. If the master and slave network connections cut off, slave master will continue to start from the last copy of the replica offset
However, if the corresponding offset is not found, it will perform a resynchronization
Five: No copy of the disk
master created in memory directly rdb, and then sent to the slave, not in their own local disk of landing
diskless-Sync-repl
repl-diskless-Sync-Delay, wait for a certain length of time before you start copying, because more slave to wait to reconnect over
Six: Expired key processing
slave does not expire key, just wait for the expired master key. If an expired master key, or by LRU out of a key, it will simulate a del command to the slave.
Seven: Copy Process
1) slave node starts, save only information master node, including the master node and the host ip, but did not start replication processes
master host and ip is where they came from, redis.conf inside slaveof configuration
(2) slave node have a regular internal task to check whether a second node to the new Master and replicate, if found, just like the Master node to establish a network connection socket
(3) slave node node transmits a ping command to Master
(. 4) password authentication, if the master set requirepass , then the salve node must send masterauth password authentication in the past
(5) master node for the first time performing a full volume copy, all data sent to the Node Slave
(6) continued to master the Node subsequent write command, asynchronous replication to the slave node
Eight: data synchronization mechanisms related to the core
Refers to the first slave when connected msater the full amount of execution replication mechanisms that process inside some of the details of your
(1) master and slave will maintain an offset
master itself will continue to accumulate offset, slave will continue to accumulate in itself offset
slave will report its second offset to the master, while also saving offset each slave master's
Not that this particular copy to use in the full amount, mainly master and slave must know their offset data, inconsistent data between each other in order to know the situation
（2）backlog
master node has a backlog, the default size is 1MB
master node to the slave node when copying data, will be data synchronization in the backlog write a
backlog is mainly used to make incremental full amount after the interrupt copy copy
（3）master run id
info server, you can see the master run id
if based on host + ip locate master node, do not fly, if the master node restart or data has changed, then the slave node should be differentiated according to the different run id, run id different to do copy the full amount
if you need to change the run id not restart redis, you can use redis-cli debug reload command
（4）psync
From node uses psync be copied from the Node Master, psync runid offset
Master the Node returns response information according to their own circumstances, it may be FULLRESYNC runid offset the full amount of the trigger replication, incremental replication may be triggered CONTINUE
3, the whole amount of replication
(1) master the implementation bgsave, rdb generate a snapshot file locally
(2) master node will be sent to salve node rdb snapshot files, copy rdb if more than 60 seconds (repl-timeout), then the slave node will assume replication failures , this parameter can be appropriately adjusted large
(3) for Gigabit Ethernet machine, usually per second 100MB, 6G files, most likely more than 60s
(4) Master the Node when generating rdb, all new write command will be cached in memory after, the salve node saved rdb, then copy the new write command to salve node
(5) Client-the Output-buffer 64MB-256MB limit Slave 60, if during replication, memory buffers continue to consume more than 64MB, or one of more than 256MB, then stop replication, replication fails
(6) slave node after receiving rdb, empty your old data, and then load rdb to your memory again, the same time offer services based on the old version of the data
(7) If the slave node opens the AOF, then performs BGREWRITEAOF immediately, AOF rewrite
rdb generated, through the network copy rdb, cleaning old data slave, slave aof rewrite, it is time-consuming
If the amount of data that is replicated between 4G ~ 6G, it is likely that the total amount of time spent copying the one and a half to two minutes
4, incremental replication
(1) If the total amount of the replication process, master-slave network connection is broken, then reconnect when salve master, will trigger incremental replication
(2) master the missing part to obtain data directly from their backlog to send to the slave node, the default backlog is 1MB
(. 3) is used to obtain data from msater according psync slave transmission backlog in the offset
5、heartbeat
From the master node will send a heartbeat message to each other
The default master sent once every 10 seconds heartbeat, salve node transmits a heartbeat every 1 second
6, asynchronous replication
After each master receives the write command, the internal write data now, and to send an asynchronous slave node
Two core Redis replication operations:
- Resynchronization (the resync)
- Full resynchronization (full resynchronization)
- Part resynchronization (partial resynchronization)
- Command propagation (command propagate)
