Redis master-slave replication, sentinel and cluster
One, Redis cluster
1. Master-slave replication
Master-slave replication is the basis of high-availability Redis. Sentinel and clusters achieve high availability on the basis of master-slave replication. Master-slave replication mainly implements multi-machine backup of data, as well as load balancing and simple failure recovery for read operations. Defects: failure recovery cannot be automated; write operations cannot be load balanced; storage capacity is limited by a single machine
2. Sentry
On the basis of master-slave replication, Sentinel realizes automatic failure recovery. Defects: Write operations cannot be load balanced; storage capacity is limited by a single machine; Sentinel cannot automatically failover the slave node. In the read-write separation scenario, the failure of the slave node will cause the read service to be unavailable, and additional monitoring of the slave node is required , Switch operation
3. Cluster
Through clustering, Redis solves the problem that write operations cannot be load balanced, and storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution
Two, Redis
Master-slave replication refers to copying the data of one Redis server to other Redis servers. The former is called the master node (Master), the latter is called the slave node (Slave); data replication is one-way, only from the master node to the slave node
By default, each Redis server is the master node; and a master node can have multiple slave nodes (or no slave nodes), but a slave node can only have one master node
1. The role of master-slave replication
①Data redundancy
Master-slave replication realizes hot backup of data, which is a data redundancy method besides persistence
②Failure recovery
When the master node has a problem, the slave node can provide services to achieve rapid failure recovery; in fact, it is a kind of service redundancy
③Load balancing
On the basis of master-slave replication, with the separation of read and write, the master node can provide the write service, and the slave node can provide the read service (that is, the application connects to the master node when writing Redis data, and the application connects to the slave node when reading Redis data), sharing the server Load; especially in the scenario of writing less and reading more, sharing the read load by multiple slave nodes can greatly increase the concurrency of the Redis server
④High-availability cornerstone
In addition to the above functions, master-slave replication is also the basis for the implementation of sentinels and clusters, so master-slave replication is the basis for Redis high availability
2. Master-slave replication process
① If a slave machine process is started, it will send a "sync command" command to the Master machine to request a synchronous connection
②Whether it is the first connection or reconnection, the Master machine will start a background process to save the data snapshot to the data file (execute the rdb operation), and the Master will also record all the commands to modify the data and cache them in the data file
③After the background process completes the caching operation, the Master machine will send the data file to the Slave machine, and the Slave machine will save the data file to the hard disk, and then load it into the memory, and then the Master machine will perform all operations that modify the data. And send it to the slave machine. If the slave fails and causes the
machine to go down, it will automatically reconnect after returning to normal. ④After the Master machine receives the connection from the slave machine, it will send its complete data file to the slave machine. If Mater receives multiple slaves at the same time Synchronize the request, the Master will start a process in the background to save the data file, and then send it to all the slave side machines to ensure that all the slave side machines are normal
Three, Redis master-slave replication
1. Build Redis master-slave replication
①Master node configuration
#Turn off the firewall
systemctl stop firewalld
setenforce 0
#Install Redis
yum install -y gcc gcc-c++ make
cd /opt #Upload the compressed package of redis to the /opt directory
tar zxvf redis-5.0.7.tar.gz
cd /opt/redis-5.0.7/
make
make PREFIX=/usr/local/redis install
cd /opt/redis-5.0.7/utils
./install_server.sh #Then press enter all the time
When you reach the step Please select the redis executable path [/usr/loca1/bin/redis-server], enter /usr/local/redis/bin/redis-server
ln -s /usr/local/redis/bin/* /usr/local/bin/ #Create soft connection
#Modify redis configuration file
vim /etc/redis/6379.conf
bind 0.0.0.0 #70 line, comment out the bind item, or modify it to 0.0.0.0, the default monitoring all network cards
daemonize yes #137 line, open the daemon
logfile /var/log/redis_6379. log #172 line, specify the log file directory
dir /var/lib/redis/6379 #264 line, specify the working directory
appendonly yes #700 line, enable the AOF persistence function
/etc/init.d/redis_6379 restart #Restart the service
②Modify the configuration of Slave node
#The configuration of the two slave nodes is the same
Install redis as above
#Modify redis configuration file
vim /etc/redis/6379.conf
bind 0.0.0.0 #70 line, modify the listening address to 0.0.0.0
daemonize yes #137 line, open the daemon
logfile /var/log/redis_6379.log #172 line, specify the log file directory
dir /var/lib/redis/6379 #264 line, specify the working directory
replicaof 192.168.200.11 6379 #287 line, uncomment and specify the Master node IP and port to be synchronized
appendonly yes #700 line, enable AOF persistence function
/etc/init.d/redis_6379 restart #Restart the service
③Verify the master-slave effect
#MasterLog View
tail -f /var/log/redis_6379.log
#Master node view
redis-cli -h 192.168.200.11
INFO replication
Four, Redis sentinel mode
The core function of Sentinel: On the basis of master-slave replication, Sentinel introduces automatic failover of the master node
1. Principle of sentinel mode
It is a distributed system used to monitor each server in the master-slave structure. When a failure occurs, a new Master is selected through a voting mechanism, and all slaves are connected to the new Master. Therefore, the number of the entire cluster running sentinels must not be less than 3 nodes
2. The role of sentinel mode
①Monitoring: The sentinel will constantly check whether the master node and slave node are operating normally
②Automatic failover: When the master node fails to work normally, the sentinel will start an automatic failover operation. It will upgrade one of the slave nodes of the failed master node to the new master node, and make other slave nodes copy the new master node
③Notification: The sentry can send the result of the failover to the client
3. The composition of the sentinel structure
①Sentinel node: The sentinel system consists of one or more sentinel nodes. The sentinel node is a special redis node and does not store data
②Data node: both the master node and the slave node are data nodes
The start of the sentinel depends on the master-slave mode, so you must install the master-slave mode before doing the sentry mode. All nodes need to deploy the sentry mode. The sentry mode will monitor whether all Redis working nodes are normal. When the Master appears When there is a problem, because other nodes lose contact with the master node, they will vote. More than half of the vote will think that there is a problem with this Master, and then the sentry room will be notified, and then one of the Slaves will be selected as the new Master.
Special attention is needed to be objective. Offline is a concept only available to the master node; if the slave node and the sentinel node fail, after being subjectively offline by the sentinel, there will be no subsequent objective offline and failover operations
4. Configure sentinel mode
① Configure sentinel mode
#All nodes are configured the same
vim /opt/redis-5.0.7/sentinel.conf
protected-mode no #17 line, turn off the protected mode
port 26379 #21 line, Redis sentinel's default listening port
daemonize yes #26 line, specify sentinel as the background startup
logfile "/var/log/sentinel.log" #36 line, specify Log storage path
dir “/var/lib/redis/6379” #65 line, specify the database storage path
sentinel monitor mymaster 192.168.200.11 6379 2 #84 line, modify the designated sentinel node to monitor the master node 192.168.200.10:6379, this The name of the master node is mymaster, and the meaning of the last 2 is related to the failure judgment of the master node: at least 2 sentinel nodes are required to agree to determine the master node failure and failover
sentinel down-after-milliseconds mymaster 30000 #113 line, judgment The time period when the server is down, the default is 30000 milliseconds (30 seconds)
sentinel failover-timeout mymaster 180000 #146 line, the maximum timeout time of the failed node is 180000 (180 seconds)
#After changing a configuration, it can be transmitted to other servers through the scp command
scp /opt/redis-5.0.7/sentinel.conf [email protected]:/opt/redis-5.0.7/sentinel.conf
scp /opt/redis-5.0.7/sentinel.conf [email protected]:/opt/redis-5.0.7/sentinel.conf
②Start sentinel mode
#Start Master first, then start Slave
cd /opt/redis-5.0.7/ #First switch to the redis directory to start
redis-sentinel sentinel.conf & #& refers to background startup
Five, Redis cluster mode
The cluster is composed of multiple nodes (Node), and Redis data is distributed among these nodes. The nodes in the cluster are divided into master nodes and slave nodes
Only the master node is responsible for the maintenance of read and write requests and cluster information; the slave node only copies the data and status information of the master node
1. The role of clusters
①Data partition: Data partition (or data sharding) is the core function of the cluster
The cluster disperses data to multiple nodes. On the one hand, it breaks through the limitation of the memory size of Redis single machine, and the storage capacity is greatly increased; on the other hand, each master node can provide external read and write services, which greatly improves the responsiveness of the cluster.
Redis stand-alone memory size limitation is mentioned in the introduction of persistence and master-slave replication; for example, if the stand-alone memory is too large, the fork operation of bgsave and bgrewriteaof may cause the main process to block, and it may be possible when the host is switched in the master-slave environment As a result, the slave node cannot provide services for a long time, and the replication buffer of the master node may overflow during the full replication phase
②High availability
The cluster supports master-slave replication and automatic failover of the master node (similar to sentinel); when any node fails, the cluster can still provide external services
2. Data sharding of Redis cluster
Redis cluster introduces the concept of hash slots.
Redis cluster has 16384 hash slots (numbered 0-16383)
. Each node in the cluster is responsible for a part of the hash slot. After
each key is checked by CRC16, the remainder of 16384 is taken to determine which one to place. Hope slot, through this value, to find the node corresponding to the corresponding slot, and then automatically jump to the corresponding node for access operations
#When there are three nodes, redis will divide the hash slot into three parts. When one part of the hash slot is broken, the entire cluster will be unavailable
3. Build Redis cluster mode
①Create a redis cluster directory
cd / etc / redis
mkdir -p redis-clusyer/redis600{1…6}
② Copy the file to the redis600 directory
for i in {1…6}
do
cp /opt/redis-5.0.7/redis.conf /etc/redis/redis-cluster/redis600$i
cp /opt/redis-5.0.7/src/redis-cli /opt/redis-5.0.7/src/redis-server /etc/redis/redis-cluster/redis600$i
done
③Modify the configuration file
cd /etc/redis/redis-cluster/redis6001 #6 ports are different
vim redis.conf
bind 0.0.0.0 #69 line, listen to all network segments
protected-mode no #88 line, modify, close the protected mode
port 6001 #92 line, modify, redis listening port,
daemonize yes #136 line, as a separate process Start
cluster-enabled yes #832 line, uncomment, enable cluster function
cluster-config-file nodes-6001.conf #840 line, uncomment, cluster name file setting
cluster-node-timeout 15000 #846, uncomment cluster timeout Time setting
appendonly yes #700 line, modify, open AOF persistence
④Start the redis node service
for d in {1…6}
do
cd /etc/redis/redis-cluster/redis600$d
redis-server redis.conf
done
ps -elf | grep redis #View process
⑤Start the cluster
redis-cli --cluster create 127.0.0.1:6001 127.0.0.1:6002 127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 127.0.0.1:6006 --cluster-replicas 1
#The six instances are divided into three groups, each group has one master and one slave, the front is the master node, and the back is the slave node. You need to enter yes when interacting below to create it.
--Replicas 1 means that each master node has 1 slave node
⑥Test cluster
redis-cli -o 6001 -c #-c jump between nodes
cluster slots #View the hash slot number range of the node
