1. High availability of
redis 1.1 The concept of high availability of redis
In web servers, high availability refers to the time when the server can be accessed normally. etc).
The calculation formula for high availability is 1-(downtime)/(downtime+running time), which is somewhat similar to the bit error rate of network transmission parameters. We use the number of 9 to represent availability:
2 nines: 99%, downtime within a year: 1% x 365 days = 3.6524 days = 87.6h
4 nines: 99.99%, downtime within one year: 0.01%×365 days=52.56min
5 nines: 99.999%, downtime within one year: 0.001%*365 days=5.265min
Eleven nines: almost a year of downtime in seconds
However, in the context of Redis, the meaning of high availability seems to be broader. In addition to ensuring the provision of normal services (such as master-slave separation, fast disaster recovery technology), it is also necessary to consider the expansion of data capacity and data security without loss, etc.
1.2 High Availability Technology of Redis
In Redis, technologies to achieve high availability mainly include persistence, master-slave replication, sentry and cluster clusters. The following describes their functions and what problems they solve.
Persistence: Persistence is the simplest high-availability method (sometimes not even classified as a high-availability method). Its main function is data backup, that is, storing data on the hard disk to ensure that data will not be lost due to process exit.
Master-slave replication: Master-slave replication is the basis of highly available Redis. Sentinels and clusters are based on master-slave replication to achieve high availability. Master-slave replication mainly realizes multi-machine backup (and synchronization) of data, as well as load balancing and simple fault recovery for read operations.
Defects: Failure recovery cannot be automated; write operations cannot be load-balanced; storage capacity is limited by a single machine.
Sentinel: Based on master-slave replication, Sentinel implements automatic failure recovery. (The master is down, find a slave to become the new master, and the sentinel node will monitor it)
Defects: Write operations cannot be load-balanced; storage capacity is limited by a single machine.
Cluster cluster: Through clustering, Redis solves the problem that write operations cannot be load-balanced, and the storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution. (6 units start, in pairs, 3 masters and 3 slaves)
Cluster cluster: Through clustering, Redis solves the problem that write operations cannot be load-balanced, and the storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution. (6 sets start, in pairs, 3 masters and 3 slaves)
2. Redis master-slave replication
Master-slave replication refers to copying the data of one Redis server to other Redis servers. The former is called the master node (Master), and the latter is called the slave node (slave); data replication is one-way, only from the master node to the slave node.
By default, each Redis server is a master node; and a master node can have multiple slave nodes (or no slave nodes), but a slave node can only have one master node.
2.1 The role of master-slave replication
Data redundancy: Master-slave replication implements hot backup of data, which is a data redundancy method other than persistence.
Fault recovery: When the master node has a problem, the slave node can provide services to achieve rapid fault recovery; it is actually a kind of service redundancy.
Load balancing: On the basis of master-slave replication, combined with read-write separation, the master node can provide write services, and the slave nodes can provide read services (that is, the application connects to the master node when writing Redis data, and the application connects to the slave node when reading Redis data) , to share the server load; especially in the scenario of writing less and reading more, sharing the read load through multiple slave nodes can greatly increase the concurrency of the Redis server.
The cornerstone of high availability: In addition to the above functions, master-slave replication is also the basis for the implementation of sentinels and clusters, so master-slave replication is the basis for high availability of Redis.
1.2 Master-slave replication process
(1) If a slave machine process is started, it will send a sync command command to the Master machine to request a synchronous connection.
(2) Whether it is the first connection or reconnection, the Master machine will start a background process to save the data snapshot to the data file (execute rdb operation), and the Master will also record all the commands to modify the data and cache them in the data file middle.
(3) After the background process completes the cache operation, the Master machine will send the data file to the slave machine, the slave machine will save the data file to the hard disk, and then load it into the memory, and then the Master machine will modify all the data files The operation is also sent to the slave machine. If the slave fails and causes a downtime, it will automatically reconnect after returning to normal.
(4) After the master machine receives the connection from the slave machine, it sends its complete data file to the slave machine. If the master receives synchronization requests from multiple slaves at the same time, the master will start a process in the background to save Data file, and then send it to all slave machines, make sure all slave machines are normal.
3. Deployment of redis master-slave (one master and two slaves)
experimental specific steps
1 Install redis one master and two slave servers
//environment preparation
systemctl stop firewalld
systemctl disable firewalld
setenforce 0
sed -i 's/enforcing/disabled/' /etc/ selinux/config
#Modify kernel parameters
vim /etc/sysctl.conf
vm.overcommit_memory = 1
net.core.somaxconn = 2048
sysctl -p
//Install redis
yum install -y gcc gcc-c++ make
tar zxvf /opt/redis-7.0.9.tar.gz -C /opt/
cd /opt/redis-7.0.9
make
make PREFIX=/usr/local/redis install
#Because the Makefile is directly provided in the Redis source package , so after decompressing the software package, you don't need to execute ./configure first to configure it, you can directly execute the make and make install commands to install it.
#Create redis working directory
mkdir /usr/local/redis/{conf,log,data}
cp /opt/redis-7.0.9/redis.conf /usr/local/redis/conf/
useradd -M -s /sbin/nologin redis
chown -R redis.redis /usr/local/redis/
#Environment variable
vim /etc/profile
PATH=$PATH:/usr/local/redis/bin #Add a line
source /etc/profile
//Define systemd service management script
vim /usr/lib/systemd/system/redis-server.service
[Unit]
Description=Redis Server
After=network.target
[Service]
User=redis
Group=redis
Type=forking
TimeoutSec=0
PIDFile=/usr/local/redis/log/redis_6379.pid
ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/conf/redis.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
2 Modify the master-slave master configuration file
----- modify the Redis configuration file (Master node operation) -----
vim /usr/local/redis/conf/redis.conf
bind 0.0.0.0
#87 line, modify the listening address to 0.0.0.0
protected-mode no
#111 line, set the local access protection mode to no
port 6379
#138 line, Redis listens to port 6379 by default
daemonize yes
#309 line, set it as a daemon process, start
pidfile /usr/local/redis/log/redis_6379.pid
#341 line, specify PID file
logfile "/usr/local/redis/log/ redis_6379.log"
#354 line, specify the log file
dir /usr/local/redis/data
#504 line, specify the directory where the persistent file is located
#requirepass abc123
#1037 line, optional, set the redis password
appendonly yes
#1380 line, open AOF
systemctl restart redis-server.service
start redis service
----- Modify Redis configuration file (Slave node operation) -----
vim /usr/local/redis/conf/redis.conf
bind 0.0.0.0
#87 line, modify the listening address to 0.0.0.0
protected-mode no
#111 line, set the local access protection mode no
port 6379
#138 line, Redis listens to port 6379 by default
daemonize yes
#309 line, set it as a daemon process, start
pidfile /usr/local/redis/log/redis_6379 in the background. pid
#341 line, specify the PID file
logfile "/usr/local/redis/log/redis_6379.log"
#354 line, specify the log file
dir /usr/local/redis/data
#504 line, specify the directory where the persistent file is located
# requirepass abc123
#1037 line, optional, set redis password
appendonly yes
#1380 line, open AOF
replicaof 192.168.80.10 6379
#528 line, specify the Master node IP and port to be synchronized
#masterauth abc123
#535 line, optional, specify the password of the Master node, only set requirepass
systemctl on the Master node restart redis-server.service
to start the service
Experimental test
master write data
127.0.0.1:6379> keys *
127.0.0.1:6379> set name zhangsan
127.0.0.1:6379> get name
two from the library to view the database
4. Redis sentinel mode
The method of master-slave switching technology is: when the server is down, a slave needs to be manually switched to the master, which requires manual intervention, which is not only time-consuming and laborious, but also causes the service to be unavailable for a period of time. In order to solve the shortcomings of master-slave replication, there is a sentinel mechanism.
Sentinel's core function: Based on master-slave replication, Sentinel introduces automatic failover of the master node.
The composition of sentinel mode:
Sentinel node: The sentinel system consists of one or more sentinel nodes, which are special redis nodes that do not store data.
Data Nodes: Both master and slave nodes are data nodes.
4.1 The role of sentinel mode
Monitoring: Sentry constantly checks that the master and slave nodes are functioning properly.
Automatic failover: When the master node fails to work normally, Sentinel will start an automatic failover operation. It will upgrade one of the slave nodes of the failed master node to a new master node, and let other slave nodes copy the new master node instead.
Notifications (reminders): Sentinels can send failover results to clients.
In addition: the sentinel node can also be independent on other hosts, and does not need to be installed on the node server for master-slave replication of redis
4.2 Failover mechanism
1. The sentinel node regularly monitors to find out whether the master node has failed.
Each sentinel node will ask the master node, slave node and other sentinel nodes to send a ping command every 1 second for a heart check. If the master node does not reply within a certain time frame or replies with an error message, then the sentinel will consider the master node to be offline subjectively (unilaterally). When more than half of the sentinel nodes think that the master node is offline subjectively, it is objectively offline.
2. When the master node fails, the sentinel node will implement the election mechanism through the Raft algorithm (election algorithm) to jointly elect a sentinel node as the leader to be responsible for handling the failover and notification of the master node. So the number of clusters running Sentinels must not be less than 3 nodes.
3. The leader sentinel node performs failover, the process is as follows:
Upgrade a slave node to a new master node, and let other slave nodes point to the new master node;
if the original master node recovers, it will also become a slave node and point to the new master node;
notify the client that the master node has been replaced.
It is important to note that objective offline is a concept unique to the master node; if the slave node and sentinel node fail, after being subjectively offline by the sentinel, there will be no subsequent objective offline and failover operations
4.3 Selection of master nodes in sentinel mode
1. Filter out unhealthy (offline) slave nodes that have not responded to the sentinel ping response.
2. Select the slave node with the highest priority configuration in the configuration file. (replica-priority, default value is 100)
3. Select the slave node with the largest replication offset, that is, the most complete replication.
The start of Sentinel depends on the master-slave mode, so the Sentry mode must be installed after the master-slave mode is installed.
5. Deployment of redis sentinel mode
Experimental component deployment
The specific operation steps of the experiment
are to deploy the sentinel mode on the basis of redis master-slave replication (just do it on the basis of the above one-master-two-slave replication)
Step 1: Modify the configuration file of the sentinel node (the same operation for all sentinel nodes)
The configuration file of the sentinel is the configuration that comes with the redis software
cp /opt/redis-7.0.9/sentinel.conf /usr/local/redis/conf/
chown redis.redis /usr/local/redis/conf/sentinel.conf
vim /usr/local/redis/conf/sentinel. conf
protected-mode no
#6 line, close the protection mode
port 26379
#10 line, the default listening port of Redis sentinel
daemonize yes
#15 line, specify sentinel as the background start
pidfile /usr/local/redis/log/redis-sentinel.pid
#20 line, specify the PID file
logfile "/usr/local/redis/log/sentinel.log"
#25 line, specify the log storage path
dir /usr/local/redis/data
#54 line, specify the database storage path
sentinel monitor mymaster 192.168.80.10 6379 2
Line #73, modify and specify that the sentinel node monitors the master node 192.168.80.10:6379. The
name of the master node is mymaster. The meaning of the last 2 is related to the fault judgment of the master node: at least two sentinel nodes must agree to
the judgment . Master node failure and failover
#sentinel auth-pass mymaster abc123
#76 line, optional, specify the password of the Master node, only set requirepass
sentinel down-after-milliseconds mymaster 3000
#114 line, the server is down The time period, the default is 30000 milliseconds (30 seconds)
sentinel failover-timeout mymaster 180000
#214 line, the interval between two failovers of the same sentinel to the same master (180 seconds)
Step 2: Start the sentinel mode
Start the master first, then start the slave
cd /usr/local/redis/conf/
redis-sentinel sentinel.conf &
-----查看哨兵信息-----
redis-cli -p 26379 info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.5023:6379,slaves=2,sentinels=3
-----Fault simulation-----
#View redis-server process number:
ps -ef | grep redis
root 57031 1 0 15:20 ? 00:00:07 /usr/local/bin/redis-server 0.0 .0.0:6379
root 57742 1 1 16:05 ? 00:00:07 redis-sentinel *:26379 [sentinel]
root 57883 57462 0 16:17 pts/1 00:00:00 grep --color=auto redis
#Kill the process number of redis-server on the Master node
kill -9 57031 #The process number of redis-server on the Master node
Experimental results
[root@localhost redis-5.0.7]# tail -f /var/log/sentinel.log
#New master creates key-value pairs
[root@localhost redis-5.0.7]# redis-cli
127.0.0.1 :6379> set newname lisi
OK
127.0.0.1:6379> get newname
"lisi"
127.0.0.1:6379>
Test Results:
6. Redis cluster mode
Cluster, that is, Redis Cluster, is a distributed storage solution introduced by Redis3.0.
The cluster consists of multiple nodes (Nodes), and Redis data is distributed among these nodes. The nodes in the cluster are divided into master nodes and slave nodes: only the master node is responsible for the maintenance of read and write requests and cluster information; the slave nodes only replicate the data and status information of the master node.
6.1 The role of the cluster
(1) Data partitioning: Data partitioning (or data sharding) is the core function of the cluster.
The cluster distributes data to multiple nodes. On the one hand, it breaks through the limit of Redis single-machine memory size, and the storage capacity is greatly increased; on the other hand, each master node can provide external read and write services, which greatly improves the responsiveness of the cluster.
Redis stand-alone memory size is limited, which was mentioned in the introduction of persistence and master-slave replication; As a result, the slave node cannot provide services for a long time, and the replication buffer of the master node may overflow during the full replication phase.
(2) High availability: The cluster supports master-slave replication and automatic failover of the master node (similar to Sentinel); when any node fails, the cluster can still provide external services.
Through the cluster, Redis solves the problem that the write operation cannot be load-balanced and the storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution.
6.2 Data Fragmentation of Redis Cluster
Redis Cluster introduces the concept of hash slots.
Redis Cluster has 16384 hash slots (numbered 0-16383).
Each node of the cluster is responsible for a portion of the hash slots.
After each Key passes the CRC16 check, take the remainder of 16384 to determine which hash slot to place. Through this value, find the node corresponding to the corresponding slot, and then directly and automatically jump to the corresponding node for access operations .
Take a cluster composed of 3 nodes as an example:
Node A contains hash slots from 0 to 5460
Node B contains hash slots from 5461 to 10922 Node
c contains hash slots from 10923 to 16383
7. Deployment of redis cluster
In a real production environment, the cluster cluster of redis requires at least six servers It can be realized, if it is because of computer performance problems
You can try redis multi-instance deployment
Deployment of experimental components
The specific steps of the experiment
Step 1
cd /usr/local/redis/
mkdir -p redis-cluster/redis600{1..6}
for i in {1..6}
do
cp /opt/redis-7.0.9/redis. conf /usr/local/redis/redis-cluster/redis600$i
cp /opt/redis-7.0.9/src/redis-cli /opt/redis-7.0.9/src/redis-server
/usr/local/redis /redis-cluster/redis600$i
done
Step 2 #Enable the cluster function:
#The configuration files of the other 5 folders are modified by analogy, and the 6 ports must be different.
cd /usr/local/redis/redis-cluster/redis6001
vim redis.conf
#bind 127.0.0.1
#87 line, comment out the bind item, monitor all network cards by default
protected-mode no
#111 line, close the protection mode
port 6001
#138 OK, modify the redis listening port
daemonize yes
#309 line, set as daemon process, start
pidfile /usr/local/redis/log/redis_6001.pid
#341 line, specify PID file
logfile "/usr/local/redis/log/redis_6001.log"
#354 Line, specify the log file
dir ./
#504 line, specify the directory where the persistent file is
appendonly yes
#1379 line, enable AOF
cluster-enabled yes
#1576 line, uncomment, enable the cluster function
cluster-config-file nodes-6001.conf
#1584 line, uncomment, cluster name file set
cluster-node-timeout 15000
#1590 line, uncomment cluster timeout setting
When 6001 is configured, you can directly copy the configuration file to 6002-6006, and then modify it directly with sed -i
Home : sed -i 's/6001/6002/g' usr/local/redis/redis-cluster/redis6002
sed -i 's/6001/6003/g' usr/local/redis/redis-cluster/redis6003
sed -i 's/6001/6004/g' usr/local/redis/redis-cluster/redis6004
sed -i 's/6001/6005/g' usr/local/redis/redis-cluster/redis6005
sed -i 's/6001/6006/g' usr/local/redis/redis-cluster/redis6006
Step 3 Start the redis node
Enter the six folders respectively and execute the command: redis-server redis.conf to start the redis node
cd /usr/local/redis/redis-cluster/redis-cluster/redis-cluster/redis6001
redis-server redis.conf
or
for d in {1..6}
do
cd /usr/local/redis/redis-cluster/redis600$d
./redis-server redis.conf
done
ps -ef | grep redis
Step 4#Start cluster
redis-cli --cluster create 127.0.0.1:6001 127.0.0.1:6002 \
127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 127.0.0.1:6006 \
--cluster-replicas 1
#The six instances are divided into three groups, one master and one slave in each group, the front is the master node, and the latter is the slave node.
When interacting below, you need to enter yes to create.
--replicas 1 means each master node has 1 slave node.
Test cluster
redis-cli -p 6001 -c #Add -c parameter, nodes can jump to each other
127.0.0.1:6001> cluster slots #View the hash slot number range of nodes
1) 1) (integer) 5461
2 ) (integer) 10922 #Hash slot number range
3) 1) "127.0.0.1"
2) (integer) 6003 #Primary node IP and port number
3) "fdca661922216dd69a63a7c9d3c4540cd6baef44"
4) 1) "127.0.0.1"
2) ( integer) 6004 #Slave node IP and port number
3) "a2c0c32aff0f38980accd2b63d6d952812e44740"
2) 1) (integer) 0
2) (integer) 5460
3) 1) "127.0.0.1"
2) (integer) 6001
3) "0e5873747a2e26bdc935bc76c2bafb19d0a54b11"
4) 1) "127.0.0.1"
2) (integer) 6006
3) "8842ef5584a85005e135fd0ee59e5a0d67b0cf8e"
3) 1) (integer) 10923
2) (integer) 16383
3) 1) "127.0.0.1"
2) (integer) 6002
3 ) "816ddaa3d1469540b2ffbcaaf9aa867646846b30"
4) 1) "127.0.0.1"
2) (integer) 6005
3) "f847077bfe6722466e96178ae8cbb09dc8b4d5eb"
127.0.0.1 :6001> set name zhangsan
-> Redirected to slot [5798] located at 127.0.0.1:6003
OK
127.0 .0.1:6001> cluster keyslot name
#View the slot number of the name key
redis-cli -p 6004 -c
127.0.0.1: 6004> keys *
#The corresponding slave node also has this data, but other nodes do not
1) "name"
redis-cli -p 6001 -c cluster nodes
Step 5: Add nodes to the Cluster cluster for dynamic expansion.
The redis 5 cluster supports dynamic expansion of nodes under load.
The existing cluster has 6 nodes 127.0.0.1:6001 - 127.0.0.1:6006, and 3 groups of master and slave nodes. Now add the fourth group of master-slave nodes 127.0.0.1:6007, 127.0.0.1:6008
1) First create a cluster node, modify the port number to 6007, 6008 to be added to the cluster cluster
2) Create a new master node 127.0.0.1:6007
redis-cli -p 6001 --cluster add-node 127.0.0.1: 6007 127.0.0.1:6001
或
redis-cli -p 6001
cluster meet 127.0.0.1 6007
cluster meet 127.0.0.1 6008
3) Create 127.0.0.1:6008 as the slave node of 127.0.0.1:6007
redis-cli --cluster add-node 127.0.0.1:6008 127.0.0.1:6001 --cluster-slave (the following master can be omitted, should There is only one master and no slave nodes)
redis-cli -p 6001 --cluster add-node 127.0.0.1:6008 127.0.0.1:6001 --cluster-slave --cluster-master-id e44678abed249e22482559136bf45280fd3ac281
An existing node needs to be specified in the command to obtain cluster information and the node ID of the master node
4) The newly added master node has no slot number. Only when the cluster is initialized, it will be allocated according to the data of the master. For example, the newly added master node needs to be manually allocated. First, use cluster nodes to obtain the node ID number of each node in the
cluster
reshard slots. Because redis 5 does not yet support automatic balancing of slots, you need to calculate the number of slots that need to be moved and manually execute the command. In this example, it is necessary to move 1365 slots from the three groups of existing master nodes to the new master node 127.0.0.1:6007 to achieve balance
redis-cli -p 6007 --cluster reshard 127.0.0.1:6001 --cluster-from \ e1a033e07f0064e6400825b4ddbcd6680c032d10 --cluster-to \ e44678abed249e22482559136bf45280fd3ac281 --cluster-slots 1365 --cluster-yes
Similarly, master nodes 6002 and 6003 also need to move 1365 slots to master node 6007
Or you can transfer hash slots interactively
or
redis-cli -p 6007 --cluster reshard 127.0.0.1:6001
How many slots do you want to move (from 1 to 16384)? 1000
#Specify the number of transfer slots
What is the receiving node ID? e44678abed249e22482559136bf45280fd3ac281
#Specify the master node ID of the number of receiving slots
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: e1a033e07f0064e6400825b4ddbcd6680c032d10
#Specify the assigned master node ID
Source node #2: done
#Enter the input and start the transfer