reference:
https://blog.csdn.net/qq_41071876/category_11284995.html
https://blog.csdn.net/weixin_45821811/article/details/119421774
https://blog.csdn.net/weixin_43001336/article/details/122816402
Redis has three modes, namely: master-slave mode, soldier mode, and Cluster mode. Rdis first used the master-slave mode as a cluster. If the master goes down, you need to manually configure the slave to be the master. Later, for high availability, the sentinel mode is proposed. In this mode, there is a sentinel to monitor the master and slave. If the master goes down, the slave can be automatically turned off. Converted to master, but it also has a problem, that is, it cannot be dynamically expanded; so the cluster cluster mode is proposed in 3.x.
master-slave mode
The master-slave mode is the simplest of the three modes. In master-slave replication, databases are divided into two categories: master database (master) and slave database (slave).
In order to avoid a single point of failure, it is common practice to replicate multiple copies of the database to be deployed on different servers, so that even if one server fails, other servers can continue to provide services. To this end, Redis provides the replication function, which can automatically synchronize the updated data to other databases when the data in one database is updated.
In the concept of replication, databases are divided into two categories, one is the master database (master), and the other is the slave database (slave). The master database can perform read and write operations, and when the write operation causes data changes, it will automatically synchronize the data to the slave database. The slave database is generally read-only and accepts data synchronized from the master database. A master database can have multiple slave databases, and a slave database can only have one master database. A master can have multiple slaves, but a slave can only correspond to one master.
principle
The slave database connects to the master database and sends the SYNC command;
after the master database receives the SYNC command, it starts to execute the BGSAVE command to generate an RDB file and uses the buffer to record all write commands executed thereafter; after the
master database BGSAVE is executed, it sends snapshots to all slave databases file, and continue to record the executed write command during sending;
discard all old data after receiving the snapshot file from the database, and load the received snapshot;
after the master database snapshot is sent, start sending the write command in the buffer to the slave database ;
The slave database finishes loading the snapshot, starts to receive command requests, and executes write commands from the master database buffer; (slave database initialization is complete) the
master database will send the same write command to the slave database every time it executes a write command, After receiving and executing the received write command from the database (the operation after the initialization of the slave database is completed)
, after disconnection and reconnection, the version after 2.8 will pass the command during the disconnection period to the slave database for incremental replication.
When the master-slave is just connected, full synchronization is performed; after full synchronization, incremental synchronization is performed. Of course, if necessary, the slave can initiate full synchronization at any time. Redis's strategy is, no matter what, it will first try to perform incremental synchronization. If it is unsuccessful, the slave machine is required to perform full synchronization.
advantage
Support master-slave replication, the master will automatically synchronize data to the slave, and can separate read and write;
in order to offload the read operation pressure of the master, the slave server can provide read-only operation services for the client, and the write service must still be done by the master Complete;
Slave can also accept connection and synchronization requests from other Slaves, which can effectively share the synchronization pressure of Master;
Master Server provides services for Slaves in a non-blocking manner. Therefore, during the Master-Slave synchronization, the client can still submit queries or modification requests;
the Slave Server also completes data synchronization in a non-blocking manner. During synchronization, if a client submits a query request, Redis returns the data before synchronization;
shortcoming
It does not have automatic fault tolerance and recovery functions. The downtime of the master and slave machines will cause the front-end part of the read and write requests to fail, and it needs to wait for the machine to restart or manually switch to recover; in the master-slave mode, if the master node goes
offline due to a failure, then the slave node Because there is no master node and the synchronization is interrupted, manual failover is required. If
multiple slaves are disconnected and need to be restarted, try not to restart at the same time. Because as long as the Slave starts, it will send a sync request to fully synchronize with the master. When multiple Slaves restart, it may cause the Master IO to increase sharply and cause downtime.
Unable to achieve dynamic expansion
sentinel mode
Sentinel (Sentinel) is a high-availability solution for Redis: a Sentinel system consisting of one or more Sentinel instances can monitor any number of master servers, and all slave servers under these master servers, and enter the monitored master server In the offline state, automatically upgrade a slave server under the offline master server to a new master server.
The role of the sentinel process
Monitoring (Monitoring): Sentinel (sentinel) will constantly check whether your Master and Slave are working properly.
Notification: When there is a problem with a monitored Redis node, the sentinel can send a notification to the administrator or other applications through the API.
Automatic failover: When a Master fails to work normally, the sentinel will start an automatic failover operation.
It will upgrade one of the slaves of the failed master to a new master, and change the other slaves of the failed master to copy the new master;
when the client tries to connect to the failed master, the cluster will also return the address of the new master to the client , so that the cluster can use the current Master to replace the failed Master.
After the Master and Slave servers are switched, the content of the configuration files of Master's redis.conf, Slave's redis.conf and sentinel.conf will change accordingly, that is, there will be one more line of slaveof in the redis.conf configuration file of the Master master server Configuration, the monitoring target of sentinel.conf will be changed accordingly.
principle
Each Sentinel (sentinel) process sends a PING command to the Master main server, Slave from the server and other Sentinel (sentinel) processes in the entire cluster once per second.
If an instance (instance) is longer than the value specified by the down-after-milliseconds option from the last valid reply to the PING command, the instance will be marked as subjectively offline (SDOWN) by the Sentinel process. If a Master master
server Marked as subjective offline (SDOWN), all Sentinel (sentinel) processes that are monitoring the Master main server must confirm that the Master main server has indeed entered the subjective offline state once per second. When there are enough Sentinel (sentinel
) ) process (greater than or equal to the value specified in the configuration file) confirms that the Master main server has entered the subjective offline state (SDOWN) within the specified time range, and the Master main server will be marked as objective offline (ODOWN). Under normal circumstances
, Each Sentinel process sends INFO commands to all Master servers and Slave servers in the cluster every 10 seconds.
When the Master main server is marked as objectively offline (ODOWN) by the Sentinel (sentinel) process, the frequency that the Sentinel (sentinel) process sends the INFO command to all Slave slave servers of the offline Master main server will be changed from once every 10 seconds to every once a second.
If there are not enough Sentinel (sentinel) processes to agree that the Master main server is offline, the objective offline status of the Master main server will be removed. If the Master main server sends a PING command to the Sentinel (sentinel) process again and returns a valid reply, the subjective offline status of the Master main server will be removed.
advantage
The sentry mode is based on the master-slave mode, and all the advantages of the master-slave mode are available in the sentry mode.
The master-slave can be switched automatically, the system is more robust and the availability is higher.
shortcoming
It is difficult to support online expansion, and online expansion will become very complicated when the cluster capacity reaches the upper limit.
If the slave node goes offline, sentinel will not fail over it, and the client connected to the slave node will not be able to obtain a new available slave node
cluster mode
Sentinel mode can basically achieve high availability and read-write separation, but in this mode, each Redis server stores the same data, which is a waste of memory, so the Cluster cluster mode is added to redis3.0 to realize the distribution of Redis storage, that is to say, different content is stored on each Redis node.
In this figure, each blue circle represents a redis server node. Any two of them are connected to each other. The client can connect to any node, and then can access any node in the cluster. access and other operations on it.
It is recommended to build a three-master and three-slave architecture for general clusters, where the three masters provide services and the three slaves provide backup functions.
Each node stores information about all master nodes and slave nodes of the cluster. All redis nodes are interconnected with each other (PING-PONG mechanism), internally using a binary protocol to optimize transmission speed and bandwidth.
principle
On each node of Redis, there are two things, one is a slot, and its value range is: 0-16383. Another is cluster, which can be understood as a cluster management plug-in. When our access key arrives, Redis will get a result according to the crc16 algorithm, and then calculate the remainder of the result to 16384, so that each key will correspond to a hash slot numbered between 0-16383, through This value, to find the node corresponding to the corresponding slot, and then directly and automatically jump to the corresponding node for access operations. Only master nodes will be assigned slots, slave nodes will not be assigned slots.
Cluster Election Principle
When the slave finds that its master has changed to a fail state, it tries to perform FailOver in order to become the new master. Since the hanged master has multiple slaves, these slaves have to compete to become the master node. The process is as follows:
Both slave1 and slave2 find that the state of the master they are connected to has changed to Fail;
they add 1 to the cluster currentEpoch (election cycle) recorded by themselves, and use the gossip protocol to broadcast FailOver_auth_request information;
other nodes receive messages from slave1 and salve2 (only the master responds ), judge the legitimacy of the request, and send FailOver_auth_ack to slave1 or slave2 (only one ack is sent for each epoch); in an election cycle, a master will only respond to the first slave that sends a message to it; slave1 collects and
returns FailOver_auth_ack, it becomes a new master after receiving more than half of the master's ack; (this is why the cluster needs at least 3 masters, if there are only two masters, after one of them hangs, only one master node is left Cannot be elected successfully)
The new master node broadcasts the Pong message to notify other cluster nodes, and there is no need to go to the election again.
Under what circumstances is the cluster unavailable?
If a master node and all slave nodes hang up, our cluster will enter the fail state.
If more than half of the master nodes are down, our cluster will also enter the fail state.
advantage
Dynamic expansion can be achieved
Automatic failover, data available during Slot migration
When encountering problems such as stand-alone memory, concurrency and traffic bottlenecks, the Cluster solution can be used to achieve load balancing
The client is directly connected to the Redis node without the need for an intermediate proxy layer. Customers The terminal does not need to connect to all nodes in the cluster, but can connect to any available node in the cluster.
shortcoming
The architecture is relatively new and there are few best practices
. In order to improve performance, the client needs to cache routing table information.
Multi-key operations are not supported, multi-key Redis transactions are not supported, and lua scripts are not supported. across nodes.