Recently, I have studied the clustering scheme of redis. The first scheme is to create a redis cluster, and the second scheme is to use the sentinel mode for master-slave replacement and fault recovery.
1. Sentinel introduction
Redis Sentinel
Sentinel (Sentinel) is a tool for monitoring the status of the Master in the redis cluster, which has been integrated in the redis2.4+ version
Sentinel functions:
1): Master status detection
2): If the Master is abnormal, the Master-Slave switch will be performed, one of the Slaves will be used as the Master, and the previous Master will be used as the Slave
3): After the Master-Slave switch, master_redis.conf, The content of slave_redis.conf and sentinel.conf will change, that is, there will be one more line of slaveof configuration in master_redis.conf, and the monitoring target of sentinel.conf will change the
working mode of Sentinel accordingly:
1): Each Sentinel will run once per second The frequency of sending a PING command to the Master, Slave and other Sentinel instances it knows
2): If an instance (instance) has been away from the last valid reply to the PING command more than the value specified by the down-after-milliseconds option, then this Instances will be marked as subjective offline by Sentinel.
3): If a Master is marked as subjective offline, all Sentinels monitoring the Master should confirm that the Master has indeed entered the subjective offline state at a frequency of once per second.
4): When a sufficient number of Sentinels (greater than or equal to the value specified in the configuration file) confirm that the Master has indeed entered the subjective offline state within the specified time range, the Master will be marked as an objective offline
5): Under normal circumstances , each Sentinel will send INFO commands to all known Masters and Slaves every 10 seconds
6): When the Master is marked as objectively offline by Sentinel, the frequency of sending INFO commands to all slaves of the offline Master by Sentinel will be changed from once every 10 seconds to once per second
7): If there are not enough Sentinels to agree that the Master has After going offline, the objective offline status of the Master will be removed.
If the Master returns a valid reply to Sentinel's PING command, the Master's subjective offline status will be removed.
Subjective offline and objective offline
Subjective offline: Subjectively Down, referred to as SDOWN, refers to the offline judgment made by the current Sentinel instance on a redis server.
Objective offline: Objectively Down, ODOWN for short, refers to the Master Server offline after multiple Sentinel instances make SDOWN judgments on the Master Server and communicate with each other through the SENTINEL is-master-down-by-addr command Judge, then open failover.
In layman's terms,
the sentinel system of redis is used to manage multiple redis servers, and can implement a functionally HA cluster. The system mainly performs three tasks:
①Monitoring (Monitoring): Redis Sentinel monitors the running status of the master server and the slave server in real time.
②Notification: When there is a problem with a Redis server being monitored, Redis Sentinel can send a notification to the system administrator, or send notifications to other programs through the API
. The architecture diagram of a simple master-slave structure plus sentinel cluster is as follows :
The above picture shows one master and one slave node, plus two clusters where Sentinel is deployed. Sentinel clusters will communicate with each other, communicate the status of redis nodes, make corresponding judgments and deal with them. The subjective offline status here and the objective offline state are more important states, they determine whether to perform failover or not.
You can subscribe to the specified channel information. When the server fails, the administrator is notified.
The client can regard Sentinel as a service that only provides subscription functions. Redis server, you can not use PUBLISH command to send information to this server, but you can use SUBSCRIBE command or PSUBSCRIBE command to get corresponding event alerts by subscribing to a given channel.
A channel can receive events with the same name as the channel. For example, a channel named +sdown can receive events for all instances to enter the subjective downline (SDOWN) state.
2. Build a redis-sentinel cluster environment
1. Create a new directory redis-sentinel under /usr/local/, and then create six directories under this directory: 7501/ 7502/ 7503/ 7504/ 7505/ 7506/.
2. Copy the reids.conf in the redis installation directory to the first four directories and name them:
Redis-7501.conf redis-7502.conf redis-7503.conf redis-7504.conf
Modify the content of the configuration file ( Take redis-7501.conf as an example):
daemonize yes
Port 7501
Bind 192.168.12.90
logfile "./redis-7501.log"- 1
- 2
- 3
- 4
3. Copy the sentinel.conf in the redis installation directory to the 7505/ and 7506/ directories and name them:
Sentinel-7505.conf sentinel-7506.conf
Modify the configuration file (take sentinel-7505.conf as an example):
port 7505
sentinel monitor mymaster 192.168.12.90 7501 2- 1
- 2
Note: We will start four redis instances later, in which redis with port 7501 is set to master, and the other three are set to slave. So my mymaster is followed by the ip and port of the master. The last '2' means I want to start. As long as there are 2 sentinels that think the master is offline, the master is considered to be offline objectively, and failover is started and a new master is elected. Usually the last parameter cannot be more than the number of sentinel instances to start.
4. Start redis and sentinel
to start 4 redis instances respectively:
redis-server redis-7501.conf
...- 1
- 2
Then log in to the three instances of 7502, 7503, and 7504, and dynamically change the master-slave relationship to become the slave of 7501:
redis-cli -h 192.168.12.90 -p 7502
192.168.12.90:7502> SLAVEOF 192.168.12.90 7501- 1
- 2
Start two sentinels in background startup mode:
redis-sentinel sentinel-7505.conf &- 1
5. Some commands of sentinel
To use the commands of sentinel, we need to use the redis-cli command to enter sentinel:
redis-cli -h 192.168.12.90 -p 7505- 1
①
Basic status information of INFO sentinel ②
SENTINEL masters
lists all monitored master servers, and the current status of these master servers
③ SENTINEL slaves
lists all slave servers of a given master server, and the current status of these slave servers
④ SENTINEL get-master -addr-by-name
returns the IP address and port number of the master server with the given name
⑤SENTINEL reset
resets all master servers whose names match the given pattern pattern. The reset operation clears all current state of the master, including failovers in progress, and removes all slaves and Sentinels of the master that have been discovered and associated so far.
⑥SENTINEL failover
When the primary server fails, it will force an automatic failover to start without asking other Sentinels for their opinions, but it will send a newest configuration to other sentinels, and other sentinels will be updated according to this configuration
6. Test:
(1) Log in to the master:
redis-cli -h 192.168.12.90 -p 7501
192.168.12.90:7501> set name "zhangsan"
[root@localhost redis-sentinel]# redis-cli -h 192.168.12.90 -p 7502
192.168.12.90:7502> get name
"zhangsan"
192.168.12.90:7502> set age 24
(error) READONLY You can't write against a read only slave.- 1
- 2
- 3
- 4
- 5
- 6
- 7
It can be seen that in our master-slave mode, the slave is read-only by default.
(2) At present, 7501 is the master. After we forcefully kill the process of 7501, we can see the information sent by sentinel:
It can be seen that sentinel has promoted the redis instance 7504 to the new master, and later started the instance 7501 as the slave of 7504, thus manually restoring the redis cluster.