Briefly
There are bottlenecks in the memory and performance of single-machine redis. In order to support massive data, it is necessary to use multi-master mode (horizontal expansion), that is, redis cluster. Redis cluster is a cluster implemented by slicing technology, which supports one master and multiple slaves, separation of read and write, and automatic failover, etc., so as to achieve its high availability.
advantage
(1) Automatically shard the data, and put a part of the data on each master
(2) Provide built-in high-availability support. When some masters are unavailable, they can continue to work. (Automatic master-slave replication, read-write separation [by default all reads and writes are completed by the master, cluster mode does not have the concept of so-called read-write separation], and failover [master-standby switching], supports hash slots of multiple masters, supports data distributed storage)
(3) Intelligent automatic switch mount relationship:
a. If some masters are mounted with multiple (>1) slaves, other nodes with relatively few slaves are down, and the slaves with more mounts will automatically switch to the nodes with less slaves.
b. The nodes in the cluster will automatically join the cluster after restarting (the priority becomes slave)
port
In the redis cluster architecture, each redis needs to release two port numbers, such as 6379, and the other is to add a port number of 10000, such as 16379. The 16379 port number is used for communication between nodes, that is, the cluster bus something, the cluster bus. The communication of the cluster bus is used for fault detection, configuration update, and failover authorization. The cluster bus uses another binary protocol, which is mainly used for efficient data exchange between nodes, occupying less network bandwidth and processing time.
algorithm
Redis cluster has a fixed 16384 hash slots, calculate the CRC16 value for each key, and then modulo 16384 to obtain the hash slot corresponding to the key. Each master in the redis cluster will hold some slots. For example, if there are 3 masters, each master may hold more than 5000 hash slots. Hash slots make it easy to add and remove nodes. To add a master, move the hash slots of other masters to the past. If you reduce a master, move its hash slots to other masters. The cost of moving hash slots is very high. low.
test environment
3 servers: 192.168.0.200, 192.168.0.201, 192.168.0.202
Each service starts two redis instances, the ports are: 7000, 7001
Create the following directory structure in the installation directory:
${redis}/cluster/
7000/
conf
data
logs
7001/
conf
data
logs
Start command (installation directory execution):
./bin/redis-server ./cluster/7000/conf/7000.conf
./bin/redis-server ./cluster/7001/conf/7001.conf
Configuration (redis.conf i.e. 7000.conf and 7001.conf)
bind 192.168.0.200
port 7001
daemonize yes
pidfile /var/run/redis_7001.pid
logfile "/usr/local/redis/cluster/7001/logs/redis.log"
dir "/usr/local/redis/cluster/7001/data"
appendonly yes
#Open the cluster cluster
cluster-enabled yes
#Configuration data for inter-cluster communication storage
cluster-config-file "/usr/local/redis/cluster/7001/data/nodes-7001.conf"
#Intercluster communication timeout
cluster-node-timeout 15000
Firewall (add the following port filtering)
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7000 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17000 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7001 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17001 -j ACCEPT
Copy the execution (ruby) script in the installation package to the bin of the installation directory
cp /usr/local/src/redis3.0/src/redis-trib.rb ./bin/
Install ruby environment online
yum install ruby
yum install rubygems
#Install redis-trib.rb to run the dependent ruby package redis-3.3.0.gem (upload "redis-3.3.0.gem" to /usr/local/src/)
gem install /usr/local/src/redis-3.3.0.gem
Command (executed by any server)
#Create a cluster (--replicas 1 means 1 slave is required)
./bin/redis-trib.rb create --replicas 1 192.168.0.200:7000 192.168.0.200:7001 192.168.0.201:7000 192.168.0.201:7001 192.168.0.202:7000 192.168.0.202:7001
#Result (select yes to agree to the assignment)
>>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 192.168.0.202:7000 192.168.0.201:7000 192.168.0.200:7000 Adding replica 192.168.0.201:7001 to 192.168.0.202:7000 Adding replica 192.168.0.202:7001 to 192.168.0.201:7000 Adding replica 192.168.0.200:7001 to 192.168.0.200:7000 M: f66e9c8f8f794c38679082f513955e7495b471ae 192.168.0.200:7000 slots:10923-16383 (5461 slots) master S: 7dd73f9e205f9db9b0896ba05a5ff3e6ccd7728c 192.168.0.200:7001 replicates f66e9c8f8f794c38679082f513955e7495b471ae M: 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd 192.168.0.201:7000 slots:5461-10922 (5462 slots) master S: 25890bb055947b74b4910f8a83c9a534fc7f824c 192.168.0.201:7001 replicates b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f M: b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f 192.168.0.202:7000 slots:0-5460 (5461 slots) master S: ca8767ec8451872b75a2659f568c9f55c6915f54 192.168.0.202:7001 replicates 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join........ >>> Performing Cluster Check (using node 192.168.0.200:7000) M: f66e9c8f8f794c38679082f513955e7495b471ae 192.168.0.200:7000 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f 192.168.0.202:7000 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 7dd73f9e205f9db9b0896ba05a5ff3e6ccd7728c 192.168.0.200:7001 slots: (0 slots) slave replicates f66e9c8f8f794c38679082f513955e7495b471ae S: 25890bb055947b74b4910f8a83c9a534fc7f824c 192.168.0.201:7001 slots: (0 slots) slave replicates b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f S: ca8767ec8451872b75a2659f568c9f55c6915f54 192.168.0.202:7001 slots: (0 slots) slave replicates 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd M: 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd 192.168.0.201:7000 slots:5461-10922 (5462 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
#Check cluster status (check any node)
./bin/redis-trib.rb check 192.168.0.200:7000
#result
>>> Performing Cluster Check (using node 192.168.0.200:7000) M: f66e9c8f8f794c38679082f513955e7495b471ae 192.168.0.200:7000 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f 192.168.0.202:7000 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 7dd73f9e205f9db9b0896ba05a5ff3e6ccd7728c 192.168.0.200:7001 slots: (0 slots) slave replicates f66e9c8f8f794c38679082f513955e7495b471ae S: 25890bb055947b74b4910f8a83c9a534fc7f824c 192.168.0.201:7001 slots: (0 slots) slave replicates b7a80819050d5edc2c7a38f3d0ae1dfb4dc1267f S: ca8767ec8451872b75a2659f568c9f55c6915f54 192.168.0.202:7001 slots: (0 slots) slave replicates 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd M: 11ebb1c83ad68a747a9cb1adf0d7be6c1929b7bd 192.168.0.201:7000 slots:5461-10922 (5462 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
#View cluster information (info can be any node)
./bin/redis-trib.rb info 192.168.0.202:7000
result:
192.168.0.202:7000 (b7a80819...) -> 3 keys | 5461 slots | 1 slaves. 192.168.0.200:7000 (f66e9c8f...) -> 1 keys | 5461 slots | 1 slaves. 192.168.0.201:7000 (11ebb1c8...) -> 1 keys | 5462 slots | 1 slaves. [OK] 5 keys in 3 masters. 0.00 keys per slot on average.
#Add a new master node to the cluster (192.168.0.203:7000, new node; 192.168.0.201:7001, any node in the cluster)
./bin/redis-trib.rb add-node 192.168.0.203:7000 192.168.0.201:7001
#Re-sharding (192.168.0.201:7001, any node in the cluster. The newly added master node slot is empty, you need to manually allocate the slot and migrate the relevant slot data. Assign (16384/master node number) such a slot, similar to the average Allocated, you can also allocate according to your actual situation.)
./bin/redis-trib.rb reshard 192.168.0.201:7001
#Add a new slave node to the cluster (establish a master-slave relationship), master-id is the node id to be added to the master master node, 192.168.0.203:7001 is the newly added slave node, and 192.168.0.201:7001 is any arbitrary node in the cluster The node is used to identify which cluster it is; if the master node --master-id is not given, redis-trib will randomly add new slave nodes to the master node with fewer slave nodes.
./bin/redis-trib.rb add-node --slave --master-id 1751e5a8a9cf9ac9bad349b3d0949107541a3c22 192.168.0.203:7001 192.168.0.201:7001
#Remove the master node (192.168.0.201:7001, any node in the cluster, 1751e5a8a9cf9ac9bad349b3d0949107541a3c22 node id of the master node, before removing the node, you need to transfer the slot on the node. After the master node is removed, the slave node hanging on it will be Prioritize and automatically select some nodes without slaves for automatic mounting)
./bin/redis-trib.rb del-node 192.168.0.201:7001 1751e5a8a9cf9ac9bad349b3d0949107541a3c22
#Remove the slave node (same command as removing the master node)
./bin/redis-trib.rb del-node 192.168.0.201:7001 38fcf92ee9a0d9b5f8e30c2fdf0ee6884cdbb00e
#redis-cli client operation is different from before
#1. The slave node cannot directly read the currently cached data, it needs to execute the readonly command first
192.168.0.202:7001> get name (error) MOVED 5798 192.168.0.201:7000 192.168.0.202:7001> readonly OK 192.168.0.202:7001> get name "zhangsan"
#2. The master node set data error (the slot corresponding to the key is not on the current node and needs to be transferred. Automatic redirection can be achieved through the redis-cli -c parameter)
192.168.0.201:7000> set age 12 (error) MOVED 741 192.168.0.202:7000