NoSQL Redis cluster deployment
Article Directory
Introduction to Redis cluster
Redis Cluster is Redis cluster, which is
a distributed storage solution officially launched by Redis in version 3.0 . It is completely decentralized and consists of multiple nodes, all of which are interconnected with each other. The Redis
client can directly connect to any node to obtain the key-value pairs in the cluster without an intermediate agent. If the node does not have the key-value specified by the user, it will automatically redirect the client to the node where the key-value is located.The Redis cluster is a mesh structure, and each node is connected to each other node through a TCP connection. In a cluster with N nodes, each node has N-1 outgoing
TCP connections and N-1 incoming connections, and these TCP connections will be kept permanently.
Data partition
The Redis cluster will distribute user data to each node, breaking the maximum storage capacity of single-machine Redis memory. The cluster introduces the concept of hash slot slots. After the construction is completed,
16384 hash slot slots will be generated . At the same time, 16384 hash slots will be roughly evenly
mapped to different nodes according to the number of nodes. When the user stores the key-value, the cluster will first perform a CRC16 check on the key and then
take a modulus of 16384 to determine which slot the key-value is placed in, so as to automatically split the data to different nodes.
Data redundancy
Redis
cluster supports master-slave replication and failure recovery. The cluster uses a master-slave replication model, and each master node master should have at least one slave node slave. Assuming that a master node fails, all its child nodes will broadcast a data packet to other master nodes in the cluster to request votes. Once a slave node receives a response from the majority of the master nodes, it will win the election and be elected The master node is responsible for processing the hash slots that the old master node was responsible for.
One: Deploy Redis cluster
1.1: Deploy Redis on the master-slave server
According to the internal failover implementation principle of Redis Cluster, Redis cluster requires at least 3 master nodes, and each master node has at least 1 slave node, so a cluster contains at least 6 nodes, three masters and three slaves, and they are deployed on different machines. on.
Due to limited conditions, we can only create a pseudo cluster on two
//主从服务器的部署都是一样的，在此仅展示主服务器的部署
[root@master1 ~]# yum install gcc gcc-c++ make -y //安装环境组件编译器
[root@master1 redis]# tar zxvf redis-5.0.7.tar.gz -C /opt //redis源码包可以直接到官网下载
[root@master1 redis]# cd /opt/redis-5.0.7/
[root@master1 redis-5.0.7]# make '//直接进行make'
[root@master1 redis-5.0.7]# make PREFIX=/usr/local/redis/ install //指定redis目录并安装
[root@master1 redis-5.0.7]# cd utils/'//回到redis源码包解压目录'
[root@master1 utils]# ./install_server.sh '//执行脚本进行配置'
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379] '//选择redis默认接口，直接回车'
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf] '//选择redis默认配置文件名称，直接回车'
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log] '//选择默认redis日志文件名称，直接回车'
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379] '//选择默认接口的默认数据文件，直接回车'
Selected default - /var/lib/redis/6379
Please select the redis executable path [] /usr/local/redis/bin/redis-server '//选择redis可执行文件路径，需要手动输入此路径：/usr/local/redis/bin/redis-server'
Selected config: '//选择的配置清单展示'
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/redis/bin/redis-server
Cli Executable : /usr/local/redis/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort. '//直接回车完成配置'
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!
[root@master1 utils]# ln -s /usr/local/redis/bin/* /usr/local/bin '//将redis命令创建软连接，便于系统识别'
[root@master1 utils]# netstat -ntap |grep 6379
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN
1.2: Master-slave server file modification
[root@master1 utils]# vim /etc/redis/6379.conf //修改主配置文件
#bind 127.0.0.1 '//注释第70行的监听127地址 表示监听所有地址
protected-mode no '//去掉第89行注释关闭安全保护'
port 6379 '//去掉第93行注释,开启端口6379'
daemonize yes '//去掉第137行注释,以独立进程启动'
cluster-enabled yes '//去掉第833行注释,开启群集功能 注意顶行写'
cluster-config-file nodes-6379.conf '//去掉第841行注释,群集名称文件设置'
cluster-node-timeout 15000 '//去掉第847行注释,群集超时时间设置'
appendonly yes '//去掉第700行注释,开启aof持久化'
[root@master1 utils]# cd /var/lib/redis/6379/
[root@master1 6379]# /etc/init.d/redis_6379 restart '//重启redis服务'
[root@master1 6379]# ls
appendonly.aof dump.rdb nodes-6379.conf '//生成了三个文件，appendonly.aof是AOF持久化文件，dump.rdb是RDB快照文件，nodes-6379.conf是节点首次启动生成的配置文件'
1.3: Install rvm and RUBY control set software on the main server
[root@master01 opt]# gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
gpg: 已创建目录‘/root/.gnupg’
gpg: 新的配置文件‘/root/.gnupg/gpg.conf’已建立
gpg: 警告：在‘/root/.gnupg/gpg.conf’里的选项于此次运行期间未被使用
gpg: 钥匙环‘/root/.gnupg/secring.gpg’已建立
gpg: 钥匙环‘/root/.gnupg/pubring.gpg’已建立
gpg: 下载密钥‘D39DC0E3’，从 hkp 服务器 keys.gnupg.net
'//把脚本文件拷贝到opt'
[root@master01 opt]# rz -E
rz waiting to receive.
[root@master01 opt]# ls
redis-5.0.7 redis-5.0.7.tar.gz rh rvm-installer.sh shuai.sh
'//增加执行权限'
[root@master01 opt]# chmod +x rvm-installer.sh
'//执行脚本'
[root@master01 opt]# ./rvm-installer.sh
'//执行环境变量'
[root@master01 opt]# source /etc/profile.d/rvm.sh
'//列出Ruby可安装的版本'
[root@master01 opt]# rvm list known
'//安装Ruby2.4.10'
[root@master01 opt]# rvm install 2.4.10
'//使用Ruby2.4.10版本'
rvm user 2.4.10
'//再次安装Redis'
gem install redis
1.4: The master-slave server adds two network cards by itself (simulating 6 nodes)
//查看新增网卡
[root@master01 opt]# ifconfig
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 20.0.0.8 netmask 255.255.255.0 broadcast 20.0.0.255
inet6 fe80::45d1:cf4f:e061:4ff1 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:84:c1:ed txqueuelen 1000 (Ethernet)
RX packets 317 bytes 39238 (38.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 171 bytes 22632 (22.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 20.0.0.9 netmask 255.255.255.0 broadcast 20.0.0.255
inet6 fe80::10b1:3e47:969:95 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:84:c1:f7 txqueuelen 1000 (Ethernet)
RX packets 122 bytes 22029 (21.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 6456 (6.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
//关闭防火墙重启服务
[root@master01 opt]# systemctl restart network
[root@master01 opt]# systemctl stop firewalld.service
[root@master01 opt]# setenforce 0
[root@localhost opt]# ifconfig
'//查看网卡信息'
ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 20.0.0.10 netmask 255.255.255.0 broadcast 20.0.0.255
inet6 fe80::b6a4:8fa8:6f93:b5fa prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:bd:4f:6d txqueuelen 1000 (Ethernet)
RX packets 94 bytes 15273 (14.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 6496 (6.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 20.0.0.11 netmask 255.255.255.0 broadcast 20.0.0.255
inet6 fe80::7d35:a3bf:dc95:8809 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:bd:4f:63 txqueuelen 1000 (Ethernet)
RX packets 85 bytes 13072 (12.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 6496 (6.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
'//重启网卡'
[root@localhost ~]# service network restart
Restarting network (via systemctl): [ 确定 ]
[root@localhost ~]# systemctl stop firewalld.service
[root@localhost ~]# setenforce 0
1.5: Create a cluster on the master server
'//配置监听所有的端口'
[root@master01 opt]# vim /etc/redis/6379.conf
bind 20.0.0.41 20.0.0.8 20.0.0.9
'//重启服务'
[root@master01 opt]# /etc/init.d/redis_6379 restart
'//'
[root@master01 opt]# redis-cli --cluster create 20.0.0.41:6379 20.0.0.8:6379 20.0.0.9:6379 20.0.0.42:6379 20.0.0.10:6379 20.0.0.11:6379 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 20.0.0.10:6379 to 20.0.0.41:6379
Adding replica 20.0.0.11:6379 to 20.0.0.8:6379
Adding replica 20.0.0.42:6379 to 20.0.0.9:6379
M: 0bf9a12ac900d640c37b4dca4beecdb9bf71392d 20.0.0.41:6379
slots:[0-5460] (5461 slots) master
M: 0bf9a12ac900d640c37b4dca4beecdb9bf71392d 20.0.0.8:6379
slots:[5461-10922] (5462 slots) master
M: 0bf9a12ac900d640c37b4dca4beecdb9bf71392d 20.0.0.9:6379
slots:[10923-16383] (5461 slots) master
S: fa928bf59976325d9cd19a5bb64b629acf8c5c89 20.0.0.42:6379
replicates 0bf9a12ac900d640c37b4dca4beecdb9bf71392d
S: fa928bf59976325d9cd19a5bb64b629acf8c5c89 20.0.0.10:6379
replicates 0bf9a12ac900d640c37b4dca4beecdb9bf71392d
S: fa928bf59976325d9cd19a5bb64b629acf8c5c89 20.0.0.11:6379
replicates 0bf9a12ac900d640c37b4dca4beecdb9bf71392d
Can I set the above configuration? (type 'yes' to accept): yes '//输入yes'
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
>>> Performing Cluster Check (using node 20.0.0.41:6379)
M: 0bf9a12ac900d640c37b4dca4beecdb9bf71392d 20.0.0.41:6379
slots:[0-16383] (16384 slots) master
1 additional replica(s)
S: fa928bf59976325d9cd19a5bb64b629acf8c5c89 20.0.0.42:6379
slots: (0 slots) slave
replicates 0bf9a12ac900d640c37b4dca4beecdb9bf71392d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
//登录
[root@master01 opt]# redis-cli -c -h 20.0.0.41
20.0.0.41:6379> quit
[root@master01 opt]# redis-cli -c -h 20.0.0.8
20.0.0.8:6379> quit
[root@master01 opt]# redis-cli -c -h 20.0.0.9
//群集验证
[root@master01 opt]# redis-cli -c -h 20.0.0.41 '//登录到14的数据库'
20.0.0.41:6379> set name lisi '//创建一个键值'
OK
20.0.0.41:6379> get name '//查看name键的值'
"lisi"
20.0.0.41:6379> exit '//退出'
[root@master01 opt]# redis-cli -c -h 20.0.0.8
20.0.0.8:6379> keys *
1) "name"
20.0.0.8:6379> get name '//查看name键的值'
"lisi"
[root@master01 opt]# redis-cli -c -h 20.0.0.9 '//登录到9的数据库'
20.0.0.9:6379> keys * '//查看键'
1) "name"
20.0.0.9:6379> get name '//查看键的值'
"lisi"
[root@master01 opt]# redis-cli -c -h 20.0.0.42 '//登录从服务器的42数据库'
20.0.0.42:6379> keys *
1) "name"
20.0.0.42:6379> get name
-> Redirected to slot [5798] located at 20.0.0.41:6379 '//可以看出从主服务器41数据库读写的'
"lisi"
[root@master01 opt]# redis-cli -c -h 20.0.0.10 '//可以看出主从的数据都是同步的'
20.0.0.10:6379> keys *
1) "name"
20.0.0.10:6379> get name
-> Redirected to slot [5798] located at 20.0.0.41:6379
"lisi"
20.0.0.41:6379> exit
[root@master01 opt]# redis-cli -c -h 20.0.0.11
20.0.0.11:6379> keys *
1) "name"
20.0.0.11:6379> get name
-> Redirected to slot [5798] located at 20.0.0.41:6379
"lisi"
20.0.0.41:6379> '//你会发现自动切换到41数据库'
//设置主服务器41数据库失效时间为50s
20.0.0.41:6379> expire name 50
(integer) 1
20.0.0.41:6379> ttl name '//查看失效时间'
(integer) 44
20.0.0.41:6379> ttl name
(integer) 41
20.0.0.41:6379> keys * '//查看键值已经没了'
(empty list or set)
//登录从服务器42数据库
[root@master01 opt]# redis-cli -c -h 20.0.0.42
20.0.0.42:6379> keys * '//可以看到数据已经没了'
(empty list or set)