Kubernetes (k8s) deploys a highly available multi-master and multi-slave Redis cluster

Kubernetes deploys a highly available multi-master and multi-slave Redis cluster

Environmental preparation
- Prepare for Kubernetes
- Prepare storage class
Deploy redis

Environmental preparation

Prepare for Kubernetes

First you need a Kubernetes cluster, as shown in the figure I have installed a Kubernetes cluster:
Insert image description here

If you don’t have a Kubernetes cluster yet, you can refer to the article I wrote:https://blog.csdn.net/m0_51510236/article/details/130842122

Prepare storage class

You also need a dynamically provisioned storage class. I have previously written an article about installing NFS dynamically provisioned storage classes:https://blog.csdn. net/m0_51510236/article/details/132641343 (the first half of this article), as shown in the picture, I have installed an NFS storage class (named nfs-client):
Insert image description here

Deploy redis

I have previously written an article about installing a highly available multi-master and multi-slave redis cluster on a physical machine:https://blog.csdn.net/m0_51510236/article/ details/132684529, this article refers to the steps to install the cluster on the physical machine.

Prepare a namespace

In order to be close to the production environment, we first create a namespace:

Command to create

kubectl create namespace deploy-test

yaml file creation (recommended)

apiVersion: v1
kind: Namespace
metadata:
  name: deploy-test
spec: {
    
    }
status: {
    
    }

Prepare redis configuration file

First we need a Redis configuration file. I have prepared it. Please note that the name is redis.conf:

# 关闭保护模式
protected-mode no

# 日志级别
loglevel warning

# 日志存放目录
logfile "/data/redis.log"

# 数据文件目录
dir /data
dbfilename dump.rdb

# 数据库数量
databases 16

# 保存数据库到数据文件
save 900 1
save 300 10
save 60 10000

# 集群相关配置
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000

First copy this file to the server, as shown in the figure:
Insert image description here

Then we need to create a ConfigMap. We can directly generate the yaml resource list file of ConfigMap through the following command:

kubectl create configmap redis-cluster-config --from-file=redis.conf -n deploy-test --dry-run=client -o yaml

The following figure is generated:
Insert image description here

Because we may deploy this application to other clusters in real deployment, we remember to save this configuration file:

apiVersion: v1
data:
  redis.conf: |
    # 关闭保护模式
    protected-mode no

    # 日志级别
    loglevel warning

    # 日志存放目录
    logfile "/data/redis.log"

    # 数据文件目录
    dir /data
    dbfilename dump.rdb

    # 数据库数量
    databases 16

    # 保存数据库到数据文件
    save 900 1
    save 300 10
    save 60 10000

    # 集群相关配置
    cluster-enabled yes
    cluster-config-file nodes-6379.conf
    cluster-node-timeout 15000
kind: ConfigMap
metadata:
  name: redis-cluster-config
  namespace: deploy-test

We save this content to a file, such as redis-deploy.yaml:
Insert image description here

Prepare the resource manifest file for deploying statefulset

I have prepared the resource list file. Here are the detailed instructions:

# 创建一个Redis部署要用到的Service
apiVersion: v1
kind: Service
metadata:
  name: deploy-redis-svc
  namespace: deploy-test
  labels:
    app: redis
spec:
  ports:
  - port: 6379
    name: redis
    targetPort: 6379
    nodePort: 30379
  selector:
    app: redis
  type: NodePort
  sessionAffinity: ClientIP

---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: deploy-redis
  namespace: deploy-test
spec:
  selector:
    matchLabels:
      app: redis
  # 引用上面定义的Service
  serviceName: "deploy-redis-svc"
  # 注意这里部署六个节点，按照本篇文章的逻辑最好不要修改
  replicas: 6
  template:
    metadata:
      labels:
        app: redis
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - command:
      	# 这里指定使用那个配置文件启动redis-server
        - "redis-server"
        - "/usr/local/etc/redis.conf"
        name: redis
        # 因为redis6集群对域名支持不好，并且Kubernetes是需要通过域名连接各个pod的，所以我们采用redis7来完成这次部署
        # 如果你能拉取到DockerHub上的镜像那么你就使用这个
        # image: docker.io/library/redis:7.0.12
        # 如果你拉取不到，那么就使用我拉取并推送到国内阿里云的镜像
        image: registry.cn-shenzhen.aliyuncs.com/xiaohh-docker/redis:7.0.12
        ports:
        - containerPort: 6379
          name: redis
        volumeMounts:
        # 挂载redis的数据卷，使用的是我在下面声明的存储类模版
        - name: redis-data
          mountPath: /data
        # 挂载redis的配置文件
        - name: redis-config
          mountPath: /usr/local/etc
          readOnly: true
      volumes:
      # 读取configmap，获取redis的配置文件，方便上面挂载这个配置文件
      - name: redis-config
        configMap:
          name: redis-cluster-config
          items:
          - key: redis.conf
            path: redis.conf
  # 定义存储类模版
  volumeClaimTemplates:
  - metadata:
      name: redis-data # 这个名字要与上面的对应
      namespace: deploy-test
    spec:
      accessModes:
      - ReadWriteMany # 设置多节点读写模式
      resources:
        requests:
          storage: 1Gi # 申请1个g的存储空间，可根据自己的服务器空间修改
      storageClassName: nfs-client # 这里填写你存储类的名字

Complete resource list file content:

apiVersion: v1
kind: Namespace
metadata:
  name: deploy-test
spec: {
    
    }
status: {
    
    }

---

apiVersion: v1
data:
  redis.conf: |
    # 关闭保护模式
    protected-mode no

    # 日志级别
    loglevel warning

    # 日志存放目录
    logfile "/data/redis.log"

    # 数据文件目录
    dir /data
    dbfilename dump.rdb

    # 数据库数量
    databases 16

    # 保存数据库到数据文件
    save 900 1
    save 300 10
    save 60 10000

    # 集群相关配置
    cluster-enabled yes
    cluster-config-file nodes-6379.conf
    cluster-node-timeout 15000
kind: ConfigMap
metadata:
  name: redis-cluster-config
  namespace: deploy-test

---

apiVersion: v1
kind: Service
metadata:
  name: deploy-redis-svc
  namespace: deploy-test
  labels:
    app: redis
spec:
  ports:
  - port: 6379
    name: redis
    targetPort: 6379
    nodePort: 30379
  selector:
    app: redis
  type: NodePort
  sessionAffinity: ClientIP

---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: deploy-redis
  namespace: deploy-test
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: "deploy-redis-svc"
  replicas: 6
  template:
    metadata:
      labels:
        app: redis
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - command:
        - "redis-server"
        - "/usr/local/etc/redis.conf"
        name: redis
        # image: docker.io/library/redis:7.0.12
        image: registry.cn-shenzhen.aliyuncs.com/xiaohh-docker/redis:7.0.12
        ports:
        - containerPort: 6379
          name: redis
        volumeMounts:
        - name: redis-data
          mountPath: /data
        - name: redis-config
          mountPath: /usr/local/etc
          readOnly: true
      volumes:
      - name: redis-config
        configMap:
          name: redis-cluster-config
          items:
          - key: redis.conf
            path: redis.conf
  volumeClaimTemplates:
  - metadata:
      name: redis-data
      namespace: deploy-test
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 1Gi
      storageClassName: nfs-client

Execute the file to complete the deployment

We can use this line of command to execute this file to complete the deployment:

kubectl apply -f redis-deploy.yaml

As shown in the figure after execution:
Insert image description here

Its deployment can be monitored with this line of command:

watch kubectl get all -o wide -n deploy-test

See that all 6 pods have been successfully run:
Insert image description here

Initialize the cluster

The pod we deployed is a statefulset type pod, so the domain name format for accessing the pod ispod名字.service名字.命名空间.svc.cluster.local, so the addresses of the six redis we access in the cluster are:

deploy-redis-0.deploy-redis-svc.deploy-test.svc.cluster.local:6379
deploy-redis-1.deploy-redis-svc.deploy-test.svc.cluster.local:6379
deploy-redis-2.deploy-redis-svc.deploy-test.svc.cluster.local:6379
deploy-redis-3.deploy-redis-svc.deploy-test.svc.cluster.local:6379
deploy-redis-4.deploy-redis-svc.deploy-test.svc.cluster.local:6379
deploy-redis-5.deploy-redis-svc.deploy-test.svc.cluster.local:6379

Next, we need to go to any pod inside and execute a line of command. Use this line of command to enter the first pod:

# 注意修改你自己的命名空间和pod名字
kubectl exec -itn deploy-test pod/deploy-redis-0 bash

As shown below after entering:
Insert image description here

Then we execute the following line of command:

redis-cli --cluster create --cluster-replicas 1 \
deploy-redis-0.deploy-redis-svc.deploy-test.svc.cluster.local:6379 \
deploy-redis-1.deploy-redis-svc.deploy-test.svc.cluster.local:6379 \
deploy-redis-2.deploy-redis-svc.deploy-test.svc.cluster.local:6379 \
deploy-redis-3.deploy-redis-svc.deploy-test.svc.cluster.local:6379 \
deploy-redis-4.deploy-redis-svc.deploy-test.svc.cluster.local:6379 \
deploy-redis-5.deploy-redis-svc.deploy-test.svc.cluster.local:6379

This prompt will appear after execution:
Insert image description here

It asks us if such an allocation method is feasible. Here, enter yes directly and press Enter. Next, when you see the following screen, it means that the cluster configuration is successful:
Insert image description here

We next use this line of command to enter the cluster:

# -c 的意思是进入集群
redis-cli -c

Then we enter this line of command to view the nodes in the cluster:

CLUSTER NODES

You can see that it is a redis cluster with three masters and three nodes:
Insert image description here

At this point, building a redis high-availability cluster on Kubernetes is complete.