K8s cluster installs StorageClass through NFS Subdirectory External Provisioner (Helm Chart method)

Kubernetes cluster installs StorageClass through NFS Subdirectory External Provisioner

basic introduction

The k8s cluster 1.26.4 is installed on the Centos7.9 machine based on Kubeadm. Install StorageClass through NFS Subdirectory External Provisioner (Helm Chart method). This article will introduce the relevant steps and precautions. NFS Subdirectory External Provisioner officially states that this method is suitable for Kubernetes >=1.9

Environment introduction

machine name	ip	system version	system kernel	cpu architecture
m1	10.11.81.152	CentOS Linux release 7.9	5.4.242-1.el7	x86
w1	10.11.81.153	CentOS Linux release 7.9	5.4.242-1.el7	x86
w2	10.11.81.154	CentOS Linux release 7.9	5.4.242-1.el7	x86
w3	10.11.81.155	CentOS Linux release 7.9	5.4.242-1.el7	x86
nfs	10.11.82.6	CentOS Linux release 7.9	3.10.0-1160.90.1	x86

1. Install the NFS server (operate on the nfs machine)

1.1 Install the software packages required by the NFS server

yum install -y nfs-utils

1.2 Set the NFS service to start at boot (the rpcbind service must be started first)

systemctl enable rpcbind
systemctl enable nfs

1.3 Start nfs service

systemctl start rpcbind
systemctl start nfs

1.4 If there is a firewall service, you need to open the rpc-bind and nfs services

firewall-cmd --zone=public --permanent --add-service={
    
    rpc-bind,mountd,nfs}
firewall-cmd --reload

2. Configure shared directory (operation on nfs machine)

2.1 After the NFS service is started, configure a shared directory on the server side

mkdir /nfs-share

2.2 Configure the directory shared by the NFS server

echo "/nfs-share    10.11.81.0/24(rw,sync,no_root_squash,no_all_squash) > /etc/exports

a./nfs-share: Shared directory location
b.10.11.81.0/24: Client IP range, * means all, ie no limit. If it is a production environment, it is recommended to strictly control the accessible IP
c.rw: permission setting, readable and writable
d.sync: synchronize shared directory
e.no_root_squash: root authorization can be used
f.no_all_squash: ordinary user authorization
2.3 can be used to make NFS configuration take effect

exportfs -r

2.4 Check the mount status

exportfs

3. Install Helm Chart (operate on m1 machine)

Official website
3.1 If you do not refer to the official website, directly execute the following command to install Helm Chart

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

#测试是否安装成功
helm version

4. Install nfs-utils on all working nodes (operate on w* nodes)

Encountered an error: Refer to the article
If this step is not performed, the following error will occur when starting nfs-installing the nfs-provisioner container in step 5 "Install nfs-provisioner":

mount: wrong fs type, bad option, bad superblock on 125.64.41.244:/data/img,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

I searched on the Internet and said that it was caused by not installing mount.nfs. It also means that mount.nfs needs to be installed on the nfs working node so that there will be no wrong fs type, bad option, bad superblock error prompts.

According to the error prompt, check the /sbin/mount. file, and it is found that there is no /sbin/mount.nfs file, just install nfs-utils.

#所有工作节点执行
yum install nfs-utils -y

5. Install nfs-provisioner (operate on m1 machine)

Official website reference

#添加repo
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner

helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=10.11.82.6 \   #nfs服务提供方机器
    --set nfs.path=/nfs-share \     #nfs服务提供方目录
    --set storageClass.name=nfs-client \  #Storage Class名称
    --set storageClass.defaultClass=true  #设置为默认Storage Class

Note: The nfs-subdir-external-provisioner image installed by default comes from registry.k8s.io. Domestic downloads may be limited. You can download related images from the following address:
https://download.csdn.net/download/weixin_46660849/87822652

#配置时可以指定镜像仓库
--set image.repository=<可以下载nfs-subdir-external-provisioner的本地或远程镜像仓库>

6. Test verification (operation on m1 machine)

6.1 Make sure the nfs-provisioner container starts correctly

#添加repo
kubectl get po
#看到如下输出
NAME                                              READY   STATUS    RESTARTS   AGE
nfs-subdir-external-provisioner-456ed7c90-acjn1   1/1     Running   0          2m

6.2 Verify that the pvc hangs on

cat <<EOF >  pvc-test.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-test
spec:
  storageClassName: "nfs-client"
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Mi
EOF