suse 12 Binary Deployment Kubernetets 1.19.7-Chapter 03-Deploy the flannel plugin

1.3, deploy flannel network

• 所有节点all needflannel

1.3.0, download the flannel binary file

k8s-01:~ # cd /opt/k8s/packages/
k8s-01:/opt/k8s/packages # mkdir flannel
k8s-01:/opt/k8s/packages # wget https://github.com/coreos/flannel/releases/download/v0.12.0/flannel-v0.12.0-linux-amd64.tar.gz
k8s-01:/opt/k8s/packages # tar xf flannel-v0.12.0-linux-amd64.tar.gz -C /opt/k8s/packages/flannel/

1.3.1, create flannel certificate and private key

k8s-01:~ # cd /opt/k8s/ssl/
k8s-01:/opt/k8s/ssl # cat > flanneld-csr.json <<EOF
{
    
    
  "CN": "flanneld",
  "hosts": [
  ],
  "key": {
    
    
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
    
    
      "C": "CN",
      "ST": "ShangHai",
      "L": "ShangHai",
      "O": "k8s",
      "OU": "bandian"
    }
  ]
}
EOF

1.3.2, generate flannel certificate and private key

k8s-01:/opt/k8s/ssl # cfssl gencert -ca=/opt/k8s/ssl/ca.pem \
-ca-key=/opt/k8s/ssl/ca-key.pem \
-config=/opt/k8s/ssl/ca-config.json \
-profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld

1.3.3, write pod network segment to etcd

k8s-01:~ # cd /opt/k8s/ssl/
k8s-01:/opt/k8s/ssl # source /opt/k8s/bin/k8s-env.sh
k8s-01:/opt/k8s/ssl # ETCDCTL_API=2 etcdctl \
--endpoints=${ETCD_ENDPOINTS} \
--ca-file=/opt/k8s/ssl/ca.pem \
--cert-file=/opt/k8s/ssl/flanneld.pem \
--key-file=/opt/k8s/ssl/flanneld-key.pem \
mk ${FLANNEL_ETCD_PREFIX}/config '{"Network":"'${CLUSTER_CIDR}'", "SubnetLen": 21, "Backend": {"Type": "vxlan"}}'

• Because the current version of flannel does 0.12.0not support it etcd v3, you need to use the etcd v2API to write the configuration, otherwise the written key will not be found when you start flanneld later

1.3.4, configure flannel for systemctl management

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # source /opt/k8s/bin/k8s-env.sh
k8s-01:/opt/k8s/conf # cat > flanneld.service << EOF
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
[Service]
Type=notify
ExecStart=/opt/k8s/bin/flanneld \\
  -etcd-cafile=/etc/kubernetes/cert/ca.pem \\
  -etcd-certfile=/etc/flanneld/cert/flanneld.pem \\
  -etcd-keyfile=/etc/flanneld/cert/flanneld-key.pem \\
  -etcd-endpoints=${ETCD_ENDPOINTS} \\
  -etcd-prefix=${FLANNEL_ETCD_PREFIX} \\
  -iface=${IFACE} \\
  -ip-masq
ExecStartPost=/opt/k8s/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=always
RestartSec=5
StartLimitInterval=0
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service
EOF

• mk-docker-opts.sh The script writes the Pod subnet information allocated to flanneld into the /run/flannel/docker file, and then uses the environment variables in this file to configure the docker0 bridge when docker starts
• flanneld interfaces communicate with other nodes where the system default route for node has a plurality of network interfaces (e.g., network and the public network), and can -ifacespecify a communication interface parameters
• -ip-masq flanneld sets SNAT rules for traffic outside the access Pod network, and at the same time sets the variable -ip-masq passed to Docker (in the /run/flannel/docker file) to false, so that Docker will no longer create SNAT rules;
  • When Docker's -ip-masq is true, the created SNAT rule is more "violent": all requests initiated by the node's Pod to access non-docker0 interfaces will be SNATed, so that the request source IP for accessing other node's Pods will be set to flannel .1 The IP of the interface, the destination Pod cannot see the real source Pod IP.
  • The SNAT rules created by flanneld are relatively mild, and only do SNAT for requests to access non-Pod network segments.

1.3.5. Distribute the flannel certificate and startup files to all nodes

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh

for host in ${NODE_IPS[@]}
do
    printf "\e[1;34m${host}\e[0m\n"
    ssh root@${host} "mkdir -p /etc/flanneld/cert"
    scp /opt/k8s/ssl/flanneld*.pem ${host}:/etc/flanneld/cert/
    scp /opt/k8s/packages/flannel/{
    
    flanneld,mk-docker-opts.sh} ${host}:/opt/k8s/bin/
    scp /opt/k8s/conf/flanneld.service ${host}:/etc/systemd/system/
done

1.3.6, configure and start the flannel service

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh

for host in ${NODE_IPS[@]}
do
    printf "\e[1;34m${host}\e[0m\n"
    ssh root@${host} "systemctl daemon-reload && \
                      systemctl enable flanneld && \
                      systemctl restart flanneld && \
                      systemctl status flanneld | grep Active"
done

1.3.7, view the list of allocated pod network segments

k8s-01:~ # ETCDCTL_API=2 etcdctl \
--endpoints=${ETCD_ENDPOINTS} \
--ca-file=/etc/kubernetes/cert/ca.pem \
--cert-file=/etc/flanneld/cert/flanneld.pem \
--key-file=/etc/flanneld/cert/flanneld-key.pem \
ls ${FLANNEL_ETCD_PREFIX}/subnets

1.3.8, check whether each node has a flannel network card

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh

for host in ${NODE_IPS[@]}
do
    printf "\e[1;34m${host}\e[0m\n"
    ssh root@${host} "ip a | grep flannel | grep -w inet"
done