I. Background
- A cluster is a binary deploy
- Together properly after deployment is complete, a variety of resources to create an object can be used normally,
- It found that after application deployment can not be cross-node communication, and the pod's ip are 172.17.0.0 segment
Second, the investigation process layer
- View node routing, found docker0 card is actually 172.17.0.0 segment (what?)
- Find the following information: the CNM based docker deployment flanel, we need to /run/flannel/subnet.env as a docker environment variables, and specify the network information flannel at startup
Third, the solution (to modify the configuration file: /usr/lib/systemd/system/docker.service)
[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket [Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker EnvironmentFile=/run/flannel/subnet.env ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock ExecReload=/bin/kill -s HUP $MAINPID TimeoutSec=0 RestartSec=2 Restart=always # Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229. # Both the old, and new location are accepted by systemd 229 and up, so using the old location # to make them work for either version of systemd. StartLimitBurst=3 # Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230. # Both the old, and new name are accepted by systemd 230 and up, so using the old name to make # this option work for either version of systemd. StartLimitInterval=60s # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Comment TasksMax if your systemd version does not supports it. # Only systemd 226 and above support this option. TasksMax=infinity # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target
Call /run/flannel/subnet.env in DOCKER_NETWORK_OPTIONS specified network segment information of pod
Fourth, supplement
- CNI in, docker0 ip nothing to do with the Pod, Pod always generates time before going to apply for their own dynamic IP
- Under CNM mode, Pod network segment has already decided upon docker engine start
- CNI mode is recommended