本文档介绍部署高可用 kube-controller-manager 集群的步骤。
该集群包含 3 个节点，启动后将通过竞争选举机制产生一个 leader 节点，其它节点为阻塞状态。当 leader 节点不可用后，剩余节点将再次进行选举产生新的 leader 节点，从而保证服务的可用性。
以下是非认证的的配置文件（用127.0.0.1连接kube-apiserver）：
cat > /lib/systemd/system/kube-controller-manager.service <<"EOF" [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/usr/local/bin/kube-controller-manager \ --address=127.0.0.1 \ --master=http://127.0.0.1:8080 \ --allocate-node-cidrs=true \ --service-cluster-ip-range=10.254.0.0/16 \ --cluster-cidr=172.30.0.0/16 \ --cluster-name=kubernetes \ --leader-elect=true \ --cluster-signing-cert-file=/etc/kubernetes/ca/ca.pem \ --cluster-signing-key-file=/etc/kubernetes/ca/ca-key.pem \ --service-account-private-key-file=/etc/kubernetes/ca/ca-key.pem \ --root-ca-file=/etc/kubernetes/ca/ca.pem \ --v=2 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target EOF
这个简单很多，但是以后可能会取消，如果要使用认证授权的方式看下面：
------------------------------------------------------------------
为保证通信安全，本文档先生成 x509 证书和私钥，kube-controller-manager 在如下两种情况下使用该证书：
与 kube-apiserver 的安全端口通信时;
在安全端口(https，10252) 输出 prometheus 格式的 metrics；
创建 kube-controller-manager 证书和私钥
cat > kube-controller-manager-csr.json <<EOF { "CN": "system:kube-controller-manager", "key": { "algo": "rsa", "size": 2048 }, "hosts": [ "127.0.0.1", "192.168.111.10", "192.168.111.11", "192.168.111.12" ], "names": [ { "C": "CN", "ST": "ChongQing", "L": "ChongQing", "O": "system:kube-controller-manager", "OU": "yunwei" } ] } EOF cfssl gencert -ca=/etc/kubernetes/ca/ca.pem \ -ca-key=/etc/kubernetes/ca/ca-key.pem \ -config=/etc/kubernetes/ca/ca-config.json \ -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
创建和分发 kubeconfig 文件
kubeconfig 文件包含访问 apiserver 的所有信息，如 apiserver 地址、CA 证书和自身使用的证书；
kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/ca/ca.pem \
--embed-certs=true \
--server=https://192.168.111.9:8443 \
--kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/ca/kube-controller-manager.pem \
--client-key=/etc/kubernetes/ca/kube-controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-context system:kube-controller-manager \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=kube-controller-manager.kubeconfig
kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig
分发 kubeconfig 到所有 master 节点：
# scp kube-controller-manager.kubeconfig 192.168.111.11:/etc/kubernetes/
# scp kube-controller-manager.kubeconfig 192.168.111.12:/etc/kubernetes/
创建和分发 kube-controller-manager systemd unit 文件：
cat > /lib/systemd/system/kube-controller-manager.service <<"EOF" [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] ExecStart=/usr/local/bin/kube-controller-manager \ --port=0 \ --secure-port=10252 \ --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \ --service-cluster-ip-range=10.254.0.0/16 \ --cluster-signing-cert-file=/etc/kubernetes/ca/ca.pem \ --cluster-signing-key-file=/etc/kubernetes/ca/ca-key.pem \ --root-ca-file=/etc/kubernetes/ca/ca.pem \ --service-account-private-key-file=/etc/kubernetes/ca/ca-key.pem \ --leader-elect=true \ --feature-gates=RotateKubeletServerCertificate=true \ --controllers=*,bootstrapsigner,tokencleaner \ --horizontal-pod-autoscaler-use-rest-clients=true \ --horizontal-pod-autoscaler-sync-period=10s \ --tls-cert-file=/etc/kubernetes/ca/kube-controller-manager.pem \ --tls-private-key-file=/etc/kubernetes/ca/kube-controller-manager-key.pem \ --use-service-account-credentials=true \ --alsologtostderr=true \ --logtostderr=false \ --log-dir=/var/log/kubernetes \ --v=2 Restart=on Restart=on-failure RestartSec=5 User=k8s [Install] WantedBy=multi-user.target EOF
-------------------------------------------------
启动kube-controller-manager
# systemctl daemon-reload&&for SERVICES in kube-controller-manager;do systemctl enable $SERVICES; systemctl start $SERVICES; systemctl status $SERVICES; done
查看输出的 metric
注意：以下命令在 kube-controller-manager 节点上执行。
kube-controller-manager 监听 10252 端口，接收 https 请求：
# ss -lnpt|grep kube-controll
# curl http://127.0.0.1:10252/metrics |head
测试 kube-controller-manager 集群的高可用
停掉一个或两个节点的 kube-controller-manager 服务，观察其它节点的日志，看是否获取了 leader 权限。
查看当前的 leader
kubectl get endpoints kube-controller-manager --namespace=kube-system -o yaml