kubernetes一步一步搭建（二）https访问

上一篇介绍http方式连接到集群， http 方式是不安全，如果内部使用还可以，若是部署到外部则需要使用 https 增加安全性。下面介绍如何在 node2 上采用 https 方式。

上面说到/var/lib/kubelet/kubeconfig文件，并非手动创建，而是通过命令行kubectl config进行配置，配置完成之后就会自动生成。

简单介绍一下ApiServer认证，认证一共有三种方式：

1）Https双向认证，是双向认证啊，不是单向认证（最安全）。

2）Http Token认证

3）Http Base认证，用户名和密码

我们之前介绍http方式是没有任何认证措施的，也就是说只要能访问master的主机都可以与其进行通信。特别说明：kubectl命令行工具既同时支持CA双向认证也支持简单认证（http base或者token）两种模式与apiserver进行通信，但其他组件只能配置成一种模式。

下面开始进行各类证书的生成以及kubeconfig文件的生成。

证书生成，可采用openssl，也可以采用CFSSL工具。下面这篇博客，采用的是CFSSL工具：http://www.cnblogs.com/netsa/p/8126155.html

我比较熟悉openssl，因此介绍openssl使用方式

一、生成各类证书

0）环境配置

[root@localhost ~]# mkdir kube-ca
[root@localhost kube-ca]#
[root@localhost kube-ca]# mkdir -p ./{certs,private,newcerts}
[root@localhost kube-ca]# touch ./index.txt  
[root@localhost kube-ca]# echo 01 > ./serial
[root@localhost kube-ca]#

修改openssl配置文件，主要是扩展x509，设置多ip。

[root@localhost kube-ca]# vi /etc/pki/tls/openssl.cnf
[ CA_default ]
#dir = /etc/pki/CA             # Where everything is kept
dir = /etc/kubernetes/kube-ca  # 重新指定目录
certs = $dir/certs             # Where the issued certs are kept
crl_dir = $dir/crl             # Where the issued crl are kept
database = $dir/index.txt      # database index file.
#unique_subject = no           # Set to 'no' to allow creation of
                               # several ctificates with same subject.
new_certs_dir = $dir/newcerts  # default place for new certs.
certificate = $dir/cacert.pem  # The CA certificate
serial = $dir/serial           # The current serial number
crlnumber = $dir/crlnumber     # the current crl number
                               # must be commented out to leave a V1 CRL
crl = $dir/crl.pem             # The current CRL
private_key = $dir/private/cakey.pem    # The private key
RANDFILE = $dir/private/.rand           # private random number file
x509_extensions = usr_cert     # 扩展x509_extension是usr_cert项

[usr_cer]
subjectAltName = @alt_names

#扩展多个IP
[alt_names]
IP.1 = 127.0.0.1
IP.2 = 192.168.1.105
IP.3 = 192.63.63.1
IP.4 = 192.63.63.20

1）生成CA根证书

生成https私钥

[root@localhost kube-ca]#
[root@localhost kube-ca]# openssl genrsa -out private/ca.key 2048
Generating RSA private key, 2048 bit long modulus
..................................+++
.......................................................................+++
e is 65537 (0x10001)
[root@localhost kube-ca]#

生成https证书

[root@localhost kube-ca]# openssl req -new -x509 -key private/ca.key -out certs/ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:BeiJing
Locality Name (eg, city) [Default City]:BeiJing
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:mykubeca.io
Email Address []:[email protected]
[root@localhost kube-ca]#

其中Common Name 是随意指定mykubeca.io。

2）生成apiserver证书

生成服务端私钥

[root@localhost kube-ca]# mkdir apiserver
[root@localhost kube-ca]# openssl genrsa -out apiserver/apiserver.key 2048
Generating RSA private key, 2048 bit long modulus
...............+++
................+++
e is 65537 (0x10001)
[root@localhost kube-ca]#

生成服务端https证书，其中CommonName可以和CA中的CommonName不同，一般情况下，CommonName为服务的域名（也可以是ip，hostname）。

[root@localhost kube-ca]# openssl req -new -key apiserver/apiserver.key -out apiserver/apiserver.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:BeiJing
Locality Name (eg, city) [Default City]:BeiJing
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:mykube.io
Email Address []:[email protected]
 

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@localhost kube-ca]#
[root@localhost kube-ca]#

CA证书进行签名，这一步非常重要，一定要执行。

[root@localhost kube-ca]# openssl ca -in apiserver/apiserver.csr -keyfile ./private/ca.key -cert ./certs/ca.crt -out apiserver/apiserver.crt
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Mar  4 05:04:56 2018 GMT
            Not After : Mar  4 05:04:56 2019 GMT
        Subject:
            countryName               = CN
            stateOrProvinceName       = BeiJing
            organizationName          = Default Company Ltd
            commonName                = mykube.io
            emailAddress              = [email protected]
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Comment:
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier:
                FC:5C:32:61:B3:A1:0C:F8:94:FE:D0:C1:4C:56:D2:C6:39:61:00:B5
            X509v3 Authority Key Identifier:
                keyid:9A:FA:EE:26:A6:59:D6:F8:01:52:2C:15:17:63:A6:85:8F:88:DE:11

            X509v3 Subject Alternative Name:
                IP Address:127.0.0.1, IP Address:192.168.1.105, IP Address:192.169.122.215, IP Address:192.169.122.1
Certificate is to be certified until Mar  4 05:04:56 2019 GMT (365 days)
Sign the certificate? [y/n]:y
 
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost kube-ca]#

3）生成客户端证书

客户端的证书生成和服务端类似，只需要保证客户端的CommonName与服务端的一致即可。当然也是需要进行CA签名，否则后面会报错的。此处不再描述生成过程。

1.2 部署证书以及kubeconfig文件

1.2.1 修改master节点

1)拷贝根证书、服务端秘钥和证书

[root@localhost kube-ca]# mkdir /etc/kubernetes/ca
[root@localhost kube-ca]# cp certs/ca.crt  apiserver/apiserver.key  apiserver/apiserver.crt /etc/kubernetes/ca
[root@localhost kube-ca]#

2)修改/etc/kubernetes/apiserver配置文件,在KUBE_API_ARGS中增加如下配置：

--client-ca-file=/etc/kubernetes/ca/ca.crt --tls-private-key-file=/etc/kubernetes/ca/apiserver.key --tls-cert-file=/etc/kubernetes/ca/apiserver.crt

重启apiserver，apiserver默认监听端口是6443端口，通过curl进行校验：

[root@localhost kube-ca] curl https://192.63.63.1:6443/api/v1/nodes --cert /etc/kubernetes/kube-ca/client/client.crt --key /etc/kubernetes/kube-ca/client/client.key --cacert /etc/kubernetes/kube-ca/certs/ca.crt -v

如果能正常显示数据，则认为证书配置成功，否则证书配置失败。

1.2.2 修改node节点

1）Node节点，一般运行kubelet、kube-proxy两个组件，为了方便二者使用同一份客户端证书。将ca根证书，client私钥和证书拷贝到node2中/etc/kubernets/ca目录中。

2）创建kebeconfig文件，依次执行一下命令，会生成两个文件kubelet.kubeconfig和kube-proxy.kubeconfig

export KUBE_APISERVER="https://192.63.63.1:6443"
kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/ca/ca.crt \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kubelet.kubeconfig

kubectl config set-credentials kubelet \
  --client-certificate=/etc/kubernetes/ca/client.crt \
  --client-key=/etc/kubernetes/ca/client.key \
  --kubeconfig=kubelet.kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet \
  --kubeconfig=kubelet.kubeconfig

kubectl config use-context default --kubeconfig=kubelet.kubeconfig

kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/ca/ca.crt \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy \
  --client-certificate=/etc/kubernetes/ca/client.crt \
  --client-key=/etc/kubernetes/ca/client.key \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=kube-proxy \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

将文件kubele.kubeconfig拷贝到/var/lib/kubelet以及文件kube-proxy.kubeconfig拷贝到/var/lib/kube-proxy中，如果没有上述目录则创建。

3）修改配置文件，

修改/etc/kubernetes/kubelet文件，在KUBELET_ARGS中增加--kubeconfig=/var/lib/kubelet/kubelet.kubeconfig

修改修改/etc/kubernetes/proxy文件，在KUBELET_ARGS中增加--kubeconfig=/var/lib/kubelet/kube-proxy.kubeconfig

重新启动kubelet和kube-proxy服务，然后在master节点中，查看nodes信息:

[root@localhost ~]#
[root@localhost ~]# kubectl get nodes
NAME        STATUS    ROLES     AGE       VERSION
127.0.0.1   Ready     <none>    17d       v1.9.1
node1       Ready     <none>    5m        v1.9.1
node2       Ready     <none>    1m        v1.9.1
[root@localhost ~]#

下面是kubelet.kubeconfig和kube-proxy.kubeconfig文件内容如下：

[root@localhost kube-proxy]# cat /var/lib/kubelet/kubelet.kubeconfig
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/ca/ca.crt
    server: https://192.169.122.1:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubelet
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: kubelet
  user:
    as-user-extra: {}
    client-certificate: /etc/kubernetes/ca/client.crt
    client-key: /etc/kubernetes/ca/client.key
[root@localhost kube-proxy]#

[root@localhost kube-proxy]# cat /var/lib/kube-proxy/kube-proxy.kubeconfig
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/ca/ca.crt
    server: https://192.169.122.1:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kube-proxy
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: kube-proxy
  user:
    as-user-extra: {}
    client-certificate: /etc/kubernetes/ca/client.crt
    client-key: /etc/kubernetes/ca/client.key
[root@localhost kube-proxy]#

二、遇到问题

问题1：
[root@localhost controller]# curl https://127.0.0.1:443/api/v1/nodes --cert /etc/kubernetes/kube-ca/client/client.crt --key /etc/kubernetes/kube-ca/client/client.key --cacert /etc/kubernetes/kube-ca/certs/ca.crt -v
* About to connect() to 127.0.0.1 port 443 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/kubernetes/kube-ca/certs/ca.crt
CApath: none
* NSS error -12190 (SSL_ERROR_PROTOCOL_VERSION_ALERT)
* Peer reports incompatible or unsupported protocol version.
* Closing connection 0

curl: (35) Peer reports incompatible or unsupported protocol version.

解决方法：

更新以下软件：
yum update nss nss-util nspr

yum update curl

问题2：之前通过kubectl get nodes提示无法找到master之类错误（具体是啥错误不清楚了）

解决方式1：kubectl get nodes --kubeconfig=XXX 指定kubeconfig文件，可以参考kubelet的文件

解决方式2：kubectl默认读取~/.kube/config,这个config文件里面有设置server地址。其实这个config文件就是kubeconfig文件。

至此，https方式访问介绍完成，后面介绍token和basic方式。

kubernetes一步一步搭建（二）https访问

猜你喜欢