The Kubernetes API Server is the core of the entire Kubernetes cluster . Not only do we need to access the API Server from outside the cluster, but sometimes we also need to access the API Server from within the Pod.
However, in a production environment, the Kubernetes API Server is "fortified". In the article " Security Configuration of Kubernetes Clusters ", I mentioned that Kubernetes authenticates client requests through methods such as client cert, static token, and basic auth . For a Process running in a Pod, sometimes these methods are appropriate, but sometimes, information such as client cert, static token or basic auth is not convenient to expose to the Process in the Pod. And through these methods, the requests authenticated by the API Server are fully authorized, and the Kubernetes cluster can be operated arbitrarily , which obviously cannot meet the security requirements. For this reason, Kubernetes recommends that you use the service account solution. This article will show you in detail how to access the API Server from a Pod through a service account.
Zero, test environment
The test environment of this article is Kubernetes 1.3.7 cluster, two nodes, and the master bears the load. The cluster is built through kube-up.sh . For the specific building method, see " An article to guide you about Kubernetes installation ".
1. What is a service account?
What is a service account? As the name implies, compared to a user account (for example, when kubectl accesses the APIServer, the user account is used), the service account is the account used by the Process in the Pod to access the Kubernetes API. It provides a process for the Process in the Pod. Identity. Compared with the global permissions of user accounts, service accounts are more suitable for some lightweight tasks and focus more on authorization to be used by Processes in certain Pods.
The service account exists as a resource in the Kubernetes cluster. We can obtain the list of service acounts in the current cluster through kubectl:
# kubectl get serviceaccount --all-namespaces
NAMESPACE NAME SECRETS AGE
default default 1 140d
kube-system default 1 140d
Let's check the details of the service account named "default" under the kube-system namespace:
# kubectl describe serviceaccount/default -n kube-system
Name: default
Namespace: kube-system
Labels: <none>
Image pull secrets: <none>
Mountable secrets: default-token-hpni0
Tokens: default-token-hpni0
We see that the service account is not complicated, it just associates a secret resource as a token, which is also called service-account-token, and this token really works in the API Server authentication process:
# kubectl get secret -n kube-system
NAME TYPE DATA AGE
default-token-hpni0 kubernetes.io/service-account-token 3 140d
# kubectl get secret default-token-hpni0 -o yaml -n kube-system
apiVersion: v1
data:
ca.crt: {base64 encoding of ca.crt data}
namespace: a3ViZS1zeXN0ZW0=
token: {base64 encoding of bearer token}
kind: Secret
metadata:
annotations:
kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: 90ded7ff-9120-11e6-a0a6-00163e1625a9
creationTimestamp: 2016-10-13T08:39:33Z
name: default-token-hpni0
namespace: kube-system
resourceVersion: "2864"
selfLink: /api/v1/namespaces/kube-system/secrets/default-token-hpni0
uid: 90e71909-9120-11e6-a0a6-00163e1625a9
type: kubernetes.io/service-account-token
We see that the secret resource of type service-account-token contains three parts: ca.crt, namespace and token.
-
ca.crt
is the CA public key certificate of the API Server , which is used by the Process in the Pod to verify the server-side digital certificate of the API Server; -
namespace
This is the base64 encoding of the value of the namespace where the Secret is located: # echo -n “kube-system”|base64 => “a3ViZS1zeXN0ZW0=” -
token
这是一段用API Server私钥签发(sign)的bearer tokens的base64编码，在API Server authenticating环节，它将派上用场。
二、API Server的service account authentication(身份验证)
前面说过，service account为Pod中的Process提供了一种身份标识，在Kubernetes的身份校验(authenticating)环节，以某个service account提供身份的Pod的用户名为：
system:serviceaccount:(NAMESPACE):(SERVICEACCOUNT)
以上面那个kube-system namespace下的“default” service account为例，使用它的Pod的username全称为：
system:serviceaccount:kube-system:default
有了username，那么credentials呢？就是上面提到的service-account-token中的token。在《Kubernetes集群的安全配置》一文中我们谈到过，API Server的authenticating环节支持多种身份校验方式：client cert、bearer token、static password auth等，这些方式中有一种方式通过authenticating（Kubernetes API Server会逐个方式尝试），那么身份校验就会通过。一旦API Server发现client发起的request使用的是service account token的方式，API Server就会自动采用signed bearer token方式进行身份校验。而request就会使用携带的service account token参与验证。该token是API Server在创建service account时用API server启动参数：–service-account-key-file的值签署(sign)生成的。如果–service-account-key-file未传入任何值，那么将默认使用–tls-private-key-file的值，即API Server的私钥（server.key）。
通过authenticating后，API Server将根据Pod username所在的group：system:serviceaccounts和system:serviceaccounts:(NAMESPACE)的权限对其进行authority 和admission control两个环节的处理。在这两个环节中，cluster管理员可以对service account的权限进行细化设置。
三、默认的service account
Kubernetes会为每个cluster中的namespace自动创建一个默认的service account资源，并命名为”default”：
# kubectl get serviceaccount --all-namespaces
NAMESPACE NAME SECRETS AGE
default default 1 140d
kube-system default 1 140d
如果Pod中没有显式指定spec.serviceAccount字段值，那么Kubernetes会将该namespace下的”default” service account自动mount到在这个namespace中创建的Pod里。我们以namespace “default”为例，我们查看一下其中的一个Pod的信息：
# kubectl describe pod/index-api-2822468404-4oofr
Name: index-api-2822468404-4oofr
Namespace: default
... ...
Containers:
index-api:
... ...
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-40z0x (ro)
Environment Variables: <none>
... ...
Volumes:
... ...
default-token-40z0x:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-40z0x
QoS Class: BestEffort
Tolerations: <none>
No events.
可以看到，kubernetes将default namespace中的service account “default”的service account token挂载(mount)到了Pod中容器的/var/run/secrets/kubernetes.io/serviceaccount路径下。
深入容器内部，查看mount的serviceaccount路径下的结构：
# docker exec 3d11ee06e0f8 ls /var/run/secrets/kubernetes.io/serviceaccount
ca.crt
namespace
token
这三个文件与上面提到的service account的token中的数据是一一对应的。
四、default service account doesn’t work
上面提到过，每个Pod都会被自动挂载一个其所在namespace的default service account，该service account用于该Pod中的Process访问API Server时使用。Pod中的Process该怎么用这个service account呢？Kubernetes官方提供了一个client-go项目可以为你演示如何使用service account访问API Server。这里我们就基于client-go项目中的examples/in-cluster/main.go来测试一下是否能成功访问API Server。
先下载client-go源码：
# go get k8s.io/client-go
# ls -F
CHANGELOG.md dynamic/ Godeps/ INSTALL.md LICENSE OWNERS plugin/ rest/ third_party/ transport/ vendor/
discovery/ examples/ informers/ kubernetes/ listers/ pkg/ README.md testing/ tools/ util/
我们改造一下examples/in-cluster/main.go，考虑到panic会导致不便于观察Pod日志，我们将panic改为输出到“标准输出”，并且不return，让Pod周期性的输出相关日志，即便fail：
// k8s.io/client-go/examples/in-cluster/main.go
... ...
func main() {
// creates the in-cluster config
config, err := rest.InClusterConfig()
if err != nil {
fmt.Println(err)
}
// creates the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
fmt.Println(err)
}
for {
pods, err := clientset.CoreV1().Pods("").List(metav1.ListOptions{})
if err != nil {
fmt.Println(err)
} else {
fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))
}
time.Sleep(10 * time.Second)
}
}
基于该main.go的go build默认输出，创建一个简单的Dockerfile：
From ubuntu:14.04
MAINTAINER Tony Bai <[email protected]>
COPY main /root/main
RUN chmod +x /root/main
WORKDIR /root
ENTRYPOINT ["/root/main"]
构建一个测试用docker image：
# docker build -t k8s/example1:latest .
... ...
# docker images|grep k8s
k8s/example1 latest ceb3efdb2f91 14 hours ago 264.4 MB
创建一份deployment manifest：
//main.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: k8s-example1
spec:
replicas: 1
template:
metadata:
labels:
run: k8s-example1
spec:
containers:
- name: k8s-example1
image: k8s/example1:latest
imagePullPolicy: IfNotPresent
我们来创建该deployment（kubectl create -f main.yaml -n kube-system），观察Pod中的main程序能否成功访问到API Server：
# kubectl logs k8s-example1-1569038391-jfxhx
the server has asked for the client to provide credentials (get pods)
the server has asked for the client to provide credentials (get pods)
API Server log(/var/log/upstart/kube-apiserver.log):
E0302 15:45:40.944496 12902 handlers.go:54] Unable to authenticate the request due to an error: crypto/rsa: verification error
E0302 15:45:50.946598 12902 handlers.go:54] Unable to authenticate the request due to an error: crypto/rsa: verification error
E0302 15:46:00.948398 12902 handlers.go:54] Unable to authenticate the request due to an error: crypto/rsa: verification error
出错了！kube-system namespace下的”default” service account似乎不好用啊！（注意：这是在kubernetes 1.3.7环境）。
五、创建一个新的自用的service account
在kubernetes github issues中，有好多issue是关于”default” service account不好用的问题，给出的解决方法似乎都是创建一个新的service account。
service account的创建非常简单，我们创建一个serviceaccount.yaml：
//serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: k8s-example1
创建该service account：
# kubectl create -f serviceaccount.yaml
serviceaccount "k8s-example1" created
# kubectl get serviceaccount
NAME SECRETS AGE
default 1 139d
k8s-example1 1 12s
修改main.yaml，让Pod显示使用这个新的service account：
//main.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: k8s-example1
spec:
replicas: 1
template:
metadata:
labels:
run: k8s-example1
spec:
serviceAccount: k8s-example1
containers:
- name: k8s-example1
image: k8s/example1:latest
imagePullPolicy: IfNotPresent
好了，我们重新创建该deployment，查看Pod日志：
# kubectl logs k8s-example1-456041623-rqj87
There are 14 pods in the cluster
There are 14 pods in the cluster
... ...
我们看到main程序使用新的service account成功通过了API Server的身份验证环节，并获得了cluster的相关信息。
六、尾声
在我的另外一个使用kubeadm安装的k8s 1.5.1环境中，我重复做了上面这个简单测试，不同的是这次我直接使用了default service account。在k8s 1.5.1下，pod的执行结果是ok的，也就是说通过default serviceaccount，我们的client-go in-cluster example程序可以顺利通过API Server的身份验证，获取到相关的Pods元信息。
七、参考资料
http://tonybai.com/2017/03/03/access-api-server-from-a-pod-through-serviceaccount/