[Cloud Native--Kubernetes] Security Mechanism

1. Security mechanism

As a management tool for distributed clusters, Kubernetes is an important task to ensure the security of the cluster. The API Server is an intermediary for the communication of various components within the cluster, and it is also the entrance of external control. Therefore, the security mechanism of Kubernetes is basically designed around protecting the API Server.

The api server is the entrance of the k8s cluster, and there are two ports by default:

• Local port 8080: used to receive HTTP requests, no external services, non-authenticated or authorized HTTP requests access API Server through this port
• Secure port 6443: Used to receive HTTPS requests for authentication and authorization, and provide external services.

Users need to pass three levels to access the k8s api server through the secure port: authentication, authorization, and access control

• Authentication authentication: used to identify user identity
• Authorization authorization: Confirm whether the resource has relevant permissions
• Admission Control: to determine whether the operation meets the requirements of the cluster
  

2. Authentication

2.1 Authentication method

• HTTP Token authentication : use a token to identify legitimate users.
  HTTP Token authentication is a way to express customers with a very long token string that is encoded in a special way and is difficult to imitate. Token is a very long and complex string, and each Token corresponds to a user name and is stored in a file that the API Server can access. When the client initiates an API call request, it needs to put Token in the HTTP Header.

• HTTP Base authentication : Authenticate the user name and password by means of username and password
  . The string encoded by the BASE64 algorithm is placed in the Heather Authorization field in the HTTP Request and sent to the server. After receiving it, the server decodes it and obtains the user name and password.

• HTTPS certificate authentication (the strictest): The client identity authentication method based on the signature of the CA root certificate.

Note: Token authentication and Base authentication can only be used for server-to-clientone-way authentication, and the client does not know whether the server is legal; and the HTTPS certificate authentication method can realizetwo-way authentication。

1. Access types that need to be authenticated:

• Access of Kubernetes components to API Server: kubectl, kubelet, kube-proxy
• Pod managed by Kubernetes accesses API Server: Pod (coredns, dashborad also run as Pod)

2. Security note:

• Controller Manager, Scheduler and API Server are on the same machine, so directly use the API Server's non-secure port access (such as port 8080)
• When kubectl, kubelet, and kube-proxy access the API Server, certificates are required for HTTPS two-way authentication, and the port number is 6443

3. Certificate issued:

• Manual issuance: When using binary deployment, you need to manually issue an HTTPS certificate with the CA
• Automatic issuance: When the kubelet accesses the API Server for the first time, it uses the token for authentication. After passing, the Controller Manager will generate a certificate for the kubelet, and the subsequent accesses will use the certificate for authentication.

4. kubeconfig
   The kubeconfig file contains cluster parameters (CA certificate, API Server address), client parameters (certificate and private key generated above), cluster context parameters (cluster name, user name). Kubenetes components (such as kubelet, kube-proxy) can switch to different clusters and connect to apiserver by specifying different kubeconfig files at startup.
   That is to say, the kubeconfig file is not only a description of a cluster , but also a filling of cluster authentication information . Contains the access method and authentication information of the cluster . The kubectl file is located at ~/.kube/config by default
5. Service Account
   Service Account is to facilitate the container in the Pod to access the API Server. Because Pods are created and destroyed dynamically, it is not feasible to manually generate certificates for each Pod. Kubenetes uses Service Account to recycle authentication, thus solving the authentication problem of Pod accessing API Server.
6. The relationship between Secret and SA
   Kubernetes has designed a resource object called Secret, which is divided into two categories:

• The service-account-token used to save the ServiceAccount
• Opaque for storing user-defined confidential information

2.2 Service Account（SA）

There are three parts in Service Account:

• Token: It is the serial number of the Token string signed with the private key of the API Server, which is used for server-side authentication when accessing the API Server
• ca.crt: CA root certificate, used by the client to verify the certificate sent by the API server
• namespace: Identify the role domain name space of this service-account-token

By default, each namespace will have a Service Account. If the Pod does not specify a Service Account when it is created, the Service Account of the namespace to which the Pod belongs will be used. Each Pod will automatically set spec.serviceAccount to default after creation (unless another Service Accout is specified).

kubectl get sa -A

After each Pod starts, it will mount the Token, ca.crt, and namespace of the ServiceAccount to /var/run/secrets/kubernetes.io/serviceaccount/

kubectl get pod -n kube-system

kubectl exec -it kube-proxy-556vt -n kube-system sh

3. Authorization

The authentication (Authentication) process is just to confirm that both parties in the communication have confirmed that the other party is trustworthy and can communicate with each other. Authentication, on the other hand, is the authority to determine which resources the requester has.
API Server currently supports the following authorization strategies:

1. AlwaysDeny : Indicates that all requests are rejected, generally used for testing
2. AlwaysAllow : All requests are allowed to be received. If the cluster does not require an authorization process, this strategy can be used, generally used for testing
3. ABAC (Attribute-Based Access Control): Attribute-based access control means matching and controlling user requests using authorization rules configured by users. That is to say, define an attribute of access type, and users can use this attribute to access the corresponding resource. This way of setting is more cumbersome, and each setting needs to define a long list of attributes.
4. Webhook : By calling the external REST service to authorize the user, K8S can be authenticated outside the cluster
5. RBAC(Role-Based Access Control): Role-based access control , K8S uses rules by default since version 1.6

3.1 RBAC

• Compared with other access control methods, RBAC has the following advantages:

1. Complete coverage of resources (Pod, Deployment, Service) and non-resources (meta information or resource status) in the cluster
2. The whole RBAC is completely completed by several API resource objects. Like other API resource objects, it can be operated with kubectl or API
3. Can be adjusted at runtime without restarting the API Server, while ABAC requires restarting the API Server

• RBAC's API resource object description
  RBAC introduces four new top-level resource objects: Role, ClusterRole, RoleBinding, and ClusterRoleBinding . All four object types can be operated by kubectl and API Server.

Official documentation : https://kubernetes.io/docs/reference/access-authn-authz/rbac/

• Role
  : Authorize the resource control authority of the specified namespace
  ClusterRole: Authorize the resource control authority of all namespaces
  If you use RoleBinding to bind ClusterRole, it will still be affected by the namespace; if you use ClusterRoleBinding to bind ClusterRole, it will affect the entire K8S cluster.

• Role binding
  RoleBinding: bind the role to the subject (ie subject)
  ClusterRoleBinding: bind the cluster role to the subject

• Subject (subject)
  User: user
  Group: user group
  ServiceAccount: service account

User is represented by a string, and its prefix system: is reserved by the system. The cluster administrator should ensure that ordinary users do not use this prefix format; the
writing format of Group is the same as that of User, and the system: prefix is ​​also reserved for the system.
When Pod uses ServiceAccount authentication, the JWT in service-account-token will save user information. With the user information, create a pair of role/role binding (cluster role/cluster role binding) resource objects to complete the permission binding.

• Role and ClusterRole
  In the RBAC API, Role represents a set of rule permissions. The permissions can only be increased (cumulative permissions). There is no operation that a resource has many permissions at the beginning and is reduced through RBAC. That is to say, there are only whitelist permissions, but no concept of blacklist permissions.

Summary:
A Role can only be defined in one namespace . If you want to cross namespaces, you can create a ClusterRole, that is to say, you don't need to bind a namespace to define a ClusterRole .

3.2 Examples

• Role example:

apiVersion: rbac.authorization.k8s.io/v1  #指定 core API 组和版本
kind: Role   #指定类型为 Role
metadata:
  namespace: default   #使用默认命名空间
  name: pod-reader     #Role 的名称
rules:   #定义规则
- apiGroups: [""]   #""表示 apiGroups 和 apiVersion 使用相同的 core API 组，即 rbac.authorization.k8s.io
  resources: ["pods"]  #资源对象为 Pod 类型
  verbs: ["get", "watch", "list"]  #被授予的操作权限

Assign the role of pod-reader to a user, then this user will have three operation permissions on Pod resource objects in the default namespace: get (obtain), watch (monitor), and list (list).

• ClusterRole example :

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" 被忽略，因为 ClusterRoles 不受名称空间限制
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]  #资源对象为 Secret 类型
  verbs: ["get", "watch", "list"]

Cluseterole is not affected by the namespace, it acts on all namespaces, so there is no need to specify the namespace

• Example of RoleBinding

1. Reference Role role binding

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: zhangsan
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Grant the pod-reader Role of the default namespace to the zhangsan user, and then the zhangsan user will have pod-reader permissions in the default namespace.

2. Using ClusterRole role binding
   RoleBinding can also refer to ClusterRole to authorize User, Group or ServiceAccount in the current namespace. This operation allows cluster administrators to define some general ClusterRoles in the entire cluster, and then use RoleBinding to reference them in different namespaces .

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-secrets
  namespace: kube-public
subjects:
- kind: User
  name: lisi
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

Note: Because the RoleBinding binding is restricted by the namespace, even if the ClusterRole role has namespace permissions, it will be restricted to the namespace set by RoleBingding, so the above lisi
users can only access secrets in the kube-public space used in

• ClusterRoleBinding example:
  use ClusterRoleBinding to authorize all namespace resource permissions in the entire cluster

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-secrets-global
subjects:
- kind: Group
  name: manager
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

ClusterRoleBinding authorizes all users in the manager group to access secrets in all namespaces.

3.3 Resources

Some resources in the Kubernetes cluster are generally represented by their name strings, and these strings generally appear in the URL address of the API; at the same time, some resources also contain sub-resources, such as the log resource is a sub-resource of pods. A sample request URL for Pod logs is as follows:
GET /api/v1/namespaces/{namespace}/pods/{name}/log

To control the access rights of these sub-resources in the RBAC authorization model, resources and sub-resources can be separated by / separator.
example:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-and-pod-logs-reader
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list"]

The resource definition in rules has the following types:

1. rules.verbs有：“get”, “list”, “watch”, “create”, “update”, “patch”, “delete”, “exec”
2. rules.resources有：“services”, “endpoints”, “pods”, “secrets”, “configmaps”, “crontabs”, “deployments”, “jobs”, “nodes”, “rolebindings”, “clusterroles”, “daemonsets”, “replicasets”, “statefulsets”, “horizontalpodautoscalers”, “replicationcontrollers”, “cronjobs”
3. rules.apiGroups有：“”,“apps”, “autoscaling”, “batch”

4. Admission Control

After passing the authentication and authentication, the client cannot get a real response from the API Server, and this request still needs to pass the layer-by-layer test of an admission control plug-in list controlled by Admission Control.

Admission Control is equipped with a plug-in list of "admission controllers". Any request sent to the API Server needs to pass the check of each admission controller in the list, and the call request is rejected by the API Server if the check is not passed.
In addition, the admission controller can also modify the request parameters to complete some automated tasks, such as the Service Account controller.
The currently configurable Admission Control is as follows:

• AlwaysAdmit: Allow all requests;
• AlwaysPullimages: Always download the image before starting the container, which is equivalent to the configuration item imagePullPolicy=Always in each container
• AlwaysDeny: prohibit all requests, generally used for testing;
• DenyExecOnPrivileged: It will intercept all requests to execute commands on Privileged Containers. If your cluster supports Privileged Containers and you want to restrict users from executing commands on these Privileged Containers, it is strongly recommended that you use it. Its functions have been merged into DenyEscalatingExec .
• ImagePolicyWebhook: This plugin will allow a Webhook program in the backend to perform the functions of the admission controller. ImagePolicyWebhook needs to use a configuration file (set through the kube-apiserver startup parameter – admission-control-config-file) to define the parameters of the backend Webhook. The plugin is currently in alpha version.
• Service Account: This plug-in automates ServiceAccount and is enabled by default. If you want to use ServiceAccount objects, it is strongly recommended to use it.
• SecurityContextDeny: This plugin will invalidate all definitions in Pods using SecurityContext. SecurityContext defines the operating system-level security settings (uid, gid, capabilityes, SELinux, etc.) in the Container. Enabling this plugin is recommended in clusters where PodSecurityPolicy is not enabled, to disable non-secure access to container settings.
• ResourceQuota: Used for resource quota management purposes, acting on the namespace, it will observe all requests to ensure that the quota on the namespace will not exceed the standard. It is recommended to arrange this plugin last in the Admission Control parameter list, so as to avoid premature allocation of resources to Pods that may be rejected by other plugins.
• LimitRanger: Used for resource limit management, acting on the namespace to ensure that the resources of the Pod are limited. Enabling this plugin will also set default settings for Pods that do not have resource limits set, for example, set a resource request of 0.1CPU for all Pods in namespace "default".
• InitialResources: is an experimental feature, designed to initialize resource requests and limit settings for Pods that have not set resource requests and limits based on the usage of their mirrored historical resources.
• NamespaceLifecycle: If you try to create a resource object in a namespace that does not exist, the creation request will be rejected. When deleting a namespace, the system will delete all objects in the namespace, save Pod, Service, etc.
• DefaultStorageClass: In order to realize the dynamic provisioning of shared storage, try to match the default StorageClass for PVCs that do not specify StorageClass or PV, and minimize the back-end storage details that users need to know when applying for PVCs.
• DefaultTolerationSeconds: This plugin sets the default "tolerance" time for Pods that do not set forgiveness tolerations and have two taints: notready:NoExecute and unreachable:NoExecute, which is 5 minutes.
• PodSecurityPolicy: This plugin is used to determine whether to control the Pod's security policy based on the Pod's security context and the available PodSecurityPolicy when creating or modifying the Pod.

5. ServiceAccount access case

5.1 Create a namespace

kubectl create namespace sa

Creating a namespace will create a serviceaccount and a secret by default

#查看sa名称空间的sa证书
kubectl get sa -n sa
#查看sa名称空间的secret
kubectl get secret -n sa

#查看sa证书
kubectl describe sa default -n sa

#查看token
kubectl describe secret default-token-4mp4g -n sa

5.2 Create a pod to verify its sa and secret

#创建一个pod
kubectl run nginx --image=nginx:1.15 -n sa
#查看pod详细信息
kubectl get pod nginx -n sa -o yaml |grep serviceAccount

Create a pod without specifying serviceaccount and use default by default

5.3 Create a sa

#在sn名称空间中创建一个sa
kubectl create sa daniel -n sa
#查询sa
kubectl get sa -n sa
#查询secret
kubectl get secret -n sa

5.4 Create pod and customize sa

vim sa-pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: nginx2
  namespace: sa
spec:
  containers:
  - name: c1
    image: nginx:1.15-alpine
    ports:
    - name: httpd
      containerPort: 80
  serviceAccountName: daniel				# 指定sa为daniel

Create pods

kubectl apply -f sa-pod.yml
#查看pod的sa
kubectl get pod nginx2 -n sa -o yaml |grep serviceAccount