Multi-user practice in kubernetes

Sometimes it is necessary for multiple users to share a cluster. In this case, a new user needs to be assigned; for the security of the cluster, it is necessary to limit the scope of authority of the new user; it is inevitable that multiple users will compete for resources. Need to limit its resource usage.

Kubernetes provides a series of mechanisms to meet the needs of multiple users, including multiple users, authentication, namespaces, resource restrictions, and so on.

Next, a user named staight will be created, which has administrator rights under the practice namespace, which has restrictions on CPU, memory, and the number of Pods.

Create user

User creation in Kubernetes generally includes static creation and dynamic creation. Static creation requires user information files to be provided when apiserver is started; dynamic creation can be dynamically added after apiserver is started.

The dynamically created authentication methods include client certificate authentication and Service Account Token authentication. If you can log in to the master, it is recommended to use client certificate authentication. Here, try to use the Service Account Token authentication method to create a user.

The Service Account belongs to the namespace, so first create the namespace/practice:

[root@node k8s]# kubectl create namespace practice
namespace/practice created

Every time a namespace is created, a new serviceaccount/default is created for it, but here is a new serviceaccount/staight:

[root@node k8s]# kubectl create -n practice serviceaccount staight
serviceaccount/staight created

At this point, a new user staight has been created, but in the view of apiserver, his full user name should be:

system:serviceaccount:practice:staight。

Switch user

After creating a user, you also need to switch to that user. The kubectl command provides the config subcommand to accomplish this purpose. This command essentially modifies the kubeconfig file located in ~/.kube/config, so you can also create a new user For Linux users, place the kubeconfig file for them to realize the use of different Kubernetes users for different Linux users.

First, you need to get the token of the serviceaccount/staight user:

[root@node k8s]# kubectl describe -n practice serviceaccount/staight |grep Token
Tokens:              staight-token-fhd4c
[root@node k8s]# kubectl describe -n practice secret/staight-token-fhd4c
Name:         staight-token-fhd4c
Namespace:    practice
Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  8 bytes
token: <TOKEN_CONTENT>

<TOKEN_CONTENT> is the token of serviceaccount/staight, set the user of kubeconfig, and its name is staight:

[root@node k8s]# kubectl config set-credentials staight --token=<TOKEN_CONTENT>
User "staight" set.

Then set the context, the name is practice-context, the reference user is staight, the cluster is the default local, and the namespace is practice (if you do not specify the namespace when using the kubectl command, the default is practice):

[root@node k8s]# kubectl config set-context practice-context --user=staight --cluster=local --namespace=practice
Context "practice-context" created.

Finally, switch to the context:

[root@node k8s]# kubectl config use-context practice-context
Switched to context "practice-context".

Try to get the pod list and found that there is no permission, but you can verify that the user switch is successful:

[root@node k8s]# kubectl get pod
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:practice:staight" cannot list resource "pods" in API group "" in the namespace "practice"

Granted permission

The newly created user does not have any permissions, so it needs to be granted permissions. Kubernetes provides multiple permission granting methods, including ABAC, Webhook, RBAC, and so on.

RBAC is the default and recommended permission granting method for Kubernetes. If you want to use other methods, you need to modify the apiserver startup parameters. RBAC mode is used here.

First, you need to create a Role/ClusterRole resource and specify the allowed permissions. Kubernetes presets clusterrole/admin, which allows write access to a single namespace except resource quotas and the namespace itself. It is very suitable for use as an administrator of a single namespace. Therefore, no new role is created here.

If you want to know what permissions are granted by clusterrole/admin, you can use the describe command to view:

[root@node k8s]# kubectl describe clusterrole admin
Name:         admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
  Resources                                       Non-Resource URLs  Resource Names  Verbs
  ---------                                       -----------------  --------------  -----

Next, you need to bind clusterrole/admin to serviceaccount/staight:

[root@node k8s]# kubectl create rolebinding practice-admin --clusterrole=admin --serviceaccount=practice:staight --namespace=practice
rolebinding.rbac.authorization.k8s.io/practice-admin created

As above, the admin role of the practice namespace is granted to the staight user. If you need to allow the user to manage multiple namespaces, you can change the -namespace parameter to create it again.

Switch to the staight user and try to get the Service Account under the practice namespace:

[root@node k8s]# kubectl config use-context practice-context 
Switched to context "practice-context".
[root@node k8s]# kubectl get serviceaccounts 
NAME      SECRETS   AGE
default   1         54m
staight   1         49m

Successfully obtained, authorization is successful.

In most cases, it is sufficient to complete the new user's management authority for a single namespace, but if you need to further restrict the use of its resources, you need to modify the resource quota of the namespace.

Resource quota

Resource quota is a mechanism used to limit the use of resources in a namespace, which includes the following two objects:

ResourceQuota: Limit the resource usage under a single namespace. Including CPU, memory, storage, the number of resource objects, etc.
LimitRanger: Set default values and range constraints for Limits and Requests of the container.

ResourceQuota

Example: Set resource quotas for the practice namespace

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-resources
  namespace: practice
spec:
  hard:
    pods: "2"
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

As above, the number of pods and the used requests and limits are restricted. You can use the describe command to view the current resource usage and limits:

[root@node k8s]# kubectl describe resourcequotas 
Name:            compute-resources
Namespace:       practice
Resource         Used  Hard
--------         ----  ----
limits.cpu       0     2
limits.memory    0     2Gi
pods             0     2
requests.cpu     0     1
requests.memory  0     1Gi

Try to create two Pods, and an error will be reported after creating again:

[root@node k8s]# kubectl create -f pod.yml 
Error from server (Forbidden): error when creating "pod.yml": pods "alpine2" is forbidden: exceeded quota: compute-resources, requested: pods=1, used: pods=2, limited: pods=2

resourcequota provides restrictions on a lot of resources. For details, please refer to the document: https://kubernetes.io/docs/concepts/policy/resource-quotas/#compute-resource-quota.

LimitRanger

LimitRanger is used to set the default requests and limits values for the container, and to limit its scope.

Example: Limit the requests value and limits value of the container under the practice namespace

apiVersion: v1
kind: LimitRange
metadata:
  name: memory-range
  namespace: practice
spec:
  limits:
  - max: # 限制容器最大limits值
      memory: 20Mi
    min: # 限制容器最小limits值
      memory: 10Mi
    default: # 默认limits值
      memory: 10Mi
    defaultRequest: # 默认requests值
      memory: 10Mi
    type: Container

As above, if the limits and requests values are not specified when creating a Pod, requests.memory: 10Mi, limits.memory: 10Mi will be automatically added to it; if the limits.memory value is less than 10Mi or greater than 20Mi at the time of creation, the request will be rejected:

[root@node k8s]# kubectl create -f pod.yml 
Error from server (Forbidden): error when creating "pod.yml": pods "alpine2" is forbidden: maximum memory usage per Container is 20Mi, but limit is 30Mi.

summary

The practice of Kubernetes multi-user includes creating users, granting permissions, and resource quotas. This article made a preliminary demonstration, but in an actual environment, you should make a more comprehensive plan for the choice of user authentication methods, what permissions to grant, and resource quotas.

Reference document
Resource Quotas: https://kubernetes.io/docs/concepts/policy/resource-quotas/

RBAC-role-based access control: https://jimmysong.io/kubernetes-handbook/concepts/rbac.html

Configure Memory and CPU Quotas for a Namespace：https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/

Configure Minimum and Maximum Memory Constraints for a Namespace：https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/

Reposted from: https://staight.github.io/2019/09/29/kubernetes%E4%B8%AD%E5%A4%9A%E7%94%A8%E6%88%B7%E7%9A%84% E5%AE%9E%E8%B7%B5/