kubernetes senior security policy of the pod

Series catalog

What is the pod security policy

pod security policy is a resource for control pod cluster level security-related options. PodSecurityPolicydefines a series of pod phase to be constraints in the system that must be met in order to coat some default constraint values. It allows administrators to control the following aspects

Control Aspect	Field Names
Privilege to run container	privileged
Use the host name space	hostPID, hostIPC
And using the host network ports	host network, host ports
Use the type of storage volumes	volumes
Use the host file system	allowedHostPaths
flex storage volume whitelist	allowedFlexVolumes
Pod has allocated data volume FSGroup	fsGroup
Read-only root file system	readOnlyRootFilesystem
User id and group id container	runAsUser, runAsGroup, supplementalGroups
Prohibition to become root	allowPrivilegeEscalation, defaultAllowPrivilegeEscalation
Linux capabilities	defaultAddCapabilities, requiredDropCapabilities, allowedCapabilities
SELinux context	seLinux
Proc allow the container loaded type	allowedProcMountTypes
The AppArmor profile used by containers	annotations
The seccomp profile used by containers	annotations
The sysctl profile used by containers	annotations

Enabling pod Security Policy

pod security policy as an optional (but highly recommended) admission implement .pod security policy controller is implemented by the admission controller is enabled, but only enabled without the authorization of the strategy will result in the entire cluster can not create a pod!

As the pod security policy api (policy / v1beta1 / podsecuritypolicy) independent of admission controlleroutside enabled for existing cluster is recommended to enable admission controlleradding and authorization policies before.

Authorization Policy

When a pod security policy resources are created (as I said before, psp (PodSecurityPolicy) pod security policy is a kubernetes resources), it will not do anything. To use it, requesting user or target pod of serviceaccount operation strategies must the useverb to authorize.

Most of kubernetes pod is not directly created by the user directly. Instead, the typical usage scenario is that they are by Deploymentor ReplicaSetindirectly created or created by other templates Controller Controller Manager. The controller is policy authorization will also All pod it creates for policy authorization. Therefore, the preferred method is to authorize serviceaccount pod were policy authorization (behind the example).

With RBAC Authorization

RBAC is kubernetes standard licensing model, and can easily be used to authorize the use of security policies.

First, a role (role) or a cluster role (clusterRole) need to be authorized 使用(use动词)strategy it wants. Authorization role similar to the following

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: <role name>
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - 一系列要进行授权的资源名称

Then cluster role (or roles) with an authorized user is bound

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: 绑定名称
roleRef:
  kind: ClusterRole
  name: 角色名称
  apiGroup: rbac.authorization.k8s.io
subjects:
# Authorize specific service accounts:
- kind: ServiceAccount
  name: 授权的serviceaccount名称
  namespace: <authorized pod namespace>
# Authorize specific users (not recommended):
- kind: User
  apiGroup: rbac.authorization.k8s.io
  name: 授权的用户名

If a character bind (not cluster bound to the role) is used, it is only in a pod and it can be under the same name space for effective policy authorization, this also applies to users and user groups

# Authorize all service accounts in a namespace:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts
# Or equivalently, all authenticated users in a namespace:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:authenticated

Troubleshooting

Controller Manager must run on a secure api port, and can not have super powers. Otherwise, the request will bypass the authentication and authorization module, it will cause all of the policies are allowed, and users can create privilege pod

Policy order

In addition to limiting pod creation and update, pod security policy is also used to provide a default value it controls many fields. When there are multiple strategies, pod security policy selection policy based on the following factors

Any successful strategy by verifying no warnings will be used
If the request is to create a pod, press the validation strategy was chosen in alphabetical order
Otherwise, if the update is a request, it will return an error because the pod is not allowed during the update operation changes

Examples

The following example assumes that you run the cluster open the pod security policy admission controller and you have cluster administrator privileges

default setting

We create a namespace and a serviceaccount as an example. We use this to simulate serviceaccount a non-administrator user

kubectl create namespace psp-example
kubectl create serviceaccount -n psp-example fake-user
kubectl create rolebinding -n psp-example fake-editor --clusterrole=edit --serviceaccount=psp-example:fake-user

In order to facilitate identification of the account we use, we create two aliases

alias kubectl-admin='kubectl -n psp-example'
alias kubectl-user='kubectl --as=system:serviceaccount:psp-example:fake-user -n psp-example'

Create a policy and a pod

The following definitions file defines a simple pod security policy (PodSecurityPolicy), this strategy only prevent the creation of privileged pod

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: example
spec:
  privileged: false  # Don't allow privileged pods!
  # The rest fills in some required fields.
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'

We use kubectl command to apply the above documents.

Now, as a non-privileged user, we created a simple pod

kubectl-user create -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name:      pause
spec:
  containers:
    - name:  pause
      image: k8s.gcr.io/pause
EOF
Error from server (Forbidden): error when creating "STDIN": pods "pause" is forbidden: unable to validate against any pod security policy: []

What happened? Despite the pod security policy has been created, whether or pod of serviceaccount fack-user did not have permission to use this strategy.

kubectl-user auth can-i use podsecuritypolicy/example
no

Create a rolebingto authorize fake-useruse examplepolicy (example is the name of the policy created earlier)

However, note that this is not the preferred way! Later examples describe the preferred way

kubectl-admin create role psp:unprivileged \
    --verb=use \
    --resource=podsecuritypolicy \
    --resource-name=example
role "psp:unprivileged" created

kubectl-admin create rolebinding fake-user:psp:unprivileged \
    --role=psp:unprivileged \
    --serviceaccount=psp-example:fake-user
rolebinding "fake-user:psp:unprivileged" created

kubectl-user auth can-i use podsecuritypolicy/example
yes

In this case, try re-create the pod

kubectl-user create -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name:      pause
spec:
  containers:
    - name:  pause
      image: k8s.gcr.io/pause
EOF
pod "pause" created

This time, as we expect, can work normally, but trying to create privileged pod will still be blocked (and therefore policy itself prevent the creation of privileged pod)

kubectl-user create -f- <<EOF
apiVersion: v1
kind: Pod
metadata:
  name:      privileged
spec:
  containers:
    - name:  pause
      image: k8s.gcr.io/pause
      securityContext:
        privileged: true
EOF
Error from server (Forbidden): error when creating "STDIN": pods "privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]

Then run one other pod

We try to create a pod, this is a little different

ubectl-user run pause --image=k8s.gcr.io/pause
deployment "pause" created

kubectl-user get pods
No resources found.

kubectl-user get events | head -n 2
LASTSEEN   FIRSTSEEN   COUNT     NAME              KIND         SUBOBJECT                TYPE      REASON                  SOURCE                                  MESSAGE
1m         2m          15        pause-7774d79b5   ReplicaSet                            Warning   FailedCreate            replicaset-controller                   Error creating: pods "pause-7774d79b5-" is forbidden: no providers available to validate pod request

From the above you can see the deployment has been successfully created (kubectl run will actually create a deployment). But using the kubectl get podcommand did not find the pod was created. Why is this? The answer is hidden in replicaset your controller. Fake-userDeployment (deployment successfully created successfully created replicaset), but when replicaset try to create a pod, which has not been authorized to use exampledefined policies.

To solve this problem, the psp:unprivilegedrole of binding (created earlier) to serviceaccount pod of (in front of us is bound on the fake-user). Serviceaccount here is default(because we do not specify other user)

See here if you still find it difficult to understand, you can go back and look, still can not understand the words you need to add about the role, RBAC users and relevant knowledge.

kubectl-admin create rolebinding default:psp:unprivileged \
    --role=psp:unprivileged \
    --serviceaccount=psp-example:default
rolebinding "default:psp:unprivileged" created

This waiting time of several minutes, replicaset controller eventually successfully created pod

kubectl-user get pods --watch
NAME                    READY     STATUS    RESTARTS   AGE
pause-7774d79b5-qrgcb   0/1       Pending   0         1s
pause-7774d79b5-qrgcb   0/1       Pending   0         1s
pause-7774d79b5-qrgcb   0/1       ContainerCreating   0         1s
pause-7774d79b5-qrgcb   1/1       Running   0         2s

Cleanup

Delete the name space to remove most of the resources used in the example

kubectl-admin delete ns psp-example
namespace "psp-example" deleted

Note that the pod is now just create a security policy has no name space, and need to be cleared separately

kubectl-admin delete psp example
podsecuritypolicy "example" deleted

Policy Example

The following is a minimum limit of strategies, policies and without admission controller pod Anson same effect

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: privileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'

The following example of a restrictive policy, the needs of users is a non-privileged user to prevent privilege escalation pod

The requirement for a non-privileged user, privileged user will bypass the restriction because

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
    apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    apparmor.security.beta.kubernetes.io/defaultProfileName:  'runtime/default'
spec:
  privileged: false
  # Required to prevent escalations to root.
  allowPrivilegeEscalation: false
  # This is redundant with non-root + disallow privilege escalation,
  # but we can provide it for defense in depth.
  requiredDropCapabilities:
    - ALL
  # Allow core volume types.
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    # Assume that persistentVolumes set up by the cluster admin are safe to use.
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    # Require the container to run without root privileges.
    rule: 'MustRunAsNonRoot'
  seLinux:
    # This policy assumes the nodes are using AppArmor rather than SELinux.
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
  readOnlyRootFilesystem: false

Policy Reference

Prerogative

It determines whether all the containers in the pod are allowed to run in privileged mode. The device does not allow access to the host vessel by default, but the vessel was allowed access privileges. This will allow the container and it has almost the same as in the process of access to the host rights. this is useful when you want to use a host of features container, such as access to network devices.

Host name space

HostPID - whether the container control process can be shared host id namespace

HostIPC - controls whether the container can share IPC namespaces host

HostNetwork - controls whether the vessel can use the network name of the node where the space pod that will allow access to the loopback device, listens localhost, and can spy on other pod on the same node network activity status.

AllowedHostPaths - allows access to the host control path

Storage volume and file system

Volumes- offers a range of storage volume type whitelist definitions allow these values and create a storage volume corresponding to the type of resources you want to get all types of storage volumes can be viewed. Storage volume type list addition. *It can be used to allow All types of storage volumes

The following are recommended to minimize the storage volume to allow the type of security policy configuration

configMap
downwardAPI
emptyDir
persistentVolumeClaim
secret
projected

AllowedHostPaths- It defines a hostPathwhitelist host available storage volume path type air path cluster mean to limit the use of non-host which is defined as a set containing a single object. pathPrefixField, type of storage allows hostpath volume mount at pathPrefixthe beginning of the field in the host path readonlyfield means must readonlymount the (ie can not be written, only read)

allowedHostPaths:
  # This allows "/foo", "/foo/", "/foo/bar" etc., but
  # disallows "/fool", "/etc/foo" etc.
  # "/foo/../" is never valid.
  - pathPrefix: "/foo"
    readOnly: true # only allow read-only mounts

Warning, a container can be unlimited access to the host file system can be many ways to elevate privileges, including reading data in the other container, abuse key system services, such as kubecctl

Hostpath directory writable storage volumes allows the container to be written to the host file system, and can traverse pathPrefixfile systems other than, readOnly: trueto use in the future kubernetes 1.11+ version, and allowedHostPathsmust be used to effectively restrict access to specificpathPrefix

ReadOnlyRootFilesystem - Limit the container must be run as root file system read-only (not writable layer)

Elevation of Privilege

This option controls the container allowPrivilegeEscalationoption. This Boolean value directly controls no_new_privsis set to process container operation. It will stop setuidto change the user ID, and the ability to prevent other files (such as prohibiting the use of ping tool). This behavior needs to be enabledMustRunAsNonRoot

AllowPrivilegeEscalation- which determines whether the security context of the container can be set allowPrivilegeEscalation=true, the default value is true to false will make a container for all the child process is no higher privilege than that of the parent process.

DefaultAllowPrivilegeEscalation, To allowPrivilegeEscalationset the default value, can be seen from the above, the default value is true. If this behavior is not what we expect, this field can be used to disallow it, but still pod explicit requestallowPrivilegeEscalation