RBAC Authorization for K8s API Access

We want to scale the compute pods by calling k8s API from inside the engine conductor container, this definitely need to be authorized and we need to grant privilege for this action.

There are some concepts you need to know in order to achieve the goal.

Service Account

what is Service Account

Processes in containers inside pods can contact the apiserver. When they do, they are authenticated as a particular service account (for example, default).

Once you create a namespace, for example test-1, there is a default service account automatically generated.

kubectl get sa -n test-1

NAME      SECRETS   AGE
default   1         139m

let's see what is inside the service account

kubectl describe sa default -n test-1

Name:                default
Namespace:           test-1
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   default-token-mtv4n
Tokens:              default-token-mtv4n
Events:              <none>

here we see there is a mountable secret default-token-mtv4n, that is the credentials to access the apiserver.

kubectl describe secret default-token-mtv4n -n test-1

Name:         default-token-mtv4n
Namespace:    test-1
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: default
              kubernetes.io/service-account.uid: 387381d3-2272-11e9-91a2-00163e0196e7

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  6 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrd...

ClusterRole

what is Role and ClusterRole

A ClusterRole can be used to grant the same permissions as a Role, but because they are cluster-scoped, they can also be used to grant access to

cluster-scoped resources (like nodes)
non-resource endpoints (like “/healthz”)
namespaced resources (like pods) across all namespaces

Here we use a cluster role called cluster-admin, it's generated by default

kubectl get clusterrole | grep cluster-admin

cluster-admin                        174m

ClusterRole Binding

what is RoleBinding and ClusterRole Binding

A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. Permissions can be granted within a namespace with a RoleBinding, or cluster-wide with a ClusterRoleBinding.

below we grant service account default in namespace test-1 the cluster admin level privilege.

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 name: <cluster role name>
subjects:
 - kind: ServiceAccount
 name: default
 namespace: test-1
roleRef:
 kind: ClusterRole
 name: cluster-admin
 apiGroup: http://rbac.authorization.k8s.io

then we can write a script, using curl to call k8s API, for example, to scale the number of compute pods

http_code=$(curl -w "%{http_code}" -sS  --cacert $CACERT  -XPATCH -H "Content-Type: application/strategic-merge-patch+json" -H "Accept: application/json" -H "Authorization: Bearer $TOKEN" "https://kubernetes.default/apis/apps/v1/namespaces/$NAMESPACE/statefulsets/is-engine-compute" --data "{\"spec\":{\"replicas\":$REP}}" -o $OUT_FILE)
if [[ $http_code -ne 200 ]]; then
    ${JQ} '{ result:.status, code: .code,  message: .message }' $OUT_FILE
    exit 1
fi