Kubernetes Access Control: Exploring Service Accounts

A tutorial on how Kubernetes uses third-party services to authenticate users.

Aug 16th, 2019 3:00am by Janakiram MSV

Featued image for: Kubernetes Access Control: Exploring Service Accounts

This is the last part of a tutorial series on Kubernetes access control. Having explored the key concepts related to authentication and authorization, we will take a closer look at service accounts.

Kubernetes has the notion of users and service account to access resources. A user is associated with a key and certificate to authenticate API requests. Any request originated outside of the cluster is authenticated using one of the configured schemes. The most common technique to authenticate requests is through X.509 certificates. Refer to the tutorial on Kubernetes authentication on creating and associating certificates with users. It’s important to recall that Kubernetes doesn’t maintain a database or profiles of users and passwords. Instead, it expects it to be managed outside of the cluster. Through the concept of authentication modules, Kubernetes can delegate authentication to a 3rd party like OpenID or Active Directory. While X.509 certificates are used for authenticating external requests, service accounts are meant to authenticate processes running within the cluster. Service accounts are associated with pods that make internal calls to the API server. Every Kubernetes installation has a service account called default that is associated with every running pod. Similarly, to enable pods to make calls to the internal API Server endpoint, there is a ClusterIP service called Kubernetes. This combination makes it possible for internal processes to call the API endpoint.

kubectl get serviceAccounts

NAME      SECRETS   AGE
default   1         122m

kubectl get svc

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1            443/TCP   123m

Notice that the service account is pointing to a secret that is mounted inside every pod. This secret contains the token expected by the API Server.

kubectl get secret

NAME                  TYPE                                  DATA   AGE
default-token-4rpmv   kubernetes.io/service-account-token   3      123m

Things get clear when we actually schedule a pod and access it. We will launch a pod that is based on BusyBox with curl command.

kubectl run -i --tty --rm curl-tns --image=radial/busyboxplus:curl

kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
If you don't see a command prompt, try pressing enter.
[ root@curl-tns-56c6d54585-6v2xp:/ ]$

While we are within the BusyBox shell, let’s try to hit the API Server endpoint.

[ root@curl-tns-56c6d54585-6v2xp:/ ]$ curl https://kubernetes:8443/api

This will not yield any result since the request lacks the authentication token. Let’s find out how to retrieve the token that can be embedded in the HTTP header. As discussed earlier, the token is mounted as a secret within the pod. Navigate to /var/run/secrets/kubernetes.io/serviceaccount to find the token.

[ root@curl-tns-56c6d54585-6v2xp:/ ]$ cd /var/run/secrets/kubernetes.io/serviceaccount

[ root@curl-tns-56c6d54585-6v2xp:/tmp/secrets/kubernetes.io/serviceaccount ]$ ls
ca.crt     namespace  token

Let’s set a few environment variables to simplify the curl command.

CA_CERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
NAMESPACE=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)

The below curl command requests the list of services running in the default namespace. Let’s see if we get a response from the API Server.

[ root@curl-tns-56c6d54585-6v2xp:~ ]$ curl --cacert $CA_CERT -H "Authorization: Bearer $TOKEN" "https://kubernetes/api/v1/namespaces/$NAMESPACE/services/"

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "services is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"services\" in API group \"\" in the namespace \"default\"",
  "reason": "Forbidden",
  "details": {
    "kind": "services"
  },
  "code": 403
}

Surprisingly, the default service account doesn’t have enough permissions to retrieve the services running in the same namespace. Remember that Kubernetes follows the convention of closed-to-open which means that by default no user or service account has any permissions. In order to fulfill this request, we need to create a role binding associating the default service account with an appropriate role. This workflow is similar to how we bound a role to Bob that gave him permission to list pods. Exit the pod and run the below command to create a role binding for the default service account.

kubectl create rolebinding default-view \
  --clusterrole=view \
  --serviceaccount=default:default \
  --namespace=default

rolebinding.rbac.authorization.k8s.io/default-view created

The above command associated the default service account with the cluster role view that enables the pod to list the resources. If you are curious to see all the available cluster roles, run the command, kubectl get clusterroles. Let’s launch the BusyBox pod again and hit the API Server.

kubectl run -i --tty --rm curl-tns --image=radial/busyboxplus:curl

kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
If you don't see a command prompt, try pressing enter.
[ root@curl-tns-56c6d54585-2cx44:/ ]$

CA_CERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
NAMESPACE=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)

curl --cacert $CA_CERT -H "Authorization: Bearer $TOKEN" "https://kubernetes/api/v1/namespaces/$NAMESPACE/services/"

{
  "kind": "ServiceList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/namespaces/default/services/",
    "resourceVersion": "11076"
  },
  "items": [
    {
      "metadata": {
        "name": "kubernetes",
        "namespace": "default",
        "selfLink": "/api/v1/namespaces/default/services/kubernetes",
        "uid": "b715a117-6be1-4de0-8830-45bddcda701c",
        "resourceVersion": "151",
        "creationTimestamp": "2019-08-13T09:45:27Z",
        "labels": {
          "component": "apiserver",
          "provider": "kubernetes"
        }
      },
      "spec": {
        "ports": [
          {
            "name": "https",
            "protocol": "TCP",
            "port": 443,
            "targetPort": 8443
          }
        ],
        "clusterIP": "10.96.0.1",
        "type": "ClusterIP",
        "sessionAffinity": "None"
      },
      "status": {
        "loadBalancer": {

        }
      }
    }
  ]
}

Feel free to create additional bindings for the default service account to check how RBAC extends to pods. This concludes the series on Kubernetes access control where we discussed the essential building blocks of authentication, authorization, and service accounts. Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at http://mi2.live.

Janakiram MSV (Jani) is a practicing architect, research analyst, and advisor to Silicon Valley startups. He focuses on the convergence of modern infrastructure powered by cloud-native technology and machine intelligence driven by generative AI. Before becoming an entrepreneur, he spent...