End user RBAC

In this lab we will set up role-based access control (RBAC) suitable for running the cluster in production. We will cover roles for using Calico. General RBAC for a production Kubernetes cluster is beyond the scope of this lab.

Using calicoctl

In order for the calicoctl tool to perform version mismatch verification (to make sure the versions for both the cluster and calicoctl are the same), whoever is using it needs to have get access to clusterinformations at the cluster level, i.e., not in a namespace. The network admin role below already has such access, but we will see that we will need to add it to the service owner user we will create further on.

  1. kubectl apply -f - <<EOF
  2. kind: ClusterRole
  3. apiVersion: rbac.authorization.k8s.io/v1
  4. metadata:
  5.   name: calicoctl-user
  6. rules:
  7.   - apiGroups: ["crd.projectcalico.org"]
  8.     resources:
  9.       - clusterinformations
  10.     verbs:
  11.       - get
  12. EOF

Network admin

A network admin is a person responsible for configuring and operating the Calico network as a whole. As such, they will need access to all Calico custom resources, as well as some associated Kubernetes resources.

Create the role

  1. kubectl apply -f - <<EOF
  2. kind: ClusterRole
  3. apiVersion: rbac.authorization.k8s.io/v1
  4. metadata:
  5.   name: network-admin
  6. rules:
  7.   - apiGroups: [""]
  8.     resources:
  9.       - pods
  10.       - nodes
  11.     verbs:
  12.       - get
  13.       - watch
  14.       - list
  15.       - update
  16.   - apiGroups: [""]
  17.     resources:
  18.       - namespaces
  19.       - serviceaccounts
  20.     verbs:
  21.       - get
  22.       - watch
  23.       - list
  24.   - apiGroups: ["networking.k8s.io"]
  25.     resources:
  26.       - networkpolicies
  27.     verbs: ["*"]
  28.   - apiGroups: ["crd.projectcalico.org"]
  29.     resources:
  30.       - felixconfigurations
  31.       - ipamblocks
  32.       - blockaffinities
  33.       - ipamhandles
  34.       - ipamconfigs
  35.       - bgppeers
  36.       - bgpconfigurations
  37.       - ippools
  38.       - hostendpoints
  39.       - clusterinformations
  40.       - globalnetworkpolicies
  41.       - globalnetworksets
  42.       - networkpolicies
  43.       - networksets
  44.     verbs: ["*"]
  45. EOF

To test out the network admin role, we’ll create a user named Nik grant them the role.

On the Kubernetes master node, create the key and certificate signing request. Note that we include /O=network-admins in the subject. This places Nik in the network-admins group.

  1. openssl req -newkey rsa:4096 \
  2.            -keyout nik.key \
  3.            -nodes \
  4.            -out nik.csr \
  5.            -subj "/O=network-admins/CN=nik"

We will sign this certificate using the main Kubernetes CA.

  1. sudo openssl x509 -req -in nik.csr \
  2.                   -CA /etc/kubernetes/pki/ca.crt \
  3.                   -CAkey /etc/kubernetes/pki/ca.key \
  4.                   -CAcreateserial \
  5.                   -out nik.crt \
  6.                   -days 365
  7. sudo chown $(id -u):$(id -g) nik.crt

Next, we create a kubeconfig file for Nik.

  1. APISERVER=$(kubectl config view -o jsonpath='{.clusters[0].cluster.server}')
  2. kubectl config set-cluster kubernetes \
  3.     --certificate-authority=/etc/kubernetes/pki/ca.crt \
  4.     --embed-certs=true \
  5.     --server=$APISERVER \
  6.     --kubeconfig=nik.kubeconfig
  8. kubectl config set-credentials nik \
  9.     --client-certificate=nik.crt \
  10.     --client-key=nik.key \
  11.     --embed-certs=true \
  12.     --kubeconfig=nik.kubeconfig
  14. kubectl config set-context default \
  15.     --cluster=kubernetes \
  16.     --user=nik \
  17.     --kubeconfig=nik.kubeconfig
  19. kubectl config use-context default --kubeconfig=nik.kubeconfig

Bind the role to the group network-admins.

  1. kubectl create clusterrolebinding network-admins --clusterrole=network-admin --group=network-admins

Test Nik’s access by creating a global network set

  1. KUBECONFIG=./nik.kubeconfig calicoctl apply -f - <<EOF
  2. apiVersion: projectcalico.org/v3
  3. kind: GlobalNetworkSet
  4. metadata:
  5.   name: niks-set
  6. spec:
  7.   nets:
  8.   - 110.120.130.0/24
  9.   - 210.220.230.0/24
  10. EOF

Verify the global network set exists

  1. KUBECONFIG=./nik.kubeconfig calicoctl get globalnetworkset -o wide

Result

  1. NAME       NETS
  2. niks-set   110.120.130.0/24,210.220.230.0/24

Delete the global network set

  1. KUBECONFIG=./nik.kubeconfig calicoctl delete globalnetworkset niks-set

Service owner

A service owner is a person responsible for operating one or more services in Kubernetes. They should be able to define network policy for their service, but don’t need to view or modify any global configuration related to Calico.

Define the role

  1. kubectl apply -f - <<EOF
  2. kind: ClusterRole
  3. apiVersion: rbac.authorization.k8s.io/v1
  4. metadata:
  5.   name: network-service-owner
  6. rules:
  7.   - apiGroups: ["networking.k8s.io"]
  8.     resources:
  9.       - networkpolicies
  10.     verbs: ["*"]
  11.   - apiGroups: ["crd.projectcalico.org"]
  12.     resources:
  13.       - networkpolicies
  14.       - networksets
  15.     verbs: ["*"]
  16. EOF

To test out the service owner role, we’ll create a user named Sam and grant them the role.

On the Kubernetes master node, create the key and certificate signing request.

  1. openssl req -newkey rsa:4096 \
  2.            -keyout sam.key \
  3.            -nodes \
  4.            -out sam.csr \
  5.            -subj "/CN=sam"

We will sign this certificate using the main Kubernetes CA.

  1. sudo openssl x509 -req -in sam.csr \
  2.                   -CA /etc/kubernetes/pki/ca.crt \
  3.                   -CAkey /etc/kubernetes/pki/ca.key \
  4.                   -CAcreateserial \
  5.                   -out sam.crt \
  6.                   -days 365
  7. sudo chown $(id -u):$(id -g) sam.crt

Next, we create a kubeconfig file for Sam.

  1. APISERVER=$(kubectl config view -o jsonpath='{.clusters[0].cluster.server}')
  2. kubectl config set-cluster kubernetes \
  3.     --certificate-authority=/etc/kubernetes/pki/ca.crt \
  4.     --embed-certs=true \
  5.     --server=$APISERVER \
  6.     --kubeconfig=sam.kubeconfig
  8. kubectl config set-credentials sam \
  9.     --client-certificate=sam.crt \
  10.     --client-key=sam.key \
  11.     --embed-certs=true \
  12.     --kubeconfig=sam.kubeconfig
  14. kubectl config set-context default \
  15.     --cluster=kubernetes \
  16.     --user=sam \
  17.     --kubeconfig=sam.kubeconfig
  19. kubectl config use-context default --kubeconfig=sam.kubeconfig

We will limit Sam’s access to a single namespace. Create the namespace

  1. kubectl create namespace sam

Bind the role to Sam in the namespace

  1. kubectl create rolebinding -n sam network-service-owner-sam --clusterrole=network-service-owner --user=sam

Also bind the calicoctl-user role to sam at the cluster level so that they can use calicoctl properly

  1. kubectl create clusterrolebinding calicoctl-user-sam --clusterrole=calicoctl-user --user=sam

Sam cannot create global network set resources (like Nik can as network admin)

  1. KUBECONFIG=./sam.kubeconfig calicoctl get globalnetworkset -o wide

Result

  1. connection is unauthorized: globalnetworksets.crd.projectcalico.org is forbidden: User "sam" cannot list resource "globalnetworksets" in API group "crd.projectcalico.org" at the cluster scope

However, Sam can create resources in their own namespace

  1. KUBECONFIG=./sam.kubeconfig calicoctl apply -f - <<EOF
  2. apiVersion: projectcalico.org/v3
  3. kind: NetworkSet
  4. metadata:
  5.   name: sams-set
  6.   namespace: sam
  7. spec:
  8.   nets:
  9.   - 110.120.130.0/24
  10.   - 210.220.230.0/24
  11. EOF

Verify the resource exists

  1. KUBECONFIG=./sam.kubeconfig calicoctl get networksets -n sam

Result

  1. NAMESPACE   NAME
  2. sam         sams-set

Delete the NetworkSet

  1. KUBECONFIG=./sam.kubeconfig calicoctl delete networkset sams-set -n sam

Next

Istio integration