Kubernetes RBAC

In KubeVela v1.4, Authentication & Authorization mechanism is introduced. This allows applications to dispatch and manage resources using the identity of the application’s creator/modifier. With this feature, it will be easy to limit the access of KubeVela users/applications and isolate their living spaces, which will make your KubeVela system safer.

To enable Authentication & Authorization in your KubeVela system, you need to do the following steps

  1. Delete the ClusterRoleBinding ends with vela-core:manager-rolebinding. Usually you can do it through:

    1. kubectl delete ClusterRoleBinding kubevela-vela-core:manager-rolebinding

  2. Upgrade the controller, and wait for the installation finished:

    1. helm upgrade --install kubevela kubevela/vela-core --create-namespace -n vela-system --set authentication.enabled=true --set authentication.withUser=true --wait

  3. Make sure your version Vela CLI v1.4.1+, refer to the installation guide.

  4. (Optional) Install vela-prism through running the following commands, which will allow you to enjoy the advanced API extensions in KubeVela.

    1. helm repo add prism https://charts.kubevela.net/prism
    2. helm repo update
    3. helm install vela-prism prism/vela-prism -n vela-system

  5. Before we start, assume we already have two managed clusters joined in KubeVela, called c2 and c3. You can refer to the multi-cluster document and see how to join managed clusters into KubeVela control plane.

  1. $ vela cluster list
  2. NAME    ALIAS   CREDENTIAL_TYPE   ENDPOINT                      ACCEPTED   LABELS
  3. local           Internal          -                             true       
  4. c3              X509Certificate   <c3 apiserver url>            true       
  5. c2              X509Certificate   <c2 apiserver url>            true

The following procedures are illustrated in the following figure. The operations below are associated with different users, which are identified by switching KUBECONFIG environment variable. (Each KUBECONFIG corresponds to one user.)

auth-procedure

  1. Let’s start with a new coming user named Alice. As the system administrator, you can assign a KubeConfig for Alice to use.
  1. $ vela auth gen-kubeconfig --user alice > alice.kubeconfig
  2. Private key generated.
  3. Certificate request generated.
  4. Certificate signing request alice generated.
  5. Certificate signing request alice approved.
  6. Signed certificate retrieved.
  1. Now alice is unable to do anything in the cluster with the given KubeConfig. We can grant her the privileges of Read/Write resources in the dev namespace of the control plane and managed cluster c2.
  1. $ vela auth grant-privileges --user alice --for-namespace dev --for-cluster=local,c2 --create-namespace
  2. ClusterRole kubevela:writer created in local.
  3. RoleBinding dev/kubevela:writer:binding created in local.
  4. ClusterRole kubevela:writer created in c2.
  5. RoleBinding dev/kubevela:writer:binding created in c2.
  6. Privileges granted.
  1. We can check the privileges of Alice by the following command
  1. $ vela auth list-privileges --user alice --cluster local,c2
  2. User=alice
  3. ├── [Cluster]  local
  4. │   └── [ClusterRole]  kubevela:writer
  5. │       ├── [Scope]  
  6. │       │   └── [Namespaced]  dev (RoleBinding kubevela:writer:binding)
  7. │       └── [PolicyRules]  
  8. │           ├── APIGroups:       *
  9. │           │   Resources:       *
  10. │           │   Verb:            get, list, watch, create, update, patch, delete
  11. │           └── NonResourceURLs: *
  12. │               Verb:            get, list, watch, create, update, patch, delete
  13. └── [Cluster]  c2
  14.     └── [ClusterRole]  kubevela:writer
  15.         ├── [Scope]  
  16.         │   └── [Namespaced]  dev (RoleBinding kubevela:writer:binding)
  17.         └── [PolicyRules]  
  18.             ├── APIGroups:       *
  19.             │   Resources:       *
  20.             │   Verb:            get, list, watch, create, update, patch, delete
  21.             └── NonResourceURLs: *
  22.                 Verb:            get, list, watch, create, update, patch, delete
  1. Alice can create an application in the dev namespace now. The application can also dispatch resources into the dev namespace of cluster c2.
  1. cat <<EOF | KUBECONFIG=alice.kubeconfig vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5.   name: podinfo
  6.   namespace: dev
  7. spec:
  8.   components:
  9.   - name: podinfo
  10.     type: webservice
  11.     properties:
  12.       image: stefanprodan/podinfo:6.0.1
  13.   policies:
  14.   - type: topology
  15.     name: topology
  16.     properties:
  17.       clusters: ["c2"]
  18. EOF
  1. Alice can see the the application is successfully deployed.
  1. $ KUBECONFIG=alice.kubeconfig vela status podinfo -n dev
  2. About:
  4.   Name:         podinfo                      
  5.   Namespace:    dev                          
  6.   Created at:   2022-05-31 17:06:14 +0800 CST
  7.   Status:       running                      
  9. Workflow:
  11.   mode: DAG
  12.   finished: true
  13.   Suspend: false
  14.   Terminated: false
  15.   Steps
  16.   - id:rk3npcpycl
  17.     name:deploy-topology
  18.     type:deploy
  19.     phase:succeeded 
  20.     message:
  22. Services:
  24.   - Name: podinfo  
  25.     Cluster: c2  Namespace: dev
  26.     Type: webservice
  27.     Healthy Ready:1/1
  28.     No trait applied
  1. If Alice wants to access resources outside the dev namespace, she will be forbidden to do so.
  1. KUBECONFIG=alice.kubeconfig kubectl get pod -n vela-system
  2. Error from server (Forbidden): pods is forbidden: User "alice" cannot list resource "pods" in API group "" in the namespace "vela-system"
  1. If Alice try to create a application in the dev namespace and the application intends to dispatch resources in c3 cluster (which Alice does not have the privileges), the application will not be able to do so.

Alice create the application podinfo-bad

  1. $ cat <<EOF | KUBECONFIG=alice.kubeconfig vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5.   name: podinfo-bad
  6.   namespace: dev
  7. spec:
  8.   components:
  9.   - name: podinfo-bad
  10.     type: webservice
  11.     properties:
  12.       image: stefanprodan/podinfo:6.0.1
  13.   policies:
  14.   - type: topology
  15.     name: topology
  16.     properties:
  17.       clusters: ["c3"]
  18. EOF

Alice checks the status of podinfo-bad, the error message will be shown.

  1. $ KUBECONFIG=alice.kubeconfig vela status podinfo-bad -n dev
  2. About:
  4.   Name:         podinfo-bad                  
  5.   Namespace:    dev                          
  6.   Created at:   2022-05-31 17:09:16 +0800 CST
  7.   Status:       runningWorkflow              
  9. Workflow:
  11.   mode: DAG
  12.   finished: false
  13.   Suspend: false
  14.   Terminated: false
  15.   Steps
  16.   - id:tw539smx7m
  17.     name:deploy-topology
  18.     type:deploy
  19.     phase:failed 
  20.     message:step deploy: run step(provider=multicluster,do=deploy): Found 1 errors. [(error encountered in cluster c3: HandleComponentsRevision: controllerrevisions.apps is forbidden: User "alice" cannot list resource "controllerrevisions" in API group "apps" in the namespace "dev")]
  22. Services:
  1. Let’s create a new KubeConfig for another new User Bob. Bob will only be granted with the readonly privileges for the resources in the dev namespace of the control plane and cluster c2.
  1. $ vela auth gen-kubeconfig --user bob > bob.kubeconfig
  2. Private key generated.
  3. Certificate request generated.
  4. Certificate signing request bob generated.
  5. Certificate signing request bob approved.
  6. Signed certificate retrieved.
  8. $ vela auth grant-privileges --user bob --for-namespace dev --for-cluster=local,c2 --readonly
  9. ClusterRole kubevela:reader created in local.
  10. RoleBinding dev/kubevela:reader:binding created in local.
  11. ClusterRole kubevela:reader created in c2.
  12. RoleBinding dev/kubevela:reader:binding created in c2.
  13. Privileges granted.
  1. User Bob can see the applications and their status under the namespace of dev.
  1. $ KUBECONFIG=bob.kubeconfig vela ls -n dev
  2. APP             COMPONENT       TYPE            TRAITS  PHASE                   HEALTHY STATUS          CREATED-TIME                 
  3. podinfo         podinfo         webservice              running                 healthy Ready:1/1       2022-05-31 17:06:14 +0800 CST
  4. podinfo-bad     podinfo-bad     webservice              workflowTerminated                              2022-05-31 17:09:16 +0800 CST
  6. $ KUBECONFIG=bob.kubeconfig vela status podinfo -n dev
  7. About:
  9.   Name:         podinfo                      
  10.   Namespace:    dev                          
  11.   Created at:   2022-05-31 17:06:14 +0800 CST
  12.   Status:       running                      
  14. Workflow:
  16.   mode: DAG
  17.   finished: true
  18.   Suspend: false
  19.   Terminated: false
  20.   Steps
  21.   - id:rk3npcpycl
  22.     name:deploy-topology
  23.     type:deploy
  24.     phase:succeeded 
  25.     message:
  27. Services:
  29.   - Name: podinfo  
  30.     Cluster: local  Namespace: dev
  31.     Type: webservice
  32.     Healthy Ready:1/1
  33.     No trait applied
  35.   - Name: podinfo  
  36.     Cluster: c2  Namespace: dev
  37.     Type: webservice
  38.     Healthy Ready:1/1
  39.     No trait applied
  1. But he is forbidden to do any mutating actions, such as deleting application in dev namespace.
  1. $ KUBECONFIG=bob.kubeconfig vela delete podinfo-bad -n dev
  2. Deleting Application "podinfo-bad"
  3. Error: delete application err: applications.core.oam.dev "podinfo-bad" is forbidden: User "bob" cannot delete resource "applications" in API group "core.oam.dev" in the namespace "dev"
  4. 2022/05/31 17:17:51 delete application err: applications.core.oam.dev "podinfo-bad" is forbidden: User "bob" cannot delete resource "applications" in API group "core.oam.dev" in the namespace "dev"
  1. Instead, User Alice can delete application.
  1. $ KUBECONFIG=alice.kubeconfig vela delete podinfo-bad -n dev
  2. application.core.oam.dev "podinfo-bad" deleted
  1. (Optional) After vela-prism installed, you will be able to list resources of the application through the following command
  1. $ KUBECONFIG=bob.kubeconfig vela status podinfo -n dev --tree --detail
  2. CLUSTER       NAMESPACE     RESOURCE           STATUS    APPLY_TIME          DETAIL
  3. c2        ─── dev       ─── Deployment/podinfo updated   2022-05-31 17:06:14 Ready: 1/1  Up-to-date: 1  Available: 1  Age: 13m
  4. local     ─── dev       ─── Deployment/podinfo updated   2022-05-31 17:06:14 Ready: 1/1  Up-to-date: 1  Available: 1  Age: 13m

Notice that if vela-prism is not installed, Alice and Bob will be forbidden to run this listing command.

The guide above demonstrates how system operators can grant limited privileges for users and therefore restrict the access of their created applications. For more detail explanation on how this capability is achieved, read the Underlying Mechanism article.

As the platform builder, you may want to bind KubeVela application with your customized identity. For example, using a manual specified ServiceAccount for the application. If you want to do so, it is not mandatory to enable the Authentication feature flag in KubeVela. Read the System Integration for more details.

Last updated on Aug 4, 2023 by Daniel Higuero