Version: v1.3

Multi-cluster Distribution

This section requires you to know the basics about how to deploy multi-cluster application with policy and workflow. You can refer to Multi-cluster Delivery for container images, they’re working in the same way.

You can reference and distribute existing Kubernetes objects with KubeVela in the following scenarios:

  • Copying secrets from the hub cluster into managed clusters.
  • Promote deployments from canary clusters into production clusters.
  • Using Kubernetes apiserver as the control plane and storing all Kubernetes objects data in external databases. Then dispatch those data into real Kuberenetes managed clusters.

To use existing Kubernetes objects in the component, you need to use the ref-objects typed component and declare which resources you want to refer to. For example, in the following example, the secret image-credential-to-copy in namespace examples will be taken as the source object for the component. Then you can use the topology policy to dispatch it into hangzhou clusters.

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: ref-objects-example
  5.   namespace: examples
  6. spec:
  7.   components:
  8.     - name: image-pull-secrets
  9.       type: ref-objects
  10.       properties:
  11.         objects:
  12.           - resource: secret
  13.             name: image-credential-to-copy
  14.   policies:
  15.     - name: topology-hangzhou-clusters
  16.       type: topology
  17.       properties:
  18.         clusterLabelSelector:
  19.           region: hangzhou

The most simple way to specify resources is to directly use resource: secret or resource: deployment to describe the kind of resources. If no name or labelSelector is set, the application will try to find the resource with the same name as the component name in the application’s namespace. You can also explicitly specify name and namespace for the target resource as well.

In addition to name and namespace, you can also specify the cluster field to let the application component refer to resources in managed clusters. You can also use the labelSelector to select resources in replace of finding resources by names.

In the following example, the application will select all deployments in the hangzhou-1 cluster inside the examples namespace, which matches the desided labels. Then the application will copy these deployments into hangzhou-2 cluster.

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: ref-objects-duplicate-deployments
  5.   namespace: examples
  6. spec:
  7.   components:
  8.     - name: duplicate-deployment
  9.       type: ref-objects
  10.       properties:
  11.         objects:
  12.           - resource: deployment
  13.             cluster: hangzhou-1
  14.             # select all deployment in the `examples` namespace in cluster `hangzhou-1` that matches the labelSelector
  15.             labelSelector:
  16.               need-duplicate: "true"
  17.   policies:
  18.     - name: topology-hangzhou-2
  19.       type: topology
  20.       properties:
  21.         clusters: ["hangzhou-2"]

In some cases, you might want to restrict the scope for the application to access resources. You can set the --ref-objects-available-scope to namespace or cluster in KubeVela controller’s bootstrap parameter, to retrict the application to be only able to refer to the resources inside the same namespace or the same cluster.

The ref-objects typed component can also be used together with traits. The implicit main workload is the first referenced object and trait patch will be applied on it. The following example demonstrate how to set the replica number for the referenced deployment while deploying it in hangzhou clusters.

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4.   name: ref-objects-multiple-resources
  5.   namespace: examples
  6. spec:
  7.   components:
  8.     - name: nginx-ref-multiple-resources
  9.       type: ref-objects
  10.       properties:
  11.         objects:
  12.           - resource: deployment
  13.           - resource: service
  14.       traits:
  15.         - type: scaler
  16.           properties:
  17.             replicas: 3
  18.   policies:
  19.     - name: topology-hangzhou-clusters
  20.       type: topology
  21.       properties:
  22.         clusterLabelSelector:
  23.           region: hangzhou

Last updated on Nov 1, 2022 by Tianxin Dong