Federate zone control plane

With Kuma you can first start with just one zone control plane and then federate it to a multi-zone deployment. This way you can:

  • see your mesh deployment in one centralized place
  • connect multiple zones and introduce cross zone connectivity
  • manage policies that are pushed to all zones

Prerequisites

Start a global control plane

Start a new Kubernetes cluster

Currently, it’s not possible to deploy global and zone control plane in the same Kubernetes cluster, therefore we need a new Kubernetes cluster.

You can skip this step if you already have a Kubernetes cluster running. It can be a cluster running locally or in a public cloud like AWS EKS, GCP GKE, etc.

  1. minikube start -p mesh-global

Setup the minikube tunnel

If you are using minikube for local testing, you can take advantage of the built-in minikube tunnel command. This command allows load balancer addresses to be provisioned using localhost.

Using nohup will allow the tunnel to continue running should your current terminal session end.

  1. nohup minikube tunnel -p mesh-global &

Deploy a global control plane

  1. helm install --kube-context=mesh-global --create-namespace --namespace kuma-system \
  2. --set controlPlane.mode=global \
  3. --set controlPlane.defaults.skipMeshCreation=true \
  4. kuma kuma/kuma

We skip default mesh creation as we will bring mesh from zone control plane in the next steps.

Sync endpoint

Find and save the external IP and port of the kuma-global-zone-sync service in the kuma-system namespace:

  1. export KDS_IP=$(kubectl --context=mesh-global get svc --namespace kuma-system kuma-global-zone-sync -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

If you are using minikube, you should use host.minikube.internal to ensure networking works correctly.

  1. export KDS_IP=host.minikube.internal

Copy resources from zone to global control plane

To federate zone control plane without any traffic interruption, we need to copy resources like secrets, meshes etc. First, we need to expose API server of zone control plane:

  1. kubectl --context=mesh-zone port-forward svc/kuma-control-plane -n kuma-system 5681:5681

Then we export resources:

  1. export ZONE_USER_ADMIN_TOKEN=$(kubectl --context=mesh-zone get secrets -n kuma-system admin-user-token -ojson | jq -r .data.value | base64 -d)
  2. kumactl config control-planes add \
  3.   --address http://localhost:5681 \
  4.   --headers "authorization=Bearer $ZONE_USER_ADMIN_TOKEN" \
  5.   --name "zone-cp" \
  6.   --overwrite  
  8. kumactl export --profile=federation-with-policies --format=kubernetes > resources.yaml

And finally, we apply resources on global control plane

  1. kubectl apply --context=mesh-global -f resources.yaml

Connect zone control plane to global control plane

Update Helm deployment of zone control plane to configure connection to the global control plane.

  1. helm upgrade --kube-context=mesh-zone --namespace kuma-system \
  2. --set controlPlane.mode=zone \
  3. --set controlPlane.zone=zone-1 \
  4. --set ingress.enabled=true \
  5. --set controlPlane.kdsGlobalAddress=grpcs://${KDS_IP}:5685 \
  6. --set controlPlane.tls.kdsZoneClient.skipVerify=true \
  7. kuma kuma/kuma

Verify federation

To verify federation, first port-forward the API service from the global control plane to port 15681 to avoid collision with previous port forward.

  1. kubectl --context=mesh-global port-forward svc/kuma-control-plane -n kuma-system 15681:5681

And then navigate to 127.0.0.1:15681/gui to see the GUI.

You should eventually see

  • a zone in list of zones
  • policies including redis MeshTrafficPermission that we applied in the quickstart guide.
  • data plane proxies for the demo application that we installed in the quickstart guide.

Apply policy on global control plane

We can check policy synchronization from global control plane to zone control plane by applying a policy on global control plane:

  1. echo "apiVersion: kuma.io/v1alpha1
  2. kind: MeshCircuitBreaker
  3. metadata:
  4.   name: demo-app-to-redis
  5.   namespace: kuma-demo
  6.   labels:
  7.     kuma.io/mesh: default
  8. spec:
  9.   targetRef:
  10.     kind: MeshSubset
  11.     tags:
  12.       app: demo-app
  13.   to:
  14.   - targetRef:
  15.       kind: MeshService
  16.       name: redis
  17.     default:
  18.       connectionLimits:
  19.         maxConnections: 2
  20.         maxPendingRequests: 8
  21.         maxRetries: 2
  22.         maxRequests: 2" | kubectl --context=mesh-global apply -f -

If we execute the following command:

  1. kubectl get --context=mesh-zone meshcircuitbreakers -A

The policy should be eventually available in zone control plane

  1. NAMESPACE     NAME                                                TARGETREF KIND   TARGETREF NAME
  2. kuma-system demo-app-to-redis-65xb45x2xfd5bf7f        MeshService      demo-app_kuma-demo_svc_5000
  3. kuma-system mesh-circuit-breaker-all-default          Mesh

Next steps

  • Read the multi-zone docs to learn more about this deployment model and cross-zone connectivity.