Google Kubernetes Engine

This example shows how to configure a multicluster mesh with asingle-network deploymentover 2 Google Kubernetes Engine clusters.

Before you begin

In addition to the prerequisites for installing Istio the following setup is required for this example:

  • This sample requires a valid Google Cloud Platform project with billing enabled. If you arenot an existing GCP user, you may be able to enroll for a $300 US Free Trial credit.

  • Install and initialize the Google Cloud SDK

Create the GKE clusters

  • Set the default project for gcloud to perform actions on:
  1. $ gcloud config set project myProject
  2. $ proj=$(gcloud config list --format='value(core.project)')
  • Create 2 GKE clusters for use with the multicluster feature. Note: —enable-ip-alias is required toallow inter-cluster direct pod-to-pod communication. The zone value must be one of theGCP zones.
  1. $ zone="us-east1-b"
  2. $ cluster="cluster-1"
  3. $ gcloud container clusters create $cluster --zone $zone --username "admin" \
  4.   --machine-type "n1-standard-2" --image-type "COS" --disk-size "100" \
  5.   --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only",\
  6. "https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring",\
  7. "https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly",\
  8. "https://www.googleapis.com/auth/trace.append" \
  9. --num-nodes "4" --network "default" --enable-cloud-logging --enable-cloud-monitoring --enable-ip-alias --async
  10. $ cluster="cluster-2"
  11. $ gcloud container clusters create $cluster --zone $zone --username "admin" \
  12.   --machine-type "n1-standard-2" --image-type "COS" --disk-size "100" \
  13.   --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only",\
  14. "https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/monitoring",\
  15. "https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly",\
  16. "https://www.googleapis.com/auth/trace.append" \
  17. --num-nodes "4" --network "default" --enable-cloud-logging --enable-cloud-monitoring --enable-ip-alias --async
  • Wait for clusters to transition to the RUNNING state by polling their statuses via the following command:
  1. $ gcloud container clusters list
  1. $ gcloud container clusters get-credentials cluster-1 --zone $zone
  2. $ gcloud container clusters get-credentials cluster-2 --zone $zone

  • Validate kubectl access to each cluster and create a cluster-admin cluster role binding tied to the Kubernetes credentials associated with your GCP user.

    • For cluster-1:
  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-1"
  2. $ kubectl get pods --all-namespaces
  3. $ kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user="$(gcloud config get-value core/account)"
  • For cluster-2:
  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-2"
  2. $ kubectl get pods --all-namespaces
  3. $ kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user="$(gcloud config get-value core/account)"

Create a Google Cloud firewall rule

To allow the pods on each cluster to directly communicate, create the following rule:

  1. $ function join_by { local IFS="$1"; shift; echo "$*"; }
  2. $ ALL_CLUSTER_CIDRS=$(gcloud container clusters list --format='value(clusterIpv4Cidr)' | sort | uniq)
  3. $ ALL_CLUSTER_CIDRS=$(join_by , $(echo "${ALL_CLUSTER_CIDRS}"))
  4. $ ALL_CLUSTER_NETTAGS=$(gcloud compute instances list --format='value(tags.items.[0])' | sort | uniq)
  5. $ ALL_CLUSTER_NETTAGS=$(join_by , $(echo "${ALL_CLUSTER_NETTAGS}"))
  6. $ gcloud compute firewall-rules create istio-multicluster-test-pods \
  7.   --allow=tcp,udp,icmp,esp,ah,sctp \
  8.   --direction=INGRESS \
  9.   --priority=900 \
  10.   --source-ranges="${ALL_CLUSTER_CIDRS}" \
  11.   --target-tags="${ALL_CLUSTER_NETTAGS}" --quiet

Install the Istio control plane

The following generates an Istio installation manifest, installs it, and enables automatic sidecar injection inthe default namespace:

  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-1"
  2. $ helm template install/kubernetes/helm/istio --name istio --namespace istio-system > $HOME/istio_master.yaml
  3. $ kubectl create ns istio-system
  4. $ helm template install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl apply -f -
  5. $ kubectl apply -f $HOME/istio_master.yaml
  6. $ kubectl label namespace default istio-injection=enabled

Wait for pods to come up by polling their statuses via the following command:

  1. $ kubectl get pods -n istio-system

Generate remote cluster manifest

  • Get the IPs of the control plane pods:
  1. $ export PILOT_POD_IP=$(kubectl -n istio-system get pod -l istio=pilot -o jsonpath='{.items[0].status.podIP}')
  2. $ export POLICY_POD_IP=$(kubectl -n istio-system get pod -l istio=mixer -o jsonpath='{.items[0].status.podIP}')
  3. $ export TELEMETRY_POD_IP=$(kubectl -n istio-system get pod -l istio-mixer-type=telemetry -o jsonpath='{.items[0].status.podIP}')
  • Generate remote cluster manifest:

Zip

  1. $ helm template install/kubernetes/helm/istio \
  2.   --namespace istio-system --name istio-remote \
  3.   --values @install/kubernetes/helm/istio/values-istio-remote.yaml@ \
  4.   --set global.remotePilotAddress=${PILOT_POD_IP} \
  5.   --set global.remotePolicyAddress=${POLICY_POD_IP} \
  6.   --set global.remoteTelemetryAddress=${TELEMETRY_POD_IP} > $HOME/istio-remote.yaml

Install remote cluster manifest

The following installs the minimal Istio components and enables automatic sidecar injection onthe namespace default in the remote cluster:

  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-2"
  2. $ kubectl create ns istio-system
  3. $ kubectl apply -f $HOME/istio-remote.yaml
  4. $ kubectl label namespace default istio-injection=enabled

Create remote cluster’s kubeconfig for Istio Pilot

The istio-remote Helm chart creates a service account with minimal access for use by Istio Pilotdiscovery.

  • Prepare environment variables for building the kubeconfig file for the service account istio-multi:
  1. $ export WORK_DIR=$(pwd)
  2. $ CLUSTER_NAME=$(kubectl config view --minify=true -o jsonpath='{.clusters[].name}')
  3. $ CLUSTER_NAME="${CLUSTER_NAME##*_}"
  4. $ export KUBECFG_FILE=${WORK_DIR}/${CLUSTER_NAME}
  5. $ SERVER=$(kubectl config view --minify=true -o jsonpath='{.clusters[].cluster.server}')
  6. $ NAMESPACE=istio-system
  7. $ SERVICE_ACCOUNT=istio-multi
  8. $ SECRET_NAME=$(kubectl get sa ${SERVICE_ACCOUNT} -n ${NAMESPACE} -o jsonpath='{.secrets[].name}')
  9. $ CA_DATA=$(kubectl get secret ${SECRET_NAME} -n ${NAMESPACE} -o jsonpath="{.data['ca\.crt']}")
  10. $ TOKEN=$(kubectl get secret ${SECRET_NAME} -n ${NAMESPACE} -o jsonpath="{.data['token']}" | base64 --decode)

An alternative to base64 —decode is openssl enc -d -base64 -A on many systems.

  • Create a kubeconfig file in the working directory for the service account istio-multi:
  1. $ cat <<EOF > ${KUBECFG_FILE}
  2. apiVersion: v1
  3. clusters:
  4.    - cluster:
  5.        certificate-authority-data: ${CA_DATA}
  6.        server: ${SERVER}
  7.      name: ${CLUSTER_NAME}
  8. contexts:
  9.    - context:
  10.        cluster: ${CLUSTER_NAME}
  11.        user: ${CLUSTER_NAME}
  12.      name: ${CLUSTER_NAME}
  13. current-context: ${CLUSTER_NAME}
  14. kind: Config
  15. preferences: {}
  16. users:
  17.    - name: ${CLUSTER_NAME}
  18.      user:
  19.        token: ${TOKEN}
  20. EOF

At this point, the remote clusters’ kubeconfig files have been created in the ${WORK_DIR} directory.The filename for a cluster is the same as the original kubeconfig cluster name.

Configure Istio control plane to discover the remote cluster

Create a secret and label it properly for each remote cluster:

  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-1"
  2. $ kubectl create secret generic ${CLUSTER_NAME} --from-file ${KUBECFG_FILE} -n ${NAMESPACE}
  3. $ kubectl label secret ${CLUSTER_NAME} istio/multiCluster=true -n ${NAMESPACE}

Deploy the Bookinfo example across clusters

  • Install Bookinfo on the first cluster. Remove the reviews-v3 deployment to deploy on remote:

ZipZip

  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-1"
  2. $ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@
  3. $ kubectl apply -f @samples/bookinfo/networking/bookinfo-gateway.yaml@
  4. $ kubectl delete deployment reviews-v3
  • Install the reviews-v3 deployment on the remote cluster.

ZipZipZipZip

  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-2"
  2. $ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@ -l service=ratings
  3. $ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@ -l service=reviews
  4. $ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@ -l account=reviews
  5. $ kubectl apply -f @samples/bookinfo/platform/kube/bookinfo.yaml@ -l app=reviews,version=v3

Note: The ratings service definition is added to the remote cluster because reviews-v3 is aclient of ratings and creating the service object creates a DNS entry. The Istio sidecar in thereviews-v3 pod will determine the proper ratings endpoint after the DNS lookup is resolved to aservice address. This would not be necessary if a multicluster DNS solution were additionally set up, e.g. asin a federated Kubernetes environment.

  • Get the istio-ingressgateway service’s external IP to access the bookinfo page to validate that Istiois including the remote’s reviews-v3 instance in the load balancing of reviews versions:
  1. $ kubectl config use-context "gke_${proj}_${zone}_cluster-1"
  2. $ kubectl get svc istio-ingressgateway -n istio-system

Access http://<GATEWAY_IP>/productpage repeatedly and each version of reviews should be equally loadbalanced,including reviews-v3 in the remote cluster (red stars). It may take several accesses (dozens) to demonstratethe equal loadbalancing between reviews versions.

Uninstalling

The following should be done in addition to the uninstall of Istio as described in theVPN-based multicluster uninstall section:

  • Delete the Google Cloud firewall rule:
  1. $ gcloud compute firewall-rules delete istio-multicluster-test-pods --quiet
  • Delete the cluster-admin cluster role binding from each cluster no longer being used for Istio:
  1. $ kubectl delete clusterrolebinding gke-cluster-admin-binding
  • Delete any GKE clusters no longer in use. The following is an example delete command for the remote cluster, cluster-2:
  1. $ gcloud container clusters delete cluster-2 --zone $zone

See also

IBM Cloud Private

Example multicluster mesh over two IBM Cloud Private clusters.

Replicated control planes

Install an Istio mesh across multiple Kubernetes clusters with replicated control plane instances.

Shared control plane (multi-network)

Install an Istio mesh across multiple Kubernetes clusters using a shared control plane for disconnected cluster networks.

Shared control plane (single-network)

Install an Istio mesh across multiple Kubernetes clusters with a shared control plane and VPN connectivity between clusters.

Simplified Multicluster Install [Experimental]

Configure an Istio mesh spanning multiple Kubernetes clusters.

DNS Certificate Management

Provision and manage DNS certificates in Istio.