Working with Istio on flat network

This document uses an example to demonstrate how to use Istio on Karmada.

Follow this guide to install the Istio control plane on karmada-host (the primary cluster) and configure member1 and member2 (the remote cluster) to use the control plane in karmada-host. All clusters reside on the network1 network, meaning there is direct connectivity between the pods in both clusters.

Istio on Karmada

Install Karmada

Install karmada control plane

Following the steps Install karmada control plane in Quick Start, you can get a Karmada.

Deploy Istio

If you are testing multicluster setup on kind you can use MetalLB to make use of EXTERNAL-IP for LoadBalancer services.

Install istioctl

Please refer to the istioctl Installation.

Prepare CA certificates

Following the steps plug-in-certificates-and-key-into-the-cluster to configure Istio CA.

Replace the cluster name cluster1 with primary, the output will looks like as follwing:

  1. root@karmada-demo  istio-on-karmada# tree certs
  2. certs
  3. ├── primary
  4. │   ├── ca-cert.pem
  5. │   ├── ca-key.pem
  6. │   ├── cert-chain.pem
  7. │   └── root-cert.pem
  8. ├── root-ca.conf
  9. ├── root-cert.csr
  10. ├── root-cert.pem
  11. ├── root-cert.srl
  12. └── root-key.pem

Install Istio on karmada-apiserver

Export KUBECONFIG and switch to karmada apiserver:

  1. # export KUBECONFIG=$HOME/.kube/karmada.config
  3. # kubectl config use-context karmada-apiserver

Create a secret cacerts in istio-system namespace:

  1. kubectl create namespace istio-system
  2. kubectl create secret generic cacerts -n istio-system \
  3.     --from-file=certs/primary/ca-cert.pem \
  4.     --from-file=certs/primary/ca-key.pem \
  5.     --from-file=certs/primary/root-cert.pem \
  6.     --from-file=certs/primary/cert-chain.pem

Create a propagation policy for cacert secret:

  1. cat <<EOF | kubectl apply -f -
  2. apiVersion: policy.karmada.io/v1alpha1
  3. kind: PropagationPolicy
  4. metadata:
  5.   name: cacerts-propagation
  6.   namespace: istio-system
  7. spec:
  8.   resourceSelectors:
  9.     - apiVersion: v1
  10.       kind: Secret
  11.       name: cacerts
  12.   placement:
  13.     clusterAffinity:
  14.       clusterNames:
  15.         - member1
  16.         - member2
  17. EOF

Run the following command to install istio CRDs on karmada apiserver:

  1. cat <<EOF | istioctl install -y --set profile=minimal -f -
  2. apiVersion: install.istio.io/v1alpha1
  3. kind: IstioOperator
  4. spec:
  5.   meshConfig:
  6.     accessLogFile: /dev/stdout
  7.   values:
  8.     global:
  9.       meshID: mesh1
  10.       multiCluster:
  11.         clusterName: primary
  12.       network: network1
  13. EOF

Karmada apiserver will not deploy a real istiod pod, you should press ctrl+c to exit installation when Processing resources for Istiod.

  1. ✔ Istio core installed                                                                                                                                                         
  2. - Processing resources for Istiod.

Install Istio on karmada host

  1. Create secret on karmada-host

Karmada host is not a member cluster, we need create the cacerts secret for istiod.

Export KUBECONFIG and switch to karmada host:

  1. # export KUBECONFIG=$HOME/.kube/karmada.config
  3. # kubectl config use-context karmada-host

Create a secret cacerts in istio-system namespace:

  1. kubectl create namespace istio-system
  2. kubectl create secret generic cacerts -n istio-system \
  3.     --from-file=certs/primary/ca-cert.pem \
  4.     --from-file=certs/primary/ca-key.pem \
  5.     --from-file=certs/primary/root-cert.pem \
  6.     --from-file=certs/primary/cert-chain.pem
  1. Create istio-kubeconfig on karmada-host
  1. kubectl get secret -nkarmada-system kubeconfig --template={{.data.kubeconfig}} | base64 -d > kind-karmada.yaml
  1. kubectl create secret generic istio-kubeconfig --from-file=config=kind-karmada.yaml -nistio-system
  1. Install istio control plane
  1. cat <<EOF | istioctl install -y --set profile=minimal -f -
  2. apiVersion: install.istio.io/v1alpha1
  3. kind: IstioOperator
  4. spec:
  5.   meshConfig:
  6.     accessLogFile: /dev/stdout
  7.   values:
  8.     global:
  9.       meshID: mesh1
  10.       multiCluster:
  11.         clusterName: primary
  12.       network: network1
  13. EOF
  1. Expose istiod service

Run the following command to create a service for the istiod service:

  1. cat <<EOF | kubectl apply -f -
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5.   name: istiod-elb
  6.   namespace: istio-system
  7. spec:
  8.   ports:
  9.   - name: https-dns
  10.     port: 15012
  11.     protocol: TCP
  12.     targetPort: 15012
  13.   selector:
  14.     app: istiod
  15.     istio: pilot
  16.   sessionAffinity: None
  17.   type: LoadBalancer
  18. EOF

Export DISCOVERY_ADDRESS:

  1. export DISCOVERY_ADDRESS=$(kubectl get svc istiod-elb -nistio-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  2. # verify
  3. echo $DISCOVERY_ADDRESS

Prepare member1 cluster secret

  1. Export KUBECONFIG and switch to karmada member1:
  1. export KUBECONFIG="$HOME/.kube/members.config"
  2. kubectl config use-context member1
  1. Create istio remote secret for member1:
  1. istioctl x create-remote-secret --name=member1 > istio-remote-secret-member1.yaml

Prepare member2 cluster secret

  1. Export KUBECONFIG and switch to karmada member2:
  1. export KUBECONFIG="$HOME/.kube/members.config"
  2. kubectl config use-context member2
  1. Create istio remote secret for member1:
  1. istioctl x create-remote-secret --name=member2 > istio-remote-secret-member2.yaml

Apply istio remote secret

Export KUBECONFIG and switch to karmada apiserver:

  1. # export KUBECONFIG=$HOME/.kube/karmada.config
  3. # kubectl config use-context karmada-apiserver

Apply istio remote secret:

  1. kubectl apply -f istio-remote-secret-member1.yaml
  3. kubectl apply -f istio-remote-secret-member2.yaml

Install istio remote

  1. Install istio remote member1

Export KUBECONFIG and switch to karmada member1:

  1. export KUBECONFIG="$HOME/.kube/members.config"
  2. kubectl config use-context member1
  1. cat <<EOF | istioctl install -y -f -
  2. apiVersion: install.istio.io/v1alpha1
  3. kind: IstioOperator
  4. spec:
  5.   values:
  6.     global:
  7.       meshID: mesh1
  8.       multiCluster:
  9.         clusterName: member1
  10.       network: network1
  11.       remotePilotAddress: ${DISCOVERY_ADDRESS}
  12. EOF
  1. Install istio remote member2

Export KUBECONFIG and switch to karmada member2:

  1. export KUBECONFIG="$HOME/.kube/members.config"
  2. kubectl config use-context member2
  1. cat <<EOF | istioctl install -y -f -
  2. apiVersion: install.istio.io/v1alpha1
  3. kind: IstioOperator
  4. spec:
  5.   values:
  6.     global:
  7.       meshID: mesh1
  8.       multiCluster:
  9.         clusterName: member2
  10.       network: network1
  11.       remotePilotAddress: ${DISCOVERY_ADDRESS}
  12. EOF

Deploy bookinfo application

Export KUBECONFIG and switch to karmada apiserver:

  1. # export KUBECONFIG=$HOME/.kube/karmada.config
  3. # kubectl config use-context karmada-apiserver

Create an istio-demo namespace:

  1. kubectl create namespace istio-demo

Label the namespace that will host the application with istio-injection=enabled:

  1. kubectl label namespace istio-demo istio-injection=enabled

Deploy your application using the kubectl command:

  1. kubectl apply -nistio-demo -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/platform/kube/bookinfo.yaml

Run the following command to create default destination rules for the Bookinfo services:

  1. kubectl apply -nistio-demo -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/destination-rule-all.yaml

Run the following command to create virtual service for the Bookinfo services:

  1. kubectl apply -nistio-demo -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/bookinfo/networking/virtual-service-all-v1.yaml

Run the following command to create propagation policy for the Bookinfo services:

  1. cat <<EOF | kubectl apply -nistio-demo -f -
  2. apiVersion: policy.karmada.io/v1alpha1
  3. kind: PropagationPolicy
  4. metadata:
  5.   name: service-propagation
  6. spec:
  7.   resourceSelectors:
  8.     - apiVersion: v1
  9.       kind: Service
  10.       name: productpage
  11.     - apiVersion: v1
  12.       kind: Service
  13.       name: details
  14.     - apiVersion: v1
  15.       kind: Service
  16.       name: reviews
  17.     - apiVersion: v1
  18.       kind: Service
  19.       name: ratings
  20.   placement:
  21.     clusterAffinity:
  22.       clusterNames:
  23.         - member1
  24.         - member2
  25. ---
  26. apiVersion: policy.karmada.io/v1alpha1
  27. kind: PropagationPolicy
  28. metadata:
  29.   name: produtpage-propagation
  30. spec:
  31.   resourceSelectors:
  32.     - apiVersion: apps/v1
  33.       kind: Deployment
  34.       name: productpage-v1
  35.     - apiVersion: v1
  36.       kind: ServiceAccount
  37.       name: bookinfo-productpage
  38.   placement:
  39.     clusterAffinity:
  40.       clusterNames:
  41.         - member1
  42. ---
  43. apiVersion: policy.karmada.io/v1alpha1
  44. kind: PropagationPolicy
  45. metadata:
  46.   name: details-propagation
  47. spec:
  48.   resourceSelectors:
  49.     - apiVersion: apps/v1
  50.       kind: Deployment
  51.       name: details-v1
  53.     - apiVersion: v1
  54.       kind: ServiceAccount
  55.       name: bookinfo-details
  56.   placement:
  57.     clusterAffinity:
  58.       clusterNames:
  59.         - member2
  60. ---
  61. apiVersion: policy.karmada.io/v1alpha1
  62. kind: PropagationPolicy
  63. metadata:
  64.   name: reviews-propagation
  65. spec:
  66.   resourceSelectors:
  67.     - apiVersion: apps/v1
  68.       kind: Deployment
  69.       name: reviews-v1
  70.     - apiVersion: apps/v1
  71.       kind: Deployment
  72.       name: reviews-v2
  73.     - apiVersion: apps/v1
  74.       kind: Deployment
  75.       name: reviews-v3
  76.     - apiVersion: v1
  77.       kind: ServiceAccount
  78.       name: bookinfo-reviews
  79.   placement:
  80.     clusterAffinity:
  81.       clusterNames:
  82.         - member1
  83.         - member2
  84. ---
  85. apiVersion: policy.karmada.io/v1alpha1
  86. kind: PropagationPolicy
  87. metadata:
  88.   name: ratings-propagation
  89. spec:
  90.   resourceSelectors:
  91.     - apiVersion: apps/v1
  92.       kind: Deployment
  93.       name: ratings-v1
  94.     - apiVersion: v1
  95.       kind: ServiceAccount
  96.       name: bookinfo-ratings
  97.   placement:
  98.     clusterAffinity:
  99.       clusterNames:
  100.         - member2
  101. EOF

Deploy fortio application using the kubectl command:

  1. kubectl apply -nistio-demo -f https://raw.githubusercontent.com/istio/istio/release-1.12/samples/httpbin/sample-client/fortio-deploy.yaml

Run the following command to create propagation policy for the fortio services:

  1. cat <<EOF | kubectl apply -nistio-demo -f -
  2. apiVersion: policy.karmada.io/v1alpha1
  3. kind: PropagationPolicy
  4. metadata:
  5.   name: fortio-propagation
  6. spec:
  7.   resourceSelectors:
  8.     - apiVersion: v1
  9.       kind: Service
  10.       name: fortio
  11.     - apiVersion: apps/v1
  12.       kind: Deployment
  13.       name: fortio-deploy
  14.   placement:
  15.     clusterAffinity:
  16.       clusterNames:
  17.         - member1
  18.         - member2
  19. EOF

Export KUBECONFIG and switch to karmada member1:

  1. export KUBECONFIG="$HOME/.kube/members.config"
  2. kubectl config use-context member1

Run the following command to verify productpage application installation:

  1. export FORTIO_POD=`kubectl get po -nistio-demo | grep fortio | awk '{print $1}'`
  2. kubectl exec -it ${FORTIO_POD} -nistio-demo -- fortio load -t 3s productpage:9080/productpage

What’s next

Folling the guide to confirm the app is accessible from outside the cluster.