Istio service mesh

Service mesh allows to monitor, visualize and control traffic between pods. Kubevirt supports running VMs as a part of Istio service mesh.

Limitations

  • Istio service mesh is only supported with a pod network masquerade or passt binding.

  • Istio uses a list of ports for its own purposes, these ports must not be explicitly specified in a VMI interface.

  • Istio only supports IPv4.

Prerequisites

  • This guide assumes that Istio is already deployed and uses Istio CNI Plugin. See Istio documentation for more information.

  • Optionally, istioctl binary for troubleshooting. See Istio installation inctructions.

  • The target namespace where the VM is created must be labelled with istio-injection=enabled label.

  • If Multus is used to manage CNI, the following NetworkAttachmentDefinition is required in the application namespace:

    1. apiVersion: "k8s.cni.cncf.io/v1"
    2. kind: NetworkAttachmentDefinition
    3. metadata:
    4.   name: istio-cni

Create a VirtualMachineInstance with enabled Istio proxy injecton

The example below specifies a VMI with masquerade network interface and sidecar.istio.io/inject annotation to register the VM to the service mesh.

  1. apiVersion: kubevirt.io/v1
  2. kind: VirtualMachineInstance
  3. metadata:
  4.   annotations:
  5.     sidecar.istio.io/inject: "true"
  6.   labels:
  7.     app: vmi-istio
  8.   name: vmi-istio
  9. spec:
  10.   domain:
  11.     devices:
  12.       interfaces:
  13.         - name: default
  14.           masquerade: {}
  15.       disks:
  16.         - disk:
  17.             bus: virtio
  18.           name: containerdisk
  19.     resources:
  20.       requests:
  21.         memory: 1024M
  22.   networks:
  23.     - name: default
  24.       pod: {}
  25.   terminationGracePeriodSeconds: 0
  26.   volumes:
  27.     - name: containerdisk
  28.       containerDisk:
  29.         image: registry:5000/kubevirt/fedora-cloud-container-disk-demo:devel

Istio expects each application to be associated with at least one Kubernetes service. Create the following Service exposing port 8080:

  1. apiVersion: v1
  2. kind: Service
  3. metadata:
  4.   name: vmi-istio
  5. spec:
  6.   selector:
  7.     app: vmi-istio
  8.   ports:
  9.     - port: 8080
  10.       name: http
  11.       protocol: TCP

Note: Each Istio enabled VMI must feature the sidecar.istio.io/inject annotation instructing KubeVirt to perform necessary network configuration.

Verification

Verify istio-proxy sidecar is deployed and able to synchronize with Istio control plane using istioctl proxy-status command. See Istio Debbuging Envoy and Istiod documentation section for more information about proxy-status subcommand.

  1. $ kubectl get pods
  2. NAME                             READY   STATUS    RESTARTS   AGE
  3. virt-launcher-vmi-istio-ncx7r    3/3     Running   0          7s
  5. $ kubectl get pods virt-launcher-vmi-istio-ncx7r -o jsonpath='{.spec.containers[*].name}'
  6. compute volumecontainerdisk istio-proxy
  8. $ istioctl proxy-status
  9. NAME                                    CDS        LDS        EDS        RDS          ISTIOD                      VERSION
  10. ...
  11. virt-launcher-vmi-istio-ncx7r.default   SYNCED     SYNCED     SYNCED     SYNCED       istiod-7c4d8c7757-hshj5     1.10.0

Troubleshooting

Istio sidecar is not deployed

  1. $ kubectl get pods
  2. NAME                             READY   STATUS    RESTARTS   AGE
  3. virt-launcher-vmi-istio-jnw6p    2/2     Running   0          37s
  5. $ kubectl get pods virt-launcher-vmi-istio-jnw6p -o jsonpath='{.spec.containers[*].name}'
  6. compute volumecontainerdisk

Resolution: Make sure the istio-injection=enabled is added to the target namespace. If the issue persists, consult relevant part of Istio documentation.

Istio sidecar is not ready

  1. $ kubectl get pods
  2. NAME                             READY   STATUS    RESTARTS   AGE
  3. virt-launcher-vmi-istio-lg5gp    2/3     Running   0          90s
  5. $ kubectl describe pod virt-launcher-vmi-istio-lg5gp
  6.   ...
  7.   Warning  Unhealthy  2d8h (x3 over 2d8h)  kubelet            Readiness probe failed: Get "http://10.244.186.222:15021/healthz/ready": dial tcp 10.244.186.222:15021: connect: no route to host
  8.   Warning  Unhealthy  2d8h (x4 over 2d8h)  kubelet            Readiness probe failed: Get "http://10.244.186.222:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Resolution: Make sure the sidecar.istio.io/inject: "true" annotation is defined in the created VMI and that masquerade or passt binding is used for pod network interface.

Virt-launcher pod for VMI is stuck at initialization phase

  1. $ kubectl get pods
  2. NAME                             READY   STATUS     RESTARTS   AGE
  3. virt-launcher-vmi-istio-44mws    0/3     Init:0/3   0          29s
  5. $ kubectl describe pod virt-launcher-vmi-istio-44mws
  6.   ...
  7.   Multus: [default/virt-launcher-vmi-istio-44mws]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in getting k8s network for pod: GetNetworkDelegates: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (istio-cni) in namespace (default): network-attachment-definitions.k8s.cni.cncf.io "istio-cni" not found

Resolution: Make sure the istio-cni NetworkAttachmentDefinition (provided in the Prerequisites section) is created in the target namespace.