Templates

Note

By deploying KubeVirt on top of OpenShift the user can benefit from the OpenShift Template functionality.

Virtual machine templates

What is a virtual machine template?

The KubeVirt projects provides a set of templates to create VMs to handle common usage scenarios. These templates provide a combination of some key factors that could be further customized and processed to have a Virtual Machine object. The key factors which define a template are

  • Workload Most Virtual Machine should be server or desktop to have maximum flexibility; the highperformance workload trades some of this flexibility to provide better performances.

  • Guest Operating System (OS) This allow to ensure that the emulated hardware is compatible with the guest OS. Furthermore, it allows to maximize the stability of the VM, and allows performance optimizations.

  • Size (flavor) Defines the amount of resources (CPU, memory) to allocate to the VM.

More documentation is available in the common templates subproject

Accessing the virtual machine templates

If you installed KubeVirt using a supported method you should find the common templates preinstalled in the cluster. Should you want to upgrade the templates, or install them from scratch, you can use one of the supported releases

To install the templates:

  1.     $ export VERSION=$(curl -s https://api.github.com/repos/kubevirt/common-templates/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
  2.     $ oc create -f https://github.com/kubevirt/common-templates/releases/download/$VERSION/common-templates-$VERSION.yaml

Editable fields

You can edit the fields of the templates which define the amount of resources which the VMs will receive.

Each template can list a different set of fields that are to be considered editable. The fields are used as hints for the user interface, and also for other components in the cluster.

The editable fields are taken from annotations in the template. Here is a snippet presenting a couple of most commonly found editable fields:

  1.     metadata:
  2.       annotations:
  3.         template.kubevirt.io/editable: |
  4.           /objects[0].spec.template.spec.domain.cpu.sockets
  5.           /objects[0].spec.template.spec.domain.cpu.cores
  6.           /objects[0].spec.template.spec.domain.cpu.threads
  7.           /objects[0].spec.template.spec.domain.resources.requests.memory

Each entry in the editable field list must be a jsonpath. The jsonpath root is the objects: element of the template. The actually editable field is the last entry (the “leaf”) of the path. For example, the following minimal snippet highlights the fields which you can edit:

  1.     objects:
  2.       spec:
  3.         template:
  4.           spec:
  5.             domain:
  6.               cpu:
  7.                 sockets:
  8.                   VALUE # this is editable
  9.                 cores:
  10.                   VALUE # this is editable
  11.                 threads:
  12.                   VALUE # this is editable
  13.               resources:
  14.                 requests:
  15.                   memory:
  16.                     VALUE # this is editable

Relationship between templates and VMs

Once processed the templates produce VM objects to be used in the cluster. The VMs produced from templates will have a vm.kubevirt.io/template label, whose value will be the name of the parent template, for example fedora-desktop-medium:

  1.       metadata:
  2.         labels:
  3.           vm.kubevirt.io/template: fedora-desktop-medium

In addition, these VMs can include an optional label vm.kubevirt.io/template-namespace, whose value will be the namespace of the parent template, for example:

  1.       metadata:
  2.         labels:
  3.           vm.kubevirt.io/template-namespace: openshift

If this label is not defined, the template is expected to belong to the same namespace as the VM.

This make it possible to query for all the VMs built from any template.

Example:

  1.     oc process -o yaml -f dist/templates/rhel8-server-tiny.yaml NAME=rheltinyvm SRC_PVC_NAME=rhel SRC_PVC_NAMESPACE=kubevirt

And the output:

  1. apiVersion: v1
  2. items:
  3. - apiVersion: kubevirt.io/v1
  4.   kind: VirtualMachine
  5.   metadata:
  6.     annotations:
  7.       vm.kubevirt.io/flavor: tiny
  8.       vm.kubevirt.io/os: rhel8
  9.       vm.kubevirt.io/validations: |
  10.         [
  11.           {
  12.             "name": "minimal-required-memory",
  13.             "path": "jsonpath::.spec.domain.resources.requests.memory",
  14.             "rule": "integer",
  15.             "message": "This VM requires more memory.",
  16.             "min": 1610612736
  17.           }
  18.         ]
  19.       vm.kubevirt.io/workload: server
  20.     labels:
  21.       app: rheltinyvm
  22.       vm.kubevirt.io/template: rhel8-server-tiny
  23.       vm.kubevirt.io/template.revision: "45"
  24.       vm.kubevirt.io/template.version: 0.11.3
  25.     name: rheltinyvm
  26.   spec:
  27.     dataVolumeTemplates:
  28.     - apiVersion: cdi.kubevirt.io/v1beta1
  29.       kind: DataVolume
  30.       metadata:
  31.         name: rheltinyvm
  32.       spec:
  33.         storage:
  34.           accessModes:
  35.           - ReadWriteMany
  36.         source:
  37.           pvc:
  38.             name: rhel
  39.             namespace: kubevirt
  40.     runStrategy: Halted
  41.     template:
  42.       metadata:
  43.         labels:
  44.           kubevirt.io/domain: rheltinyvm
  45.           kubevirt.io/size: tiny
  46.       spec:
  47.         domain:
  48.           cpu:
  49.             cores: 1
  50.             sockets: 1
  51.             threads: 1
  52.           devices:
  53.             disks:
  54.             - disk:
  55.                 bus: virtio
  56.               name: rheltinyvm
  57.             - disk:
  58.                 bus: virtio
  59.               name: cloudinitdisk
  60.             interfaces:
  61.             - masquerade: {}
  62.               name: default
  63.             networkInterfaceMultiqueue: true
  64.             rng: {}
  65.           resources:
  66.             requests:
  67.               memory: 1.5Gi
  68.         networks:
  69.         - name: default
  70.           pod: {}
  71.         terminationGracePeriodSeconds: 180
  72.         volumes:
  73.         - dataVolume:
  74.             name: rheltinyvm
  75.           name: rheltinyvm
  76.         - cloudInitNoCloud:
  77.             userData: |-
  78.               #cloud-config
  79.               user: cloud-user
  80.               password: lymp-fda4-m1cv
  81.               chpasswd: { expire: False }
  82.           name: cloudinitdisk
  83. kind: List
  84. metadata: {}

You can add the VM from the template to the cluster in one go

  1.     oc process rhel8-server-tiny NAME=rheltinyvm SRC_PVC_NAME=rhel SRC_PVC_NAMESPACE=kubevirt | oc apply -f -

Please note that after the generation step VM and template objects have no relationship with each other besides the aforementioned label. Changes in templates do not automatically affect VMs or vice versa.

common template customization

The templates provided by the kubevirt project provide a set of conventions and annotations that augment the basic feature of the openshift templates. You can customize your kubevirt-provided templates editing these annotations, or you can add them to your existing templates to make them consumable by the kubevirt services.

Here’s a description of the kubevirt annotations. Unless otherwise specified, the following keys are meant to be top-level entries of the template metadata, like

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: windows-10
  5.   annotations:
  6.     openshift.io/display-name: "Generic demo template"

All the following annotations are prefixed with defaults.template.kubevirt.io, which is omitted below for brevity. So the actual annotations you should use will look like

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: windows-10
  5.   annotations:
  6.     defaults.template.kubevirt.io/disk: default-disk
  7.     defaults.template.kubevirt.io/volume: default-volume
  8.     defaults.template.kubevirt.io/nic: default-nic
  9.     defaults.template.kubevirt.io/network: default-network

Unless otherwise specified, all annotations are meant to be safe defaults, both for performance and compatibility, and hints for the CNV-aware UI and tooling.

disk

See the section references below.

Example:

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: Linux
  5.   annotations:
  6.     defaults.template.kubevirt.io/disk: rhel-disk

nic

See the section references below.

Example:

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: Windows
  5.   annotations:
  6.     defaults.template.kubevirt.io/nic: my-nic

volume

See the section references below.

Example:

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: Linux
  5.   annotations:
  6.     defaults.template.kubevirt.io/volume: custom-volume

network

See the section references below.

Example:

  1. apiVersion: v1
  2. kind: Template
  3. metadata:
  4.   name: Linux
  5.   annotations:
  6.     defaults.template.kubevirt.io/network: fast-net

references

The default values for network, nic, volume, disk are meant to be the name of a section later in the document that the UI will find and consume to find the default values for the corresponding types. For example, considering the annotation defaults.template.kubevirt.io/disk: my-disk: we assume that later in the document it exists an element called my-disk that the UI can use to find the data it needs. The names actually don’t matter as long as they are legal for kubernetes and consistent with the content of the document.

complete example

demo-template.yaml

  1. apiversion: v1
  2. items:
  3. - apiversion: kubevirt.io/v1
  4.   kind: virtualmachine
  5.   metadata:
  6.     labels:
  7.       vm.kubevirt.io/template: rhel7-generic-tiny
  8.     name: rheltinyvm
  9.     osinfoname: rhel7.0
  10.     defaults.template.kubevirt.io/disk: rhel-default-disk
  11.     defaults.template.kubevirt.io/nic: rhel-default-net
  12.   spec:
  13.     runStrategy: Halted
  14.     template:
  15.       spec:
  16.         domain:
  17.           cpu:
  18.             sockets: 1
  19.             cores: 1
  20.             threads: 1
  21.           devices:
  22.             rng: {}
  23.           resources:
  24.             requests:
  25.               memory: 1g
  26.         terminationgraceperiodseconds: 0
  27.         volumes:
  28.         - containerDisk:
  29.           image: registry:5000/kubevirt/cirros-container-disk-demo:devel
  30.           name: rhel-default-disk
  31.         networks:
  32.         - genie:
  33.           networkName: flannel
  34.           name: rhel-default-net
  35. kind: list
  36. metadata: {}

once processed becomes: demo-vm.yaml

  1. apiVersion: kubevirt.io/v1
  2. kind: VirtualMachine
  3. metadata:
  4.   labels:
  5.     vm.kubevirt.io/template: rhel7-generic-tiny
  6.   name: rheltinyvm
  7.   osinfoname: rhel7.0
  8. spec:
  9.   runStrategy: Halted
  10.   template:
  11.     spec:
  12.       domain:
  13.         cpu:
  14.           sockets: 1
  15.           cores: 1
  16.           threads: 1
  17.         resources:
  18.           requests:
  19.             memory: 1g
  20.         devices:
  21.           rng: {}
  22.           disks:
  23.           - disk:
  24.             name: rhel-default-disk
  25.         interfaces:
  26.         - bridge: {}
  27.           name: rhel-default-nic
  28.       terminationgraceperiodseconds: 0
  29.       volumes:
  30.       - containerDisk:
  31.           image: registry:5000/kubevirt/cirros-container-disk-demo:devel
  32.         name: containerdisk
  33.       networks:
  34.       - genie:
  35.           networkName: flannel
  36.         name: rhel-default-nic

Virtual machine creation

Overview

The KubeVirt projects provides a set of templates to create VMs to handle common usage scenarios. These templates provide a combination of some key factors that could be further customized and processed to have a Virtual Machine object.

The key factors which define a template are - Workload Most Virtual Machine should be server or desktop to have maximum flexibility; the highperformance workload trades some of this flexibility to provide better performances. - Guest Operating System (OS) This allow to ensure that the emulated hardware is compatible with the guest OS. Furthermore, it allows to maximize the stability of the VM, and allows performance optimizations. - Size (flavor) Defines the amount of resources (CPU, memory) to allocate to the VM.

Openshift Console

VMs can be created through OpenShift Cluster Console UI. This UI supports creation VM using templates and templates features - flavors and workload profiles. To create VM from template, choose WorkLoads in the left panel >> choose Virtualization >> press to the “Create Virtual Machine” blue button >> choose “Create from wizard”. Next, you have to see “Create Virtual Machine” window

Common-templates

There is the common-templates subproject. It provides official prepared and useful templates. You can also create templates by hand. You can find an example below, in the “Example template” section.

Example template

In order to create a virtual machine via OpenShift CLI, you need to provide a template defining the corresponding object and its metadata.

NOTE Only VirtualMachine object is currently supported.

Here is an example template that defines an instance of the VirtualMachine object:

  1. apiVersion: template.openshift.io/v1
  2. kind: Template
  3. metadata:
  4.   name: fedora-desktop-large
  5.   annotations:
  6.     openshift.io/display-name: "Fedora 32+ VM"
  7.     description: >-
  8.       Template for Fedora 32 VM or newer.
  9.       A PVC with the Fedora disk image must be available.
  10.       Recommended disk image:
  11.       https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.qcow2
  12.     tags: "hidden,kubevirt,virtualmachine,fedora"
  13.     iconClass: "icon-fedora"
  14.     openshift.io/provider-display-name: "KubeVirt"
  15.     openshift.io/documentation-url: "https://github.com/kubevirt/common-templates"
  16.     openshift.io/support-url: "https://github.com/kubevirt/common-templates/issues"
  17.     template.openshift.io/bindable: "false"
  18.     template.kubevirt.io/version: v1alpha1
  19.     defaults.template.kubevirt.io/disk: rootdisk
  20.     template.kubevirt.io/editable: |
  21.       /objects[0].spec.template.spec.domain.cpu.sockets
  22.       /objects[0].spec.template.spec.domain.cpu.cores
  23.       /objects[0].spec.template.spec.domain.cpu.threads
  24.       /objects[0].spec.template.spec.domain.resources.requests.memory
  25.       /objects[0].spec.template.spec.domain.devices.disks
  26.       /objects[0].spec.template.spec.volumes
  27.       /objects[0].spec.template.spec.networks
  28.     name.os.template.kubevirt.io/fedora32: Fedora 32 or higher
  29.     name.os.template.kubevirt.io/fedora33: Fedora 32 or higher
  30.     name.os.template.kubevirt.io/silverblue32: Fedora 32 or higher
  31.     name.os.template.kubevirt.io/silverblue33: Fedora 32 or higher
  32.   labels:
  33.     os.template.kubevirt.io/fedora32: "true"
  34.     os.template.kubevirt.io/fedora33: "true"
  35.     os.template.kubevirt.io/silverblue32: "true"
  36.     os.template.kubevirt.io/silverblue33: "true"
  37.     workload.template.kubevirt.io/desktop: "true"
  38.     flavor.template.kubevirt.io/large: "true"
  39.     template.kubevirt.io/type: "base"
  40.     template.kubevirt.io/version: "0.11.3"
  41. objects:
  42. - apiVersion: kubevirt.io/v1
  43.   kind: VirtualMachine
  44.   metadata:
  45.     name: ${NAME}
  46.     labels:
  47.       vm.kubevirt.io/template: fedora-desktop-large
  48.       vm.kubevirt.io/template.version: "0.11.3"
  49.       vm.kubevirt.io/template.revision: "45"
  50.       app: ${NAME}
  51.     annotations:
  52.       vm.kubevirt.io/os: "fedora"
  53.       vm.kubevirt.io/workload: "desktop"
  54.       vm.kubevirt.io/flavor: "large"
  55.       vm.kubevirt.io/validations: |
  56.         [
  57.           {
  58.             "name": "minimal-required-memory",
  59.             "path": "jsonpath::.spec.domain.resources.requests.memory",
  60.             "rule": "integer",
  61.             "message": "This VM requires more memory.",
  62.             "min": 1073741824
  63.           }
  64.         ]
  65.   spec:
  66.     dataVolumeTemplates:
  67.     - apiVersion: cdi.kubevirt.io/v1beta1
  68.       kind: DataVolume
  69.       metadata:
  70.         name: ${NAME}
  71.       spec:
  72.         storage:
  73.           accessModes:
  74.             - ReadWriteMany
  75.         source:
  76.           pvc:
  77.             name: ${SRC_PVC_NAME}
  78.             namespace: ${SRC_PVC_NAMESPACE}
  79.     runStrategy: Halted
  80.     template:
  81.       metadata:
  82.         labels:
  83.           kubevirt.io/domain: ${NAME}
  84.           kubevirt.io/size: large
  85.       spec:
  86.         domain:
  87.           cpu:
  88.             sockets: 2
  89.             cores: 1
  90.             threads: 1
  91.           resources:
  92.             requests:
  93.               memory: 8Gi
  94.           devices:
  95.             rng: {}
  96.             networkInterfaceMultiqueue: true
  97.             inputs:
  98.               - type: tablet
  99.                 bus: virtio
  100.                 name: tablet
  101.             disks:
  102.             - disk:
  103.                 bus: virtio
  104.               name: ${NAME}
  105.             - disk:
  106.                 bus: virtio
  107.               name: cloudinitdisk
  108.             interfaces:
  109.             - masquerade: {}
  110.               name: default
  111.         terminationGracePeriodSeconds: 180
  112.         networks:
  113.         - name: default
  114.           pod: {}
  115.         volumes:
  116.         - dataVolume:
  117.             name: ${NAME}
  118.           name: ${NAME}
  119.         - cloudInitNoCloud:
  120.             userData: |-
  121.               #cloud-config
  122.               user: fedora
  123.               password: ${CLOUD_USER_PASSWORD}
  124.               chpasswd: { expire: False }
  125.           name: cloudinitdisk
  126. parameters:
  127. - description: VM name
  128.   from: 'fedora-[a-z0-9]{16}'
  129.   generate: expression
  130.   name: NAME
  131. - name: SRC_PVC_NAME
  132.   description: Name of the PVC to clone
  133.   value: 'fedora'
  134. - name: SRC_PVC_NAMESPACE
  135.   description: Namespace of the source PVC
  136.   value: kubevirt-os-images
  137. - description: Randomized password for the cloud-init user fedora
  138.   from: '[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}'
  139.   generate: expression
  140.   name: CLOUD_USER_PASSWORD

Note that the template above defines free parameters (NAME, SRC_PVC_NAME, SRC_PVC_NAMESPACE, CLOUD_USER_PASSWORD) and the NAME parameter does not have specified default value.

An OpenShift template has to be converted into the JSON file via oc process command, that also allows you to set the template parameters.

A complete example can be found in the KubeVirt repository.

!> You need to be logged in by oc login command.

  1. $ oc process -f cluster/vmi-template-fedora.yaml\
  2.     -p NAME=testvmi \
  3.     -p SRC_PVC_NAME=fedora \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5. {
  6.     "kind": "List",
  7.     "apiVersion": "v1",
  8.     "metadata": {},
  9.     "items": [
  10.         {

The JSON file is usually applied directly by piping the processed output to oc create command.

  1. $ oc process -f cluster/examples/vm-template-fedora.yaml \
  2.     -p NAME=testvm \
  3.     -p SRC_PVC_NAME=fedora \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5.     | oc create -f -
  6. virtualmachine.kubevirt.io/testvm created

The command above results in creating a Kubernetes object according to the specification given by the template \(in this example it is an instance of the VirtualMachine object\).

It’s possible to get list of available parameters using the following command:

  1. $ oc process -f dist/templates/fedora-desktop-large.yaml --parameters
  2. NAME                  DESCRIPTION                                          GENERATOR           VALUE
  3. NAME                  VM name                                              expression          fedora-[a-z0-9]{16}
  4. SRC_PVC_NAME          Name of the PVC to clone                                                 fedora
  5. SRC_PVC_NAMESPACE     Namespace of the source PVC                                              kubevirt-os-images
  6. CLOUD_USER_PASSWORD   Randomized password for the cloud-init user fedora   expression          [a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}

Starting virtual machine from the created object

The created object is now a regular VirtualMachine object and from now it can be controlled by accessing Kubernetes API resources. The preferred way how to do this from within the OpenShift environment is to use oc patch command.

  1. $ oc patch virtualmachine testvm --type merge -p '{"spec":{"runStrategy":"Always"}}'
  2. virtualmachine.kubevirt.io/testvm patched

Do not forget about virtctl tool. Using it in the real cases instead of using kubernetes API can be more convenient. Example:

  1. $ virtctl start testvm
  2. VM testvm was scheduled to start

As soon as VM starts, Kubernetes creates new type of object - VirtualMachineInstance. It has similar name to VirtualMachine. Example (not full output, it’s too big):

  1. $ kubectl describe vm testvm
  2. name:         testvm
  3. Namespace:    myproject
  4. Labels:       kubevirt-vm=vm-testvm
  5.               kubevirt.io/os=fedora33
  6. Annotations:  <none>
  7. API Version:  kubevirt.io/v1
  8. Kind:         VirtualMachine

Cloud-init script and parameters

Kubevirt VM templates, just like kubevirt VM/VMI yaml configs, supports cloud-init scripts

Hack - use pre-downloaded image

Kubevirt VM templates, just like kubevirt VM/VMI yaml configs, can use pre-downloaded VM image, which can be a useful feature especially in the debug/development/testing cases. No special parameters required in the VM template or VM/VMI yaml config. The main idea is to create Kubernetes PersistentVolume and PersistentVolumeClaim corresponding to existing image in the file system. Example:

  1. ---
  2. kind: PersistentVolume
  3. apiVersion: v1
  4. metadata:
  5.   name: mypv
  6.   labels:
  7.     type: local
  8. spec:
  9.   storageClassName: manual
  10.   capacity:
  11.     storage: 10G
  12.   accessModes:
  13.     - ReadWriteOnce
  14.   hostPath:
  15.     path: "/mnt/sda1/images/testvm"
  16. ---
  17. kind: PersistentVolumeClaim
  18. apiVersion: v1
  19. metadata:
  20.   name: mypvc
  21. spec:
  22.   storageClassName: manual
  23.   accessModes:
  24.     - ReadWriteOnce
  25.   resources:
  26.     requests:
  27.       storage: 10G

Using DataVolumes

Kubevirt VM templates are using dataVolumeTemplates. Before using dataVolumes, CDI has to be installed in cluster. After that, source Datavolume can be created.

  1. ---
  2. apiVersion: cdi.kubevirt.io/v1beta1
  3. kind: DataVolume
  4. metadata:
  5.   name: fedora-datavolume-original
  6.   namespace: kubevirt
  7. spec:
  8.   source:
  9.     registry:
  10.       url: "image_url"
  11.   storage:
  12.     resources:
  13.       requests:
  14.         storage: 30Gi

After import is completed, VM can be created:

  1. $ oc process -f cluster/examples/vm-template-fedora.yaml \
  2.     -p NAME=testvmi \
  3.     -p SRC_PVC_NAME=fedora-datavolume-original \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5.     | oc create -f -
  6. virtualmachine.kubevirt.io/testvm created

Additional information

You can follow Virtual Machine Lifecycle Guide for further reference.