Templates

Note

By deploying KubeVirt on top of OpenShift the user can benefit from the OpenShift Template functionality.

Virtual machine templates

What is a virtual machine template?

The KubeVirt projects provides a set of templates to create VMs to handle common usage scenarios. These templates provide a combination of some key factors that could be further customized and processed to have a Virtual Machine object. The key factors which define a template are

  • Workload Most Virtual Machine should be server or desktop to have maximum flexibility; the highperformance workload trades some of this flexibility to provide better performances.

  • Guest Operating System (OS) This allow to ensure that the emulated hardware is compatible with the guest OS. Furthermore, it allows to maximize the stability of the VM, and allows performance optimizations.

  • Size (flavor) Defines the amount of resources (CPU, memory) to allocate to the VM.

More documentation is available in the common templates subproject

Accessing the virtual machine templates

If you installed KubeVirt using a supported method you should find the common templates preinstalled in the cluster. Should you want to upgrade the templates, or install them from scratch, you can use one of the supported releases

To install the templates:

  1.     $ export VERSION=$(curl -s https://api.github.com/repos/kubevirt/common-templates/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
  2.     $ oc create -f https://github.com/kubevirt/common-templates/releases/download/$VERSION/common-templates-$VERSION.yaml

Editable fields

You can edit the fields of the templates which define the amount of resources which the VMs will receive.

Each template can list a different set of fields that are to be considered editable. The fields are used as hints for the user interface, and also for other components in the cluster.

The editable fields are taken from annotations in the template. Here is a snippet presenting a couple of most commonly found editable fields:

  1.     metadata:
  2.       annotations:
  3.         template.kubevirt.io/editable: |
  4.           /objects[0].spec.template.spec.domain.cpu.sockets
  5.           /objects[0].spec.template.spec.domain.cpu.cores
  6.           /objects[0].spec.template.spec.domain.cpu.threads
  7.           /objects[0].spec.template.spec.domain.resources.requests.memory

Each entry in the editable field list must be a jsonpath. The jsonpath root is the objects: element of the template. The actually editable field is the last entry (the “leaf”) of the path. For example, the following minimal snippet highlights the fields which you can edit:

  1.     objects:
  2.       spec:
  3.         template:
  4.           spec:
  5.             domain:
  6.               cpu:
  7.                 sockets:
  8.                   VALUE # this is editable
  9.                 cores:
  10.                   VALUE # this is editable
  11.                 threads:
  12.                   VALUE # this is editable
  13.               resources:
  14.                 requests:
  15.                   memory:
  16.                     VALUE # this is editable

Relationship between templates and VMs

Once processed the templates produce VM objects to be used in the cluster. The VMs produced from templates will have a vm.kubevirt.io/template label, whose value will be the name of the parent template, for example fedora-desktop-medium:

  1.       metadata:
  2.         labels:
  3.           vm.kubevirt.io/template: fedora-desktop-medium

In addition, these VMs can include an optional label vm.kubevirt.io/template-namespace, whose value will be the namespace of the parent template, for example:

  1.       metadata:
  2.         labels:
  3.           vm.kubevirt.io/template-namespace: openshift

If this label is not defined, the template is expected to belong to the same namespace as the VM.

This make it possible to query for all the VMs built from any template.

Example:

  1.     oc process -o yaml -f dist/templates/rhel8-server-tiny.yaml NAME=rheltinyvm SRC_PVC_NAME=rhel SRC_PVC_NAMESPACE=kubevirt

And the output:

  1.     apiVersion: v1
  2.     items:
  3.     - apiVersion: kubevirt.io/v1
  4.       kind: VirtualMachine
  5.       metadata:
  6.         annotations:
  7.           vm.kubevirt.io/flavor: tiny
  8.           vm.kubevirt.io/os: rhel8
  9.           vm.kubevirt.io/validations: |
  10.             [
  11.               {
  12.                 "name": "minimal-required-memory",
  13.                 "path": "jsonpath::.spec.domain.resources.requests.memory",
  14.                 "rule": "integer",
  15.                 "message": "This VM requires more memory.",
  16.                 "min": 1610612736
  17.               }
  18.             ]
  19.           vm.kubevirt.io/workload: server
  20.         labels:
  21.           app: rheltinyvm
  22.           vm.kubevirt.io/template: rhel8-server-tiny
  23.           vm.kubevirt.io/template.revision: "45"
  24.           vm.kubevirt.io/template.version: 0.11.3
  25.         name: rheltinyvm
  26.       spec:
  27.         dataVolumeTemplates:
  28.         - apiVersion: cdi.kubevirt.io/v1beta1
  29.           kind: DataVolume
  30.           metadata:
  31.             name: rheltinyvm
  32.           spec:
  33.             pvc:
  34.               accessModes:
  35.               - ReadWriteMany
  36.               resources:
  37.                 requests:
  38.                   storage: 30Gi
  39.             source:
  40.               pvc:
  41.                 name: rhel
  42.                 namespace: kubevirt
  43.         running: false
  44.         template:
  45.           metadata:
  46.             labels:
  47.               kubevirt.io/domain: rheltinyvm
  48.               kubevirt.io/size: tiny
  49.           spec:
  50.             domain:
  51.               cpu:
  52.                 cores: 1
  53.                 sockets: 1
  54.                 threads: 1
  55.               devices:
  56.                 disks:
  57.                 - disk:
  58.                     bus: virtio
  59.                   name: rheltinyvm
  60.                 - disk:
  61.                     bus: virtio
  62.                   name: cloudinitdisk
  63.                 interfaces:
  64.                 - masquerade: {}
  65.                   name: default
  66.                 networkInterfaceMultiqueue: true
  67.                 rng: {}
  68.               resources:
  69.                 requests:
  70.                   memory: 1.5Gi
  71.             networks:
  72.             - name: default
  73.               pod: {}
  74.             terminationGracePeriodSeconds: 180
  75.             volumes:
  76.             - dataVolume:
  77.                 name: rheltinyvm
  78.               name: rheltinyvm
  79.             - cloudInitNoCloud:
  80.                 userData: |-
  81.                   #cloud-config
  82.                   user: cloud-user
  83.                   password: lymp-fda4-m1cv
  84.                   chpasswd: { expire: False }
  85.               name: cloudinitdisk
  86.     kind: List
  87.     metadata: {}

You can add the VM from the template to the cluster in one go

  1.     oc process rhel8-server-tiny NAME=rheltinyvm SRC_PVC_NAME=rhel SRC_PVC_NAMESPACE=kubevirt | oc apply -f -

Please note that after the generation step VM and template objects have no relationship with each other besides the aforementioned label. Changes in templates do not automatically affect VMs or vice versa.

common template customization

The templates provided by the kubevirt project provide a set of conventions and annotations that augment the basic feature of the openshift templates. You can customize your kubevirt-provided templates editing these annotations, or you can add them to your existing templates to make them consumable by the kubevirt services.

Here’s a description of the kubevirt annotations. Unless otherwise specified, the following keys are meant to be top-level entries of the template metadata, like

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: windows-10
  5.       annotations:
  6.         openshift.io/display-name: "Generic demo template"

All the following annotations are prefixed with defaults.template.kubevirt.io, which is omitted below for brevity. So the actual annotations you should use will look like

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: windows-10
  5.       annotations:
  6.         defaults.template.kubevirt.io/disk: default-disk
  7.         defaults.template.kubevirt.io/volume: default-volume
  8.         defaults.template.kubevirt.io/nic: default-nic
  9.         defaults.template.kubevirt.io/network: default-network

Unless otherwise specified, all annotations are meant to be safe defaults, both for performance and compatibility, and hints for the CNV-aware UI and tooling.

disk

See the section references below.

Example:

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: Linux
  5.       annotations:
  6.         defaults.template.kubevirt.io/disk: rhel-disk

nic

See the section references below.

Example:

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: Windows
  5.       annotations:
  6.         defaults.template.kubevirt.io/nic: my-nic

volume

See the section references below.

Example:

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: Linux
  5.       annotations:
  6.         defaults.template.kubevirt.io/volume: custom-volume

network

See the section references below.

Example:

  1.     apiVersion: v1
  2.     kind: Template
  3.     metadata:
  4.       name: Linux
  5.       annotations:
  6.         defaults.template.kubevirt.io/network: fast-net

references

The default values for network, nic, volume, disk are meant to be the name of a section later in the document that the UI will find and consume to find the default values for the corresponding types. For example, considering the annotation defaults.template.kubevirt.io/disk: my-disk: we assume that later in the document it exists an element called my-disk that the UI can use to find the data it needs. The names actually don’t matter as long as they are legal for kubernetes and consistent with the content of the document.

complete example

demo-template.yaml

  1. apiversion: v1
  2. items:
  3. - apiversion: kubevirt.io/v1
  4.   kind: virtualmachine
  5.   metadata:
  6.     labels:
  7.       vm.kubevirt.io/template: rhel7-generic-tiny
  8.     name: rheltinyvm
  9.     osinfoname: rhel7.0
  10.     defaults.template.kubevirt.io/disk: rhel-default-disk
  11.     defaults.template.kubevirt.io/nic: rhel-default-net
  12.   spec:
  13.     running: false
  14.     template:
  15.       spec:
  16.         domain:
  17.           cpu:
  18.             sockets: 1
  19.             cores: 1
  20.             threads: 1
  21.           devices:
  22.             rng: {}
  23.           resources:
  24.             requests:
  25.               memory: 1g
  26.         terminationgraceperiodseconds: 0
  27.         volumes:
  28.         - containerDisk:
  29.           image: registry:5000/kubevirt/cirros-container-disk-demo:devel
  30.           name: rhel-default-disk
  31.         networks:
  32.         - genie:
  33.           networkName: flannel
  34.           name: rhel-default-net
  35. kind: list
  36. metadata: {}

once processed becomes: demo-vm.yaml

  1. apiVersion: kubevirt.io/v1
  2. kind: VirtualMachine
  3. metadata:
  4.   labels:
  5.     vm.kubevirt.io/template: rhel7-generic-tiny
  6.   name: rheltinyvm
  7.   osinfoname: rhel7.0
  8. spec:
  9.   running: false
  10.   template:
  11.     spec:
  12.       domain:
  13.         cpu:
  14.           sockets: 1
  15.           cores: 1
  16.           threads: 1
  17.         resources:
  18.           requests:
  19.             memory: 1g
  20.         devices:
  21.           rng: {}
  22.           disks:
  23.           - disk:
  24.             name: rhel-default-disk
  25.         interfaces:
  26.         - bridge: {}
  27.           name: rhel-default-nic
  28.       terminationgraceperiodseconds: 0
  29.       volumes:
  30.       - containerDisk:
  31.           image: registry:5000/kubevirt/cirros-container-disk-demo:devel
  32.         name: containerdisk
  33.       networks:
  34.       - genie:
  35.           networkName: flannel
  36.         name: rhel-default-nic

Virtual machine creation

Overview

The KubeVirt projects provides a set of templates to create VMs to handle common usage scenarios. These templates provide a combination of some key factors that could be further customized and processed to have a Virtual Machine object.

The key factors which define a template are - Workload Most Virtual Machine should be server or desktop to have maximum flexibility; the highperformance workload trades some of this flexibility to provide better performances. - Guest Operating System (OS) This allow to ensure that the emulated hardware is compatible with the guest OS. Furthermore, it allows to maximize the stability of the VM, and allows performance optimizations. - Size (flavor) Defines the amount of resources (CPU, memory) to allocate to the VM.

Openshift Console

VMs can be created through OpenShift Cluster Console UI. This UI supports creation VM using templates and templates features - flavors and workload profiles. To create VM from template, choose WorkLoads in the left panel >> choose Virtualization >> press to the “Create Virtual Machine” blue button >> choose “Create from wizard”. Next, you have to see “Create Virtual Machine” window

Common-templates

There is the common-templates subproject. It provides official prepared and useful templates. You can also create templates by hand. You can find an example below, in the “Example template” section.

Example template

In order to create a virtual machine via OpenShift CLI, you need to provide a template defining the corresponding object and its metadata.

NOTE Only VirtualMachine object is currently supported.

Here is an example template that defines an instance of the VirtualMachine object:

  1. apiVersion: template.openshift.io/v1
  2. kind: Template
  3. metadata:
  4.   name: fedora-desktop-large
  5.   annotations:
  6.     openshift.io/display-name: "Fedora 32+ VM"
  7.     description: >-
  8.       Template for Fedora 32 VM or newer.
  9.       A PVC with the Fedora disk image must be available.
  10.       Recommended disk image:
  11.       https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.qcow2
  12.     tags: "hidden,kubevirt,virtualmachine,fedora"
  13.     iconClass: "icon-fedora"
  14.     openshift.io/provider-display-name: "KubeVirt"
  15.     openshift.io/documentation-url: "https://github.com/kubevirt/common-templates"
  16.     openshift.io/support-url: "https://github.com/kubevirt/common-templates/issues"
  17.     template.openshift.io/bindable: "false"
  18.     template.kubevirt.io/version: v1alpha1
  19.     defaults.template.kubevirt.io/disk: rootdisk
  20.     template.kubevirt.io/editable: |
  21.       /objects[0].spec.template.spec.domain.cpu.sockets
  22.       /objects[0].spec.template.spec.domain.cpu.cores
  23.       /objects[0].spec.template.spec.domain.cpu.threads
  24.       /objects[0].spec.template.spec.domain.resources.requests.memory
  25.       /objects[0].spec.template.spec.domain.devices.disks
  26.       /objects[0].spec.template.spec.volumes
  27.       /objects[0].spec.template.spec.networks
  28.     name.os.template.kubevirt.io/fedora32: Fedora 32 or higher
  29.     name.os.template.kubevirt.io/fedora33: Fedora 32 or higher
  30.     name.os.template.kubevirt.io/silverblue32: Fedora 32 or higher
  31.     name.os.template.kubevirt.io/silverblue33: Fedora 32 or higher
  32.   labels:
  33.     os.template.kubevirt.io/fedora32: "true"
  34.     os.template.kubevirt.io/fedora33: "true"
  35.     os.template.kubevirt.io/silverblue32: "true"
  36.     os.template.kubevirt.io/silverblue33: "true"
  37.     workload.template.kubevirt.io/desktop: "true"
  38.     flavor.template.kubevirt.io/large: "true"
  39.     template.kubevirt.io/type: "base"
  40.     template.kubevirt.io/version: "0.11.3"
  41. objects:
  42. - apiVersion: kubevirt.io/v1
  43.   kind: VirtualMachine
  44.   metadata:
  45.     name: ${NAME}
  46.     labels:
  47.       vm.kubevirt.io/template: fedora-desktop-large
  48.       vm.kubevirt.io/template.version: "0.11.3"
  49.       vm.kubevirt.io/template.revision: "45"
  50.       app: ${NAME}
  51.     annotations:
  52.       vm.kubevirt.io/os: "fedora"
  53.       vm.kubevirt.io/workload: "desktop"
  54.       vm.kubevirt.io/flavor: "large"
  55.       vm.kubevirt.io/validations: |
  56.         [
  57.           {
  58.             "name": "minimal-required-memory",
  59.             "path": "jsonpath::.spec.domain.resources.requests.memory",
  60.             "rule": "integer",
  61.             "message": "This VM requires more memory.",
  62.             "min": 1073741824
  63.           }
  64.         ]
  65.   spec:
  66.     dataVolumeTemplates:
  67.     - apiVersion: cdi.kubevirt.io/v1beta1
  68.       kind: DataVolume
  69.       metadata:
  70.         name: ${NAME}
  71.       spec:
  72.         pvc:
  73.           accessModes:
  74.             - ReadWriteMany
  75.           resources:
  76.             requests:
  77.               storage: 30Gi
  78.         source:
  79.           pvc:
  80.             name: ${SRC_PVC_NAME}
  81.             namespace: ${SRC_PVC_NAMESPACE}
  82.     running: false
  83.     template:
  84.       metadata:
  85.         labels:
  86.           kubevirt.io/domain: ${NAME}
  87.           kubevirt.io/size: large
  88.       spec:
  89.         domain:
  90.           cpu:
  91.             sockets: 2
  92.             cores: 1
  93.             threads: 1
  94.           resources:
  95.             requests:
  96.               memory: 8Gi
  97.           devices:
  98.             rng: {}
  99.             networkInterfaceMultiqueue: true
  100.             inputs:
  101.               - type: tablet
  102.                 bus: virtio
  103.                 name: tablet
  104.             disks:
  105.             - disk:
  106.                 bus: virtio
  107.               name: ${NAME}
  108.             - disk:
  109.                 bus: virtio
  110.               name: cloudinitdisk
  111.             interfaces:
  112.             - masquerade: {}
  113.               name: default
  114.         terminationGracePeriodSeconds: 180
  115.         networks:
  116.         - name: default
  117.           pod: {}
  118.         volumes:
  119.         - dataVolume:
  120.             name: ${NAME}
  121.           name: ${NAME}
  122.         - cloudInitNoCloud:
  123.             userData: |-
  124.               #cloud-config
  125.               user: fedora
  126.               password: ${CLOUD_USER_PASSWORD}
  127.               chpasswd: { expire: False }
  128.           name: cloudinitdisk
  129. parameters:
  130. - description: VM name
  131.   from: 'fedora-[a-z0-9]{16}'
  132.   generate: expression
  133.   name: NAME
  134. - name: SRC_PVC_NAME
  135.   description: Name of the PVC to clone
  136.   value: 'fedora'
  137. - name: SRC_PVC_NAMESPACE
  138.   description: Namespace of the source PVC
  139.   value: kubevirt-os-images
  140. - description: Randomized password for the cloud-init user fedora
  141.   from: '[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}'
  142.   generate: expression
  143.   name: CLOUD_USER_PASSWORD

Note that the template above defines free parameters (NAME, SRC_PVC_NAME, SRC_PVC_NAMESPACE, CLOUD_USER_PASSWORD) and the NAME parameter does not have specified default value.

An OpenShift template has to be converted into the JSON file via oc process command, that also allows you to set the template parameters.

A complete example can be found in the KubeVirt repository.

!> You need to be logged in by oc login command.

  1. $ oc process -f cluster/vmi-template-fedora.yaml\
  2.     -p NAME=testvmi \
  3.     -p SRC_PVC_NAME=fedora \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5. {
  6.     "kind": "List",
  7.     "apiVersion": "v1",
  8.     "metadata": {},
  9.     "items": [
  10.         {

The JSON file is usually applied directly by piping the processed output to oc create command.

  1. $ oc process -f cluster/examples/vm-template-fedora.yaml \
  2.     -p NAME=testvm \
  3.     -p SRC_PVC_NAME=fedora \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5.     | oc create -f -
  6. virtualmachine.kubevirt.io/testvm created

The command above results in creating a Kubernetes object according to the specification given by the template \(in this example it is an instance of the VirtualMachine object\).

It’s possible to get list of available parameters using the following command:

  1. $ oc process -f dist/templates/fedora-desktop-large.yaml --parameters
  2. NAME                  DESCRIPTION                                          GENERATOR           VALUE
  3. NAME                  VM name                                              expression          fedora-[a-z0-9]{16}
  4. SRC_PVC_NAME          Name of the PVC to clone                                                 fedora
  5. SRC_PVC_NAMESPACE     Namespace of the source PVC                                              kubevirt-os-images
  6. CLOUD_USER_PASSWORD   Randomized password for the cloud-init user fedora   expression          [a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}

Starting virtual machine from the created object

The created object is now a regular VirtualMachine object and from now it can be controlled by accessing Kubernetes API resources. The preferred way how to do this from within the OpenShift environment is to use oc patch command.

  1. $ oc patch virtualmachine testvm --type merge -p '{"spec":{"running":true}}'
  2. virtualmachine.kubevirt.io/testvm patched

Do not forget about virtctl tool. Using it in the real cases instead of using kubernetes API can be more convenient. Example:

  1. $ virtctl start testvm
  2. VM testvm was scheduled to start

As soon as VM starts, Kubernetes creates new type of object - VirtualMachineInstance. It has similar name to VirtualMachine. Example (not full output, it’s too big):

  1. $ kubectl describe vm testvm
  2. name:         testvm
  3. Namespace:    myproject
  4. Labels:       kubevirt-vm=vm-testvm
  5.               kubevirt.io/os=fedora33
  6. Annotations:  <none>
  7. API Version:  kubevirt.io/v1
  8. Kind:         VirtualMachine

Cloud-init script and parameters

Kubevirt VM templates, just like kubevirt VM/VMI yaml configs, supports cloud-init scripts

Hack - use pre-downloaded image

Kubevirt VM templates, just like kubevirt VM/VMI yaml configs, can use pre-downloaded VM image, which can be a useful feature especially in the debug/development/testing cases. No special parameters required in the VM template or VM/VMI yaml config. The main idea is to create Kubernetes PersistentVolume and PersistentVolumeClaim corresponding to existing image in the file system. Example:

  1. ---
  2. kind: PersistentVolume
  3. apiVersion: v1
  4. metadata:
  5.   name: mypv
  6.   labels:
  7.     type: local
  8. spec:
  9.   storageClassName: manual
  10.   capacity:
  11.     storage: 10G
  12.   accessModes:
  13.     - ReadWriteOnce
  14.   hostPath:
  15.     path: "/mnt/sda1/images/testvm"
  16. ---
  17. kind: PersistentVolumeClaim
  18. apiVersion: v1
  19. metadata:
  20.   name: mypvc
  21. spec:
  22.   storageClassName: manual
  23.   accessModes:
  24.     - ReadWriteOnce
  25.   resources:
  26.     requests:
  27.       storage: 10G

Using DataVolumes

Kubevirt VM templates are using dataVolumeTemplates. Before using dataVolumes, CDI has to be installed in cluster. After that, source Datavolume can be created.

  1. ---
  2. apiVersion: cdi.kubevirt.io/v1beta1
  3. kind: DataVolume
  4. metadata:
  5.   name: fedora-datavolume-original
  6.   namespace: kubevirt
  7. spec:
  8.   source:
  9.     registry:
  10.       url: "image_url"
  11.   pvc:
  12.     accessModes:
  13.       - ReadWriteOnce
  14.     resources:
  15.       requests:
  16.         storage: 30Gi

After import is completed, VM can be created:

  1. $ oc process -f cluster/examples/vm-template-fedora.yaml \
  2.     -p NAME=testvmi \
  3.     -p SRC_PVC_NAME=fedora-datavolume-original \
  4.     -p SRC_PVC_NAMESPACE=kubevirt \
  5.     | oc create -f -
  6. virtualmachine.kubevirt.io/testvm created

Additional information

You can follow Virtual Machine Lifecycle Guide for further reference.