Using huge pages with virtual machines

You can use huge pages as backing memory for virtual machines in your cluster.

Prerequisites

• Nodes must have pre-allocated huge pages configured.

What huge pages do

Memory is managed in blocks known as pages. On most systems, a page is 4Ki. 1Mi of memory is equal to 256 pages; 1Gi of memory is 256,000 pages, and so on. CPUs have a built-in memory management unit that manages a list of these pages in hardware. The Translation Lookaside Buffer (TLB) is a small hardware cache of virtual-to-physical page mappings. If the virtual address passed in a hardware instruction can be found in the TLB, the mapping can be determined quickly. If not, a TLB miss occurs, and the system falls back to slower, software-based address translation, resulting in performance issues. Since the size of the TLB is fixed, the only way to reduce the chance of a TLB miss is to increase the page size.

A huge page is a memory page that is larger than 4Ki. On x86_64 architectures, there are two common huge page sizes: 2Mi and 1Gi. Sizes vary on other architectures. To use huge pages, code must be written so that applications are aware of them. Transparent Huge Pages (THP) attempt to automate the management of huge pages without application knowledge, but they have limitations. In particular, they are limited to 2Mi page sizes. THP can lead to performance degradation on nodes with high memory utilization or fragmentation due to defragmenting efforts of THP, which can lock memory pages. For this reason, some applications may be designed to (or recommend) usage of pre-allocated huge pages instead of THP.

In OKD Virtualization, virtual machines can be configured to consume pre-allocated huge pages.

Configuring huge pages for virtual machines

You can configure virtual machines to use pre-allocated huge pages by including the memory.hugepages.pageSize and resources.requests.memory parameters in your virtual machine configuration.

The memory request must be divisible by the page size. For example, you cannot request 500Mi memory with a page size of 1Gi.

If you edit a running virtual machine, the virtual machine must be rebooted for the changes to take effect.

Prerequisites

• Nodes must have pre-allocated huge pages configured.

Procedure

1. In your virtual machine configuration, add the resources.requests.memory and memory.hugepages.pageSize parameters to the spec.domain. The following configuration snippet is for a virtual machine that requests a total of 4Gi memory with a page size of 1Gi:

   kind: VirtualMachine
   ...
   spec:
     domain:
       resources:
         requests:
           memory: "4Gi" (1)
       memory:
         hugepages:
           pageSize: "1Gi" (2)
   ...

   1	The total amount of memory requested for the virtual machine. This value must be divisible by the page size.
   2	The size of each huge page. Valid values for x86_64 architecture are 1Gi and 2Mi. The page size must be smaller than the requested memory.

2. Apply the virtual machine configuration:

   $ oc apply -f <virtual_machine>.yaml