PXE Boot Installation

Starting from version 0.2.0, Harvester can be installed automatically. This document provides an example to do an automatic installation with PXE boot.

We recommend using iPXE to perform the network boot. It has more features than the traditional PXE Boot program and is likely available in modern NIC cards. If the iPXE firmware is not available for your NIC card, the iPXE firmware images can be loaded from the TFTP server first.

To see sample iPXE scripts, please visit Harvester iPXE Examples.

Prerequisite

Important

Nodes need to have at least 8G of RAM because the installer loads the full ISO file into tmpfs.

Preparing HTTP Servers

An HTTP server is required to serve boot files. Let’s assume the NGINX HTTP server’s IP is 10.100.0.10, and it serves the /usr/share/nginx/html/ directory with the path http://10.100.0.10/.

Preparing Boot Files

  • Download the required files from the Harvester releases page.

    • The ISO: harvester-<version>-amd64.iso
    • The kernel: harvester-<version>-vmlinuz-amd64
    • The initrd: harvester-<version>-initrd-amd64
    • The rootfs squashfs image: harvester-<version>-rootfs-amd64.squashfs

  • Serve the files.

    Copy or move the downloaded files to an appropriate location so they can be downloaded via the HTTP server. For example:

    1. sudo mkdir -p /usr/share/nginx/html/harvester/
    2. sudo cp /path/to/harvester-<version>-amd64.iso /usr/share/nginx/html/harvester/
    3. sudo cp /path/to/harvester-<version>-vmlinuz-amd64 /usr/share/nginx/html/harvester/
    4. sudo cp /path/to/harvester-<version>-initrd-amd64 /usr/share/nginx/html/harvester/
    5. sudo cp /path/to/harvester-<version>-rootfs-amd64.squashfs /usr/share/nginx/html/harvester/

Preparing iPXE Boot Scripts

When performing an automatic installation, there are two modes:

  • CREATE: we are installing a node to construct an initial Harvester cluster.
  • JOIN: we are installing a node to join an existing Harvester cluster.

CREATE Mode

Warning

Security Risks: The configuration file below contains credentials which should be kept secret. Please do not make the configuration file publicly accessible.

Create a Harvester configuration file called config-create.yaml for CREATE mode. Modify the values as needed:

  1. # cat /usr/share/nginx/html/harvester/config-create.yaml
  2. token: token
  3. os:
  4.   hostname: node1
  5.   ssh_authorized_keys:
  6.   - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbeUa9A7Kee+hcCleIXYxuaPksn2m4PZTd4T7wPcse8KbsQfttGRax6vxQXoPO6ehddqOb2nV7tkW2mEhR50OE7W7ngDHbzK2OneAyONYF44bmMsapNAGvnsBKe9rNrev1iVBwOjtmyVLhnLrJIX+2+3T3yauxdu+pmBsnD5OIKUrBrN1sdwW0rA2rHDiSnzXHNQM3m02aY6mlagdQ/Ovh96h05QFCHYxBc6oE/mIeFRaNifa4GU/oELn3a6HfbETeBQz+XOEN+IrLpnZO9riGyzsZroB/Y3Ju+cJxH06U0B7xwJCRmWZjuvfFQUP7RIJD1gRGZzmf3h8+F+oidkO2i5rbT57NaYSqkdVvR6RidVLWEzURZIGbtHjSPCi4kqD05ua8r/7CC0PvxQb1O5ILEdyJr2ZmzhF6VjjgmyrmSmt/yRq8MQtGQxyKXZhJqlPYho4d5SrHi5iGT2PvgDQaWch0I3ndEicaaPDZJHWBxVsCVAe44Wtj9g3LzXkyu3k= [email protected]
  7.   password: rancher
  8. install:
  9.   mode: create
  10.   networks:
  11.     harvester-mgmt:       # (Mandatory) The management bond name.
  12.       interfaces:
  13.       - name: ens5
  14.       default_route: true
  15.       method: dhcp
  16.       bond_options:
  17.         mode: balance-tlb
  18.         miimon: 100
  19.     harvester-vlan:       # (Optional) The VLAN network bond name. If VLAN NIC names vary from
  20.       interfaces:         # host to host, consider creating a bonding device. Users can then select
  21.       - name: ens6        # `harvester-vlan` as the VLAN network NIC in the Harvester GUI.
  22.       method: none
  23.       bond_options:
  24.         mode: balance-tlb
  25.         miimon: 100
  26.   device: /dev/sda
  27.   iso_url: http://10.100.0.10/harvester/harvester-<version>-amd64.iso
  28.   vip: 10.100.0.100       # The VIP to access the Harvester GUI. Make sure the IP is free to use.
  29.   vip_mode: static        # Or dhcp, check configuration file for more information.

For machines that needs to be installed using CREATE mode, the following is an iPXE script that boots the kernel with the above config:

  1. #!ipxe
  2. kernel harvester-<version>-vmlinuz ip=dhcp net.ifnames=1 rd.cos.disable rd.noverifyssl console=tty1 root=live:http://10.100.0.10/harvester/rootfs.squashfs harvester.install.automatic=true harvester.install.config_url=http://10.100.0.10/harvester/config-create.yaml
  3. initrd harvester-<version>-initrd
  4. boot

This assumes the iPXE script is stored in /usr/share/nginx/html/harvester/ipxe-create.

Note

If there are multiple network interfaces on the installing machine, the user can use ip=<interface>:dhcp to specify the booting interface (e.g., ip=eth1:dhcp).

JOIN Mode

Warning

Security Risks: The configuration file below contains credentials which should be kept secret. Please do not make the configuration file publicly accessible.

Create a Harvester configuration file called config-join.yaml for JOIN mode. Modify the values as needed:

  1. # cat /usr/share/nginx/html/harvester/config-join.yaml
  2. server_url: https://10.100.0.130:8443
  3. token: token
  4. os:
  5.   hostname: node2
  6.   ssh_authorized_keys:
  7.   - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbeUa9A7Kee+hcCleIXYxuaPksn2m4PZTd4T7wPcse8KbsQfttGRax6vxQXoPO6ehddqOb2nV7tkW2mEhR50OE7W7ngDHbzK2OneAyONYF44bmMsapNAGvnsBKe9rNrev1iVBwOjtmyVLhnLrJIX+2+3T3yauxdu+pmBsnD5OIKUrBrN1sdwW0rA2rHDiSnzXHNQM3m02aY6mlagdQ/Ovh96h05QFCHYxBc6oE/mIeFRaNifa4GU/oELn3a6HfbETeBQz+XOEN+IrLpnZO9riGyzsZroB/Y3Ju+cJxH06U0B7xwJCRmWZjuvfFQUP7RIJD1gRGZzmf3h8+F+oidkO2i5rbT57NaYSqkdVvR6RidVLWEzURZIGbtHjSPCi4kqD05ua8r/7CC0PvxQb1O5ILEdyJr2ZmzhF6VjjgmyrmSmt/yRq8MQtGQxyKXZhJqlPYho4d5SrHi5iGT2PvgDQaWch0I3ndEicaaPDZJHWBxVsCVAe44Wtj9g3LzXkyu3k= [email protected]
  8.   dns_nameservers:
  9.   - 1.1.1.1
  10.   - 8.8.8.8
  11.   password: rancher
  12. install:
  13.   mode: join
  14.   networks:
  15.     harvester-mgmt:       # (Mandatory) The management bond name.
  16.       interfaces:
  17.       - name: ens5
  18.       default_route: true
  19.       method: dhcp
  20.       bond_options:
  21.         mode: balance-tlb
  22.         miimon: 10
  23.     harvester-vlan:       # (Optional) The VLAN network bond name. If VLAN NIC names vary from
  24.       interfaces:         # host to host, consider creating a bonding device. Users can then select
  25.       - name: ens6        # `harvester-vlan` as the VLAN network NIC in the Harvester GUI.
  26.       method: none
  27.       bond_options:
  28.         mode: balance-tlb
  29.         miimon: 100
  30.   device: /dev/sda
  31.   iso_url: http://10.100.0.10/harvester/harvester-<version>-amd64.iso

Note that the mode is join and the server_url needs to be provided.

For machines that needs to be installed in JOIN mode, the following is an iPXE script that boots the kernel with the above config:

  1. #!ipxe
  2. kernel harvester-<version>-vmlinuz ip=dhcp net.ifnames=1 rd.cos.disable rd.noverifyssl console=tty1 root=live:http://10.100.0.10/harvester/rootfs.squashfs harvester.install.automatic=true harvester.install.config_url=http://10.100.0.10/harvester/config-join.yaml
  3. initrd harvester-<version>-initrd
  4. boot

This assumes the iPXE script is stored in /usr/share/nginx/html/harvester/ipxe-join.

DHCP Server Configuration

The following is an example of how to configure the ISC DHCP server to offer iPXE scripts:

  1. option architecture-type code 93 = unsigned integer 16;
  3. subnet 10.100.0.0 netmask 255.255.255.0 {
  4.     option routers 10.100.0.10;
  5.         option domain-name-servers 192.168.2.1;
  6.     range 10.100.0.100 10.100.0.253;
  7. }
  9. group {
  10.   # create group
  11.   if exists user-class and option user-class = "iPXE" {
  12.     # iPXE Boot
  13.     if option architecture-type = 00:07 {
  14.       filename "http://10.100.0.10/harvester/ipxe-create-efi";
  15.     } else {
  16.       filename "http://10.100.0.10/harvester/ipxe-create";
  17.     }
  18.   } else {
  19.     # PXE Boot
  20.     if option architecture-type = 00:07 {
  21.       # UEFI
  22.       filename "ipxe.efi";
  23.     } else {
  24.       # Non-UEFI
  25.       filename "undionly.kpxe";
  26.     }
  27.   }
  29.   host node1 { hardware ethernet 52:54:00:6b:13:e2; }
  30. }
  33. group {
  34.   # join group
  35.   if exists user-class and option user-class = "iPXE" {
  36.     # iPXE Boot
  37.     if option architecture-type = 00:07 {
  38.       filename "http://10.100.0.10/harvester/ipxe-join-efi";
  39.     } else {
  40.       filename "http://10.100.0.10/harvester/ipxe-join";
  41.     }
  42.   } else {
  43.     # PXE Boot
  44.     if option architecture-type = 00:07 {
  45.       # UEFI
  46.       filename "ipxe.efi";
  47.     } else {
  48.       # Non-UEFI
  49.       filename "undionly.kpxe";
  50.     }
  51.   }
  53.   host node2 { hardware ethernet 52:54:00:69:d5:92; }
  54. }

The config file declares a subnet and two groups. The first group is for hosts to boot using CREATE mode and the other one is for JOIN mode. By default, the iPXE path is chosen, but if it sees a PXE client it offers the iPXE image according to the client architecture. Please prepare those images and a TFTP server first.

Harvester Configuration

For more information about Harvester configuration, please refer to the Harvester configuration page.

Users can also provide configuration via kernel parameters. For example, to specify the CREATE install mode, users can pass the harvester.install.mode=create kernel parameter when booting. Values passed through kernel parameters have higher priority than values specified in the config file.

UEFI HTTP Boot support

UEFI firmware supports loading a boot image from an HTTP server. This section demonstrates how to use UEFI HTTP boot to load the iPXE program and perform an automatic installation.

Serve the iPXE Program

Download the iPXE UEFI program from http://boot.ipxe.org/ipxe.efi and make sure ipxe.efi can be downloaded from the HTTP server. For example:

  1. cd /usr/share/nginx/html/harvester/
  2. wget http://boot.ipxe.org/ipxe.efi

The file now can be downloaded from http://10.100.0.10/harvester/ipxe.efi locally.

DHCP Server Configuration

If the user plans to use the UEFI HTTP boot feature by getting a dynamic IP first, the DHCP server needs to provide the iPXE program URL when it sees such a request. The following is an updated ISC DHCP server group example:

  1. group {
  2.   # create group
  3.   if exists user-class and option user-class = "iPXE" {
  4.     # iPXE Boot
  5.     if option architecture-type = 00:07 {
  6.       filename "http://10.100.0.10/harvester/ipxe-create-efi";
  7.     } else {
  8.       filename "http://10.100.0.10/harvester/ipxe-create";
  9.     }
  10.   } elsif substring (option vendor-class-identifier, 0, 10) = "HTTPClient" {
  11.     # UEFI HTTP Boot
  12.     option vendor-class-identifier "HTTPClient";
  13.     filename "http://10.100.0.10/harvester/ipxe.efi";
  14.   } else {
  15.     # PXE Boot
  16.     if option architecture-type = 00:07 {
  17.       # UEFI
  18.       filename "ipxe.efi";
  19.     } else {
  20.       # Non-UEFI
  21.       filename "undionly.kpxe";
  22.     }
  23.   }
  25.   host node1 { hardware ethernet 52:54:00:6b:13:e2; }
  26. }

The elsif substring statement is new, and it offers http://10.100.0.10/harvester/ipxe.efi when it sees a UEFI HTTP boot DHCP request. After the client fetches the iPXE program and runs it, the iPXE program will send a DHCP request again and load the iPXE script from the URL http://10.100.0.10/harvester/ipxe-create-efi.

The iPXE Script for UEFI Boot

It’s mandatory to specify the initrd image for UEFI boot in the kernel parameters. The following is an updated version of iPXE script for CREATE mode.

  1. #!ipxe
  2. kernel harvester-<version>-vmlinuz initrd=harvester-<version>-initrd ip=dhcp net.ifnames=1 rd.cos.disable rd.noverifyssl console=tty1 root=live:http://10.100.0.10/harvester/rootfs.squashfs harvester.install.automatic=true harvester.install.config_url=http://10.100.0.10/harvester/config-create.yaml
  3. initrd harvester-<version>-initrd
  4. boot

The parameter initrd=harvester-<version>-initrd is required.