Container-Optimized OS (COS) Support

Requirements

Note: Longhorn currently supports Container-Optimized OS only when used as the base image for Google Kubernetes Engine (GKE), which includes a pre-configured Kubernetes environment. The following information may not apply to manually created Kubernetes environments, including Kubernetes provisioned with other orchestrators.

The Container-Optimized OS (COS) does not include a package manager and does not allow non-containerized applications to run. Additionally, its root filesystem is mounted as read-only, which poses a challenge for IO operations.

In GKE, Kubernetes tackles these constraints by housing necessary dependencies in a chroot environment (/home/kubernetes/containerized_mounter/rootfs) and mounting directories within it, enabling the execution of required tasks.

Longhorn provides a GKE COS node agent daemonset, which leverages GKE Kubernetes solutions to configure and run necessary dependencies. This agent is responsible for the following operations:

  • Mounting the Longhorn data path.
  • Loading the kernel module.
  • Installing and running the iSCSI daemon.

GKE COS Node Agent Installation

  1. Configure the Longhorn GKE COS node agent. You can use the default settings, if applicable.

    Tip: You can use a comma-separated list when specifying values for the node-agent container’s environment variable (LONGHORN_DATA_PATHS).

    Example:

    1.  containers:
    2.    - name: node-agent
    3.      env:
    4.        - name: LONGHORN_DATA_PATHS
    5.          value: /var/lib/longhorn1,/var/lib/longhorn2

  2. Install the Longhorn GKE COS node agent.

    1. kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.7.1/deploy/prerequisite/longhorn-gke-cos-node-agent.yaml

  3. Check the agent pod’s status. Example:

    1. $ kubectl -n longhorn-system get pod -l app=longhorn-gke-cos-node
    2. NAME                                READY   STATUS    RESTARTS      AGE
    3. longhorn-gke-cos-node-agent-222w8   1/1     Running   1 (86m ago)   86m
    4. longhorn-gke-cos-node-agent-8r26h   1/1     Running   1 (86m ago)   86m
    5. longhorn-gke-cos-node-agent-nwhsw   1/1     Running   1 (86m ago)   86m

  4. Check the installation result in the agent pod logs.

    1. Completed!
    2. Keep the container running for iscsi daemon

    Note: The agent installs the iSCSI daemon (iscsid) in a container using a package manager. However, the package manager attempts to initiate iSCSI services through systemd, which the container environment does not fully support. As a result, you will likely see error logs similar to System has not been booted with systemd as init system (PID 1). Can't operate. To work around this, the script manually starts the daemon instead of relying on systemd. You can disregard the mentioned errors in this context.

  5. Verify that the dependent kernel module is loaded. You must run the command on the host.

    1. $ lsmod | grep -q iscsi_tcp && echo "The iSCSI module is loaded" || echo "The iSCSI module is NOT loaded"
    2. The iSCSI module is loaded

  6. Verify that the iSCSI daemon is running. You must run the command on the host.

    1. $ ps aux | grep -q '[i]scsid' && echo "The iSCSI daemon is running" || echo "The iSCSI daemon is NOT running"
    2. The iSCSI daemon is running

  7. Verify that the Longhorn data path (/var/lib/longhorn) is mounted on the host. If you specified multiple Longhorn data paths, run the command for each path on the host.

    1. $ findmnt --noheadings "/var/lib/longhorn"
    2. /var/lib/longhorn /dev/sda1[/var/lib/longhorn] ext4   rw,relatime,commit=30

Limitations

In COS clusters, Longhorn currently supports only V1 data volumes.

References