Storage

CSI

k0s supports a wide range of different storage options by utilizing Container Storage Interface (CSI). All Kubernetes storage solutions are supported and users can easily select the storage that fits best for their needs.

When the storage solution implements CSI, kubernetes can communicate with the storage to create and configure persistent volumes. This makes it easy to dynamically provision the requested volumes. It also expands the supported storage solutions from the previous generation, in-tree volume plugins. More information about the CSI concept is described on the Kubernetes Blog.

k0s storage

Installing 3rd party storage solutions

Follow your storage driver’s installation instructions. Note that by default the Kubelet installed by k0s uses a slightly different path for its working directory (/varlib/k0s/kubelet instead of /var/lib/kubelet). Consult the CSI driver’s configuration documentation on how to customize this path. The actual path can differ if you defined the flag --data-dir.

Example storage solutions

Different Kubernetes storage solutions are explained in the official Kubernetes storage documentation. All of them can be used with k0s. Here are some popular ones:

  • Rook-Ceph (Open Source)
  • MinIO (Open Source)
  • Gluster (Open Source)
  • Longhorn (Open Source)
  • Amazon EBS
  • Google Persistent Disk
  • Azure Disk
  • Portworx

If you are looking for a fault-tolerant storage with data replication, you can find a k0s tutorial for configuring Ceph storage with Rook in here.

Bundled OpenEBS storage (deprecated)

Bundled OpenEBS was deprecated in favor of running it as a helm extension, this documentation is maintained as a reference for existing installations.

This was done for three reasons:

  1. By installing it as a helm extension, users have more control and flexibility without adding complexity.
  2. It allows users to choose the OpenEBS version independent of their k0s version.
  3. It makes the k0s configuration more consistent.

For new installations or to migrate existing installations, please refer to the OpenEBS extension page.

The OpenEBS extension is enabled by setting spec.extensions.storage.type to`openebs_local_storage:

  1. spec:
  2.   extensions:
  3.     storage:
  4.       type: openebs_local_storage

The cluster will have two storage classes available for you to use:

  1. k0s kubectl get storageclass
  1. NAME               PROVISIONER        RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
  2. openebs-device     openebs.io/local   Delete          WaitForFirstConsumer   false                  24s
  3. openebs-hostpath   openebs.io/local   Delete          WaitForFirstConsumer   false                  24s

The openebs-hostpath is the storage class that maps to the /var/openebs/local

The openebs-device is not configured and could be configured by manifest deployer accordingly to the OpenEBS documentation

Example usage

Use following manifests as an example of pod with mounted volume:

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.   name: nginx-pvc
  5.   namespace: default
  6. spec:
  7.   accessModes:
  8.     - ReadWriteOnce
  9.   storageClassName: openebs-hostpath
  10.   resources:
  11.     requests:
  12.       storage: 5Gi
  13. ---
  14. apiVersion: apps/v1
  15. kind: Deployment
  16. metadata:
  17.   name: nginx
  18.   namespace: default
  19.   labels:
  20.     app: nginx
  21. spec:
  22.   selector:
  23.     matchLabels:
  24.       app: nginx
  25.   strategy:
  26.     type: Recreate
  27.   template:
  28.     metadata:
  29.       labels:
  30.         app: nginx
  31.     spec:
  32.       containers:
  33.       - image: nginx 
  34.         name: nginx
  35.         volumeMounts:
  36.         - name: persistent-storage
  37.           mountPath: /var/lib/nginx
  38.       volumes:
  39.       - name: persistent-storage
  40.         persistentVolumeClaim:
  41.           claimName: nginx-pvc
  1. k0s kubectl apply -f nginx.yaml
  1. persistentvolumeclaim/nginx-pvc created
  2. deployment.apps/nginx created
  3. bash-5.1# k0s kc get pods
  4. NAME                    READY   STATUS    RESTARTS   AGE
  5. nginx-d95bcb7db-gzsdt   1/1     Running   0          30s
  1. k0s kubectl get pv
  1. NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM               STORAGECLASS       REASON   AGE
  2. pvc-9a7fae2d-eb03-42c3-aaa9-1a807d5df12f   5Gi        RWO            Delete           Bound    default/nginx-pvc   openebs-hostpath            30s