Volumes and Storage

When deploying an application that needs to retain data, you’ll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the application’s pod fails.

A persistent volume (PV) is a piece of storage in the Kubernetes cluster, while a persistent volume claim (PVC) is a request for storage. For details on how PVs and PVCs work, refer to the official Kubernetes documentation on storage.

This page describes how to set up persistent storage with a local storage provider, or with Longhorn.

What’s different about K3s storage?

K3s removes several optional volume plugins and all built-in (sometimes referred to as “in-tree”) cloud providers. We do this in order to achieve a smaller binary size and to avoid dependence on third-party cloud or data center technologies and services, which may not be available in many K3s use cases. We are able to do this because their removal affects neither core Kubernetes functionality nor conformance.

The following volume plugins have been removed from K3s:

  • cephfs
  • fc
  • flocker
  • git_repo
  • glusterfs
  • portworx
  • quobyte
  • rbd
  • storageos

Both components have out-of-tree alternatives that can be used with K3s: The Kubernetes Container Storage Interface (CSI) and Cloud Provider Interface (CPI).

Kubernetes maintainers are actively migrating in-tree volume plugins to CSI drivers. For more information on this migration, please refer here.

Setting up the Local Storage Provider

K3s comes with Rancher’s Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation here.

Create a hostPath backed persistent volume claim and a pod to utilize it:

pvc.yaml

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.   name: local-path-pvc
  5.   namespace: default
  6. spec:
  7.   accessModes:
  8.     - ReadWriteOnce
  9.   storageClassName: local-path
  10.   resources:
  11.     requests:
  12.       storage: 2Gi

pod.yaml

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: volume-test
  5.   namespace: default
  6. spec:
  7.   containers:
  8.   - name: volume-test
  9.     image: nginx:stable-alpine
  10.     imagePullPolicy: IfNotPresent
  11.     volumeMounts:
  12.     - name: volv
  13.       mountPath: /data
  14.     ports:
  15.     - containerPort: 80
  16.   volumes:
  17.   - name: volv
  18.     persistentVolumeClaim:
  19.       claimName: local-path-pvc

Apply the yaml:

  1. kubectl create -f pvc.yaml
  2. kubectl create -f pod.yaml

Confirm the PV and PVC are created:

  1. kubectl get pv
  2. kubectl get pvc

The status should be Bound for each.

Setting up Longhorn

Volumes and Storage - 图1warning

Longhorn does not support ARM32.

K3s supports Longhorn, an open-source distributed block storage system for Kubernetes.

Below we cover a simple example. For more information, refer to the official documentation.

Apply the longhorn.yaml to install Longhorn:

  1. kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.6.0/deploy/longhorn.yaml

Longhorn will be installed in the namespace longhorn-system.

Create a persistent volume claim and a pod to utilize it:

pvc.yaml

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.   name: longhorn-volv-pvc
  5. spec:
  6.   accessModes:
  7.     - ReadWriteOnce
  8.   storageClassName: longhorn
  9.   resources:
  10.     requests:
  11.       storage: 2Gi

pod.yaml

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: volume-test
  5.   namespace: default
  6. spec:
  7.   containers:
  8.   - name: volume-test
  9.     image: nginx:stable-alpine
  10.     imagePullPolicy: IfNotPresent
  11.     volumeMounts:
  12.     - name: volv
  13.       mountPath: /data
  14.     ports:
  15.     - containerPort: 80
  16.   volumes:
  17.   - name: volv
  18.     persistentVolumeClaim:
  19.       claimName: longhorn-volv-pvc

Apply the yaml to create the PVC and pod:

  1. kubectl create -f pvc.yaml
  2. kubectl create -f pod.yaml

Confirm the PV and PVC are created:

  1. kubectl get pv
  2. kubectl get pvc

The status should be Bound for each.