Prerequisites (iSCSI client)

OpenEBS configuration flow

OpenEBS provides block volume support through the iSCSI protocol. Therefore, the iSCSI client (initiator) presence on all Kubernetes nodes is required. Choose the platform below to find the steps to verify if the iSCSI client is installed and running or to find the steps to install the iSCSI client.

Choose the platform for iSCSI client settings

Prerequisites - 图2Ubuntu

Prerequisites - 图3RHEL

Prerequisites - 图4CentOS

Prerequisites - 图5EKS

Prerequisites - 图6GKE

Prerequisites - 图7AKS

Prerequisites - 图8OpenShift

Prerequisites - 图9Rancher

Prerequisites - 图10ICP

Provide feedback if a platform is missing in the above list.

Linux platforms

Installation of the iSCSI initiator service and tools depends on your host O/S or the kubelet container. You can follow the below steps for installation/verification of the required packages. It is a mandatory step to verify the iSCSI services and make sure that it is running on all the worker nodes. OpenEBS uses the iSCSI protocol to connect to the block volumes.

Ubuntu

Verify iSCSI services are configured

If an iSCSI initiator is already installed on your node, check that the initiator name is configured and iSCSI service is running using the following commands.

  1. sudo cat /etc/iscsi/initiatorname.iscsi
  2. systemctl status iscsid

If service status is showing as Inactive, then you may have to enable and start the iscsid service using the following command.

  1. sudo systemctl enable iscsid && sudo systemctl start iscsid

The following is the expected output.

  1. systemctl status iscsid
  2. ● iscsid.service - iSCSI initiator daemon (iscsid)
  3.    Loaded: loaded (/lib/systemd/system/iscsid.service; disabled; vendor preset: enabled)
  4.    Active: active (running) since Mon 2019-02-18 11:00:07 UTC; 1min 51s ago
  5.      Docs: man:iscsid(8)
  6.   Process: 11185 ExecStart=/sbin/iscsid (code=exited, status=0/SUCCESS)
  7.   Process: 11170 ExecStartPre=/lib/open-iscsi/startup-checks.sh (code=exited, status=0/SUCCESS)
  8.  Main PID: 11187 (iscsid)
  9.     Tasks: 2 (limit: 4915)
  10.    CGroup: /system.slice/iscsid.service
  11.            ├─11186 /sbin/iscsid
  12.            └─11187 /sbin/iscsid

Install the iSCSI tools

If an iSCSI initiator is not installed on your node, install open-iscsi packages using the following commands.

  1. sudo apt-get update
  2. sudo apt-get install open-iscsi
  3. sudo service open-iscsi restart

You can verify the iSCSI installation from above section.

Red Hat Enterprise Linux

Verify iSCSI services are configured

In Red Hat Enterprise Linux 7, the iSCSI service is lazily started by default: the service starts after running the iscsiadm command. If an iSCSI initiator is already installed on the node, check that the initiator name is configured using the following command.

  1.  cat /etc/iscsi/initiatorname.iscsi

Check iSCSI service is running using the following command.

  1.  systemctl status iscsid

If the status is showing as Inactive, then you may have to enable and start the iscsid service using the following command.

  1. sudo systemctl enable iscsid && sudo systemctl start iscsid

The following is the expected output.

  1. systemctl status iscsid
  2. ● iscsid.service - Open-iSCSI
  3.    Loaded: loaded (/usr/lib/systemd/system/iscsid.service; enabled; vendor preset: disabled)
  4.    Active: active (running) since Tue 2019-02-19 12:19:08 IST; 2h 37min ago
  5.      Docs: man:iscsid(8)
  6.            man:iscsiadm(8)
  7.  Main PID: 2138 (iscsid)
  8.     Tasks: 2
  9.    CGroup: /system.slice/iscsid.service
  10.            ├─2137 /usr/sbin/iscsid
  11.            └─2138 /usr/sbin/iscsid
  13. Feb 19 12:19:08 master-1550555590.mayalab.com systemd[1]: Starting Open-iSCSI...
  14. Feb 19 12:19:08 master-1550555590.mayalab.com iscsid[2136]: iSCSI logger with pid=2137 started!
  15. Feb 19 12:19:08 master-1550555590.mayalab.com systemd[1]: Failed to read PID from file /var/run/iscsid.pid: Invalid argument
  16. Feb 19 12:19:08 master-1550555590.mayalab.com systemd[1]: Started Open-iSCSI.
  17. Feb 19 12:19:09 master-1550555590.mayalab.com iscsid[2137]: iSCSI daemon with pid=2138 started!

Install the iSCSI tools

If an iSCSI initiator is not installed on your node, install iscsi-initiator-utils packages using the following command.

  1. yum install iscsi-initiator-utils -y

You can verify the iSCSI installation from above section.

CentOS

Verify iSCSI services are configured

If an iSCSI initiator is already installed on your node, check that the initiator name is configured using the following commands.

  1.  cat /etc/iscsi/initiatorname.iscsi

Check iSCSI service is running using the following command.

  1.  systemctl status iscsid

If the status is showing as Inactive, then you may have to enable and start the iscsid service using the following command.

  1. sudo systemctl enable iscsid && sudo systemctl start iscsid

Install the iSCSI tools

If an iSCSI initiator is not installed on your node, install iscsi-initiator-utils packages using the following command.

  1. yum install iscsi-initiator-utils -y

You can verify the iSCSI installation from the above section.

Managed Kubernetes Services on Public Cloud

Amazon Elastic Container Service for Kubernetes (EKS)

Amazon EKS clusters can be brought up with either an AmazonLinux AMI or an Ubuntu 18.04 AMI.

For clusters running with the AmazonLinux AMI

Verify iSCSI services are configured

If an iSCSI initiator is already installed on your node, check that the initiator name is configured using the following commands.

  1.  cat /etc/iscsi/initiatorname.iscsi

Check the iSCSI service is running using the following command.

  1.  systemctl status iscsid

If the status is showing as Inactive, then you may have to enable and start the iscsid service using the following command.

  1. sudo systemctl enable iscsid && sudo systemctl start iscsid

Install the iSCSI tools

If an iSCSI initiator is not installed on your node, install iscsi-initiator-utils packages using the following command.

  1. yum install iscsi-initiator-utils -y

You can verify the iSCSI installation from the above section.

For clusters running with the Ubuntu 18.04 AMI

For setting up iSCSI clients on Ubuntu nodes, see the instructions above.

Google Kubernetes Engine (GKE)

GKE Container-Optimized OS does not come with an iSCSI client preinstalled and does not allow installation of an iSCSI client. Therefore, OpenEBS does not work on Kubernetes clusters which are running GKE Container-Optimized OS version of the image on the worker nodes.

Select Ubuntu as the image version for the node pools in the custom settings. For setting up iSCSI clients on Ubuntu nodes, see the instructions above.

Azure Kubernetes Service (AKS)

On Azure Kubernetes Service you need to verify that the open-iscsi packages are installed and running the service on the kubelet. This can be checked by connecting to the nodes through SSH using the public IP addresses by running the following command.

  1. ssh azureuser@40.xx.yyy.zzz

Note: azureuser is a default username.

Verify iSCSI services are configured

Obtain the container ID of the hyperkube kubelet on each node using the following command.

  1. sudo docker ps | grep "hyperkube kubelet"

Following is the example output:

  1. 3aab0f9a48e2    k8s-gcrio.azureedge.net/hyperkube-amd64:v1.8.7   "/hyperkube kubele..."   48 minutes ago      Up 48 minutes                           eager_einstein

Once kubelet container ID is obtained, you need to get to the shell of this container using the following command.

  1. sudo docker exec -it <container ID> bash

Example:

  1. sudo docker exec -it 3aab0f9a48e2 bash

Check the status of the iSCSI service by using the following command.

  1. service open-iscsi status

Install the iSCSI tools

You have to get the kubelet container ID using the steps mentioned in the above section. Once kubelet container ID is obtained, you need to get to the shell of this container using the following command.

  1. sudo docker exec -it <container ID> bash

Example:

  1. sudo docker exec -it 3aab0f9a48e2 bash

Run the following commands to install and configure iSCSI service in the kubelet.

  1. apt-get update
  2. apt install -y open-iscsi
  3. exit

You can verify the iSCSI installation from the above section.

Kubernetes On-Prem Solutions

Red Hat OpenShift

OpenEBS can be installed using Openshift on the following Operating Systems. Tested OpenShift versions are 3.9 and 3.10.

  1. On RHEL 7
  2. On CentOS 7

On RHEL

Latest tested RHEL versions are 7.5, 7.6.

For setting up iSCSI clients on RHEL nodes, see the instructions above.

On CentOS

Latest tested CentOS version is CentOS 7.

For setting up iSCSI clients on CentOS nodes, see the instructions above.

Rancher

  • If you are using RancherOS as the operating system for your Kubernetes cluster, you simply need to enable the iSCSI service and start it on all the hosts or nodes.
  • If you are using Ubuntu or RHEL as the operating system for your Kubernetes cluster, you need to
    • Verify iSCSI initiators are installed on all nodes (and )
    • Add the extra_binds under Kubelet service in cluster YAML file to mount the iSCSI binary and configuration inside the Kubelet.

iSCSI services On RancherOS

To run iSCSI services, execute the following commands on each of the cluster hosts or nodes.

  1. sudo ros s enable open-iscsi
  2. sudo ros s up open-iscsi

Run the below commands on all the nodes to make sure the below directories are persistent, by default these directories are ephemeral.

  1. ros config set rancher.services.user-volumes.volumes  [/home:/home,/opt:/opt,/var/lib/kubelet:/var/lib/kubelet,/etc/kubernetes:/etc/kubernetes,/var/openebs]
  2. system-docker rm all-volumes
  3. reboot

iSCSI services on RHEL or Ubuntu

Step1: Verify iSCSI initiator is installed and services are running

Operating systemiSCSI PackageCommands
RHEL / CentOSiscsi-initiator-utilsyum install iscsi-initiator-utils -y
sudo systemctl enable iscsid && sudo systemctl start iscsid
modprobe iscsi_tcp
Ubuntu/Debianopen-iscsisudo apt install open-iscsi
sudo systemctl enable iscsid && sudo systemctl start iscsid
modprobe iscsi_tcp

Step2: Add extra_binds under kubelet service in cluster YAML

After installing the initiator tool on your nodes, edit the YAML for your cluster, editing the kubelet configuration to mount the iSCSI binary and configuration, as shown in the sample below.

  1. services:
  2.     kubelet: 
  3.       extra_binds: 
  4.         - "/etc/iscsi:/etc/iscsi"
  5.         - "/sbin/iscsiadm:/sbin/iscsiadm"
  6.         - "/var/lib/iscsi:/var/lib/iscsi"
  7.         - "/lib/modules"

IBM Cloud Private (ICP)

OpenEBS can be installed using ICP on the following Operating Systems. Latest tested ICP versions are 2.1.0.3 and 3.1.1.

  1. On RHEL 7

  2. On Ubuntu

On RHEL

Latest tested RHEL versions are 7.5, 7.6.

For setting up iSCSI clients on RHEL nodes, see the instructions above.

On Ubuntu

Latest tested Ubuntu version are Ubuntu 16.04 LTS and 18.04 LTS.

For setting up iSCSI clients on Ubuntu nodes, see the instructions above.

See Also:

OpenEBS Installation

OpenEBS Architecture