OpenShift

OpenShift adds a number of security and other enhancements to Kubernetes. In particular, security context constraints allow the cluster admin to define exactly which permissions are allowed to pods running in the cluster. You will need to define those permissions that allow the Rook pods to run.

Rook Privileges

To orchestrate the storage platform, Rook requires the following access in the cluster:

  • Create hostPath volumes, for persistence by the Ceph mon and osd pods
  • Run pods in privileged mode, for access to /dev and hostPath volumes
  • Host networking for the Rook agent and clusters that require host networking
  • Ceph OSDs require host PIDs for communication on the same node

Security Context Constraints

Before starting the Rook operator or cluster, create the security context constraints needed by the Rook pods. The following yaml is found in scc.yaml under /cluster/examples/kubernetes/ceph.

  1. kind: SecurityContextConstraints
  2. apiVersion: v1
  3. metadata:
  4.   name: rook-ceph
  5. allowPrivilegedContainer: true
  6. allowHostNetwork: true
  7. allowHostDirVolumePlugin: true
  8. priority: 
  9. allowedCapabilities: []
  10. allowHostPorts: false
  11. allowHostPID: true
  12. allowHostIPC: false
  13. readOnlyRootFilesystem: false
  14. requiredDropCapabilities: []
  15. defaultAddCapabilities: []
  16. runAsUser:
  17.   type: RunAsAny
  18. seLinuxContext:
  19.   type: MustRunAs
  20. fsGroup:
  21.   type: MustRunAs
  22. supplementalGroups:
  23.   type: RunAsAny
  24. allowedFlexVolumes: 
  25.   - driver: "rook.io/rook"
  26. volumes:
  27.   - configMap
  28.   - downwardAPI
  29.   - emptyDir
  30.   - flexVolume
  31.   - hostPath
  32.   - persistentVolumeClaim
  33.   - projected
  34.   - secret
  35. users:
  36.   # A user needs to be added for each rook service account. 
  37.   # This assumes running in the default sample "rook-ceph" and "rook-ceph-system" namespaces.
  38.   # If other namespaces or service accounts are configured, they need to be updated here.
  39.   - system:serviceaccount:rook-ceph-system:rook-ceph-system
  40.   - system:serviceaccount:rook-ceph:default
  41.   - system:serviceaccount:rook-ceph:rook-ceph-cluster

Important to note is that if you plan on running Rook in namespaces other than the defaults of rook-ceph-system and rook-ceph, the example scc will need to be modified to accommodate for your namespaces where the Rook pods are running.

To create the scc you will need a privileged account:

  1. oc login -u system:admin

Now create the scc:

  1. oc create -f scc.yaml

Rook Settings

There are some Rook settings that also need to be adjusted to work in OpenShift.

Operator Settings

There is an environment variable that needs to be set in the operator spec that will allow Rook to run in OpenShift clusters.

  • ROOK_HOSTPATH_REQUIRES_PRIVILEGED: Must be set to true. Writing to the hostPath is required for the Ceph mon and osd pods. Given the restricted permissions in OpenShift with SELinux, the pod must be running privileged in order to write to the hostPath volume.
  1. - name: ROOK_HOSTPATH_REQUIRES_PRIVILEGED
  2.     value: "true"

Cluster Settings

The cluster settings in cluster.yaml are largely isolated from the differences in OpenShift. There is perhaps just one to take note of:

  • dataDirHostPath: Ensure that it points to a valid, writable path on the host systems.

Object Store Settings

In OpenShift, ports less than 1024 cannot be bound. In the object store CRD, ensure the port is modified to meet this requirement.

  1. gateway:
  2.     port: 8080

You can expose a different port such as 80 by creating a service.

MiniShift

There is a known issue in MiniShift that does not allow Rook to be tested in some common end-to-end scenarios. Flex drivers are not currently supported, which means that block and file volumes cannot be mounted. See this tracking issue.