Rook Toolbox

The Rook toolbox is a container with common tools used for rook debugging and testing. The toolbox is based on CentOS, so more tools of your choosing can be easily installed with yum.

Running the Toolbox in Kubernetes

The rook toolbox can run as a deployment in a Kubernetes cluster. After you ensure you have a running Kubernetes cluster with rook deployed (see the Kubernetes instructions), launch the rook-ceph-tools pod.

Save the tools spec as toolbox.yaml:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: rook-ceph-tools
  5.   namespace: rook-ceph
  6.   labels:
  7.     app: rook-ceph-tools
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: rook-ceph-tools
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: rook-ceph-tools
  17.     spec:
  18.       dnsPolicy: ClusterFirstWithHostNet
  19.       containers:
  20.       - name: rook-ceph-tools
  21.         image: rook/ceph:v1.1.0
  22.         command: ["/tini"]
  23.         args: ["-g", "--", "/usr/local/bin/toolbox.sh"]
  24.         imagePullPolicy: IfNotPresent
  25.         env:
  26.           - name: ROOK_ADMIN_SECRET
  27.             valueFrom:
  28.               secretKeyRef:
  29.                 name: rook-ceph-mon
  30.                 key: admin-secret
  31.         securityContext:
  32.           privileged: true
  33.         volumeMounts:
  34.           - mountPath: /dev
  35.             name: dev
  36.           - mountPath: /sys/bus
  37.             name: sysbus
  38.           - mountPath: /lib/modules
  39.             name: libmodules
  40.           - name: mon-endpoint-volume
  41.             mountPath: /etc/rook
  42.       # if hostNetwork: false, the "rbd map" command hangs, see https://github.com/rook/rook/issues/2021
  43.       hostNetwork: true
  44.       volumes:
  45.         - name: dev
  46.           hostPath:
  47.             path: /dev
  48.         - name: sysbus
  49.           hostPath:
  50.             path: /sys/bus
  51.         - name: libmodules
  52.           hostPath:
  53.             path: /lib/modules
  54.         - name: mon-endpoint-volume
  55.           configMap:
  56.             name: rook-ceph-mon-endpoints
  57.             items:
  58.             - key: data
  59.               path: mon-endpoints

Launch the rook-ceph-tools pod:

  1. kubectl create -f toolbox.yaml

Wait for the toolbox pod to download its container and get to the running state:

  1. kubectl -n rook-ceph get pod -l "app=rook-ceph-tools"

Once the rook-ceph-tools pod is running, you can connect to it with:

  1. kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='{.items[0].metadata.name}') bash

All available tools in the toolbox are ready for your troubleshooting needs. Example:

  1. ceph status
  2. ceph osd status
  3. ceph df
  4. rados df

When you are done with the toolbox, you can remove the deployment:

  1. kubectl -n rook-ceph delete deployment rook-ceph-tools

Troubleshooting without the Toolbox

The Ceph tools will commonly be the only tools needed to troubleshoot a cluster. In that case, you can connect to any of the rook pods and execute the ceph commands in the same way that you would in the toolbox pod such as the mon pods or the operator pod. If connecting to the mon pods, make sure you connect to the mon most recently started. The mons keep the config updated in memory after starting and may not have the latest config on disk. For example, after starting the cluster connect to the mon2 pod instead of mon0.