Rook Toolbox

The Rook toolbox is a container with common tools used for rook debugging and testing. The toolbox is based on CentOS, so more tools of your choosing can be easily installed with yum.

The toolbox can be run in two modes:

  1. Interactive: Start a toolbox pod where you can connect and execute Ceph commands from a shell
  2. One-time job: Run a script with Ceph commands and collect the results from the job log

Prerequisite: Before running the toolbox you should have a running Rook cluster deployed (see the Quickstart Guide).

Interactive Toolbox

The rook toolbox can run as a deployment in a Kubernetes cluster where you can connect and run arbitrary Ceph commands.

Save the tools spec as toolbox.yaml:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: rook-ceph-tools
  5.   namespace: rook-ceph
  6.   labels:
  7.     app: rook-ceph-tools
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: rook-ceph-tools
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: rook-ceph-tools
  17.     spec:
  18.       dnsPolicy: ClusterFirstWithHostNet
  19.       containers:
  20.       - name: rook-ceph-tools
  21.         image: rook/ceph:v1.4.1
  22.         command: ["/tini"]
  23.         args: ["-g", "--", "/usr/local/bin/toolbox.sh"]
  24.         imagePullPolicy: IfNotPresent
  25.         env:
  26.           - name: ROOK_CEPH_USERNAME
  27.             valueFrom:
  28.               secretKeyRef:
  29.                 name: rook-ceph-mon
  30.                 key: ceph-username
  31.           - name: ROOK_CEPH_SECRET
  32.             valueFrom:
  33.               secretKeyRef:
  34.                 name: rook-ceph-mon
  35.                 key: ceph-secret
  36.         volumeMounts:
  37.           - mountPath: /etc/ceph
  38.             name: ceph-config
  39.           - name: mon-endpoint-volume
  40.             mountPath: /etc/rook
  41.       volumes:
  42.         - name: mon-endpoint-volume
  43.           configMap:
  44.             name: rook-ceph-mon-endpoints
  45.             items:
  46.             - key: data
  47.               path: mon-endpoints
  48.         - name: ceph-config
  49.           emptyDir: {}
  50.       tolerations:
  51.         - key: "node.kubernetes.io/unreachable"
  52.           operator: "Exists"
  53.           effect: "NoExecute"
  54.           tolerationSeconds: 5

Launch the rook-ceph-tools pod:

  1. kubectl create -f toolbox.yaml

Wait for the toolbox pod to download its container and get to the running state:

  1. kubectl -n rook-ceph get pod -l "app=rook-ceph-tools"

Once the rook-ceph-tools pod is running, you can connect to it with:

  1. kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='{.items[0].metadata.name}') bash

All available tools in the toolbox are ready for your troubleshooting needs.

Example:

  • ceph status
  • ceph osd status
  • ceph df
  • rados df

When you are done with the toolbox, you can remove the deployment:

  1. kubectl -n rook-ceph delete deployment rook-ceph-tools

Toolbox Job

If you want to run Ceph commands as a one-time operation and collect the results later from the logs, you can run a script as a Kubernetes Job. The toolbox job will run a script that is embedded in the job spec. The script has the full flexibility of a bash script.

In this example, the ceph status command is executed when the job is created.

  1. apiVersion: batch/v1
  2. kind: Job
  3. metadata:
  4.   name: rook-ceph-toolbox-job
  5.   namespace: rook-ceph
  6.   labels:
  7.     app: ceph-toolbox-job
  8. spec:
  9.   template:
  10.     spec:
  11.       initContainers:
  12.       - name: config-init
  13.         image: rook/ceph:v1.4.1
  14.         command: ["/usr/local/bin/toolbox.sh"]
  15.         args: ["--skip-watch"]
  16.         imagePullPolicy: IfNotPresent
  17.         env:
  18.         - name: ROOK_CEPH_USERNAME
  19.           valueFrom:
  20.             secretKeyRef:
  21.               name: rook-ceph-mon
  22.               key: ceph-username
  23.         - name: ROOK_CEPH_SECRET
  24.           valueFrom:
  25.             secretKeyRef:
  26.               name: rook-ceph-mon
  27.               key: ceph-secret
  28.         volumeMounts:
  29.         - mountPath: /etc/ceph
  30.           name: ceph-config
  31.         - name: mon-endpoint-volume
  32.           mountPath: /etc/rook
  33.       containers:
  34.       - name: script
  35.         image: rook/ceph:v1.4.1
  36.         volumeMounts:
  37.         - mountPath: /etc/ceph
  38.           name: ceph-config
  39.           readOnly: true
  40.         command:
  41.         - "bash"
  42.         - "-c"
  43.         - |
  44.           # Modify this script to run any ceph, rbd, radosgw-admin, or other commands that could
  45.           # be run in the toolbox pod. The output of the commands can be seen by getting the pod log.
  46.           #
  47.           # example: print the ceph status
  48.           ceph status
  49.       volumes:
  50.       - name: mon-endpoint-volume
  51.         configMap:
  52.           name: rook-ceph-mon-endpoints
  53.           items:
  54.           - key: data
  55.             path: mon-endpoints
  56.       - name: ceph-config
  57.         emptyDir: {}
  58.       restartPolicy: Never

Create the toolbox job:

  1. kubectl create -f toolbox-job.yaml

After the job completes, see the results of the script:

  1. kubectl -n rook-ceph logs -l job-name=rook-ceph-toolbox-job