Setting up Hubble Observability

Hubble is the observability layer of Cilium and can be used to obtain cluster-wide visibility into the network and security layer of your Kubernetes cluster. For more information about Hubble and its components, see the Observability section.

Note

This guide assumes that Cilium has been correctly installed in your Kubernetes cluster. Please see Quick Installation for more information. If unsure, run cilium status and validate that Cilium is up and running.

Enable Hubble in Cilium

Cilium CLIHelm

In order to enable Hubble, run the command cilium hubble enable as shown below:

  1. $ cilium hubble enable
  2. 🔑 Found existing CA in secret cilium-ca
  3. ✨ Patching ConfigMap cilium-config to enable Hubble...
  4. ♻️  Restarted Cilium pods
  5. 🔑 Generating certificates for Relay...
  6. 2021/04/13 17:11:23 [INFO] generate received request
  7. 2021/04/13 17:11:23 [INFO] received CSR
  8. 2021/04/13 17:11:23 [INFO] generating key: ecdsa-256
  9. 2021/04/13 17:11:23 [INFO] encoded CSR
  10. 2021/04/13 17:11:23 [INFO] signed certificate with serial number 365589302067830033295858933512588007090526050046
  11. 2021/04/13 17:11:24 [INFO] generate received request
  12. 2021/04/13 17:11:24 [INFO] received CSR
  13. 2021/04/13 17:11:24 [INFO] generating key: ecdsa-256
  14. 2021/04/13 17:11:24 [INFO] encoded CSR
  15. 2021/04/13 17:11:24 [INFO] signed certificate with serial number 644167683731852948186644541769558498727586273511
  16. ✨ Deploying Relay...

Tip

Enabling Hubble requires the TCP port 4244 to be open on all nodes running Cilium. This is required for Relay to operate correctly.

Run cilium status to validate that Hubble is enabled and running:

  1. $ cilium status
  2.     /¯¯\
  3.  /¯¯\__/¯¯\    Cilium:         OK
  4.  \__/¯¯\__/    Operator:       OK
  5.  /¯¯\__/¯¯\    Hubble:         OK
  6.  \__/¯¯\__/    ClusterMesh:    disabled
  7.     \__/
  9. DaemonSet         cilium                   Desired: 3, Ready: 3/3, Available: 3/3
  10. Deployment        cilium-operator          Desired: 1, Ready: 1/1, Available: 1/1
  11. Deployment        hubble-relay             Desired: 1, Ready: 1/1, Available: 1/1
  12. Containers:       cilium                   Running: 3
  13.                   cilium-operator          Running: 1
  14.                   hubble-relay             Running: 1
  15. Image versions    cilium-operator          quay.io/cilium/operator-generic:v1.9.5: 1
  16.                   hubble-relay             quay.io/cilium/hubble-relay:v1.9.5: 1
  17.                   cilium                   quay.io/cilium/cilium:v1.9.5: 3

If you installed Cilium via helm install, you may enable Hubble Relay and UI with the following command:

  1. helm upgrade cilium cilium/cilium --version 1.12.0 \
  2.    --namespace kube-system \
  3.    --reuse-values \
  4.    --set hubble.relay.enabled=true \
  5.    --set hubble.ui.enabled=true

Install the Hubble Client

In order to access the observability data collected by Hubble, install the Hubble CLI:

LinuxMacOSWindows

Download the latest hubble release:

  1. export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
  2. HUBBLE_ARCH=amd64
  3. if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
  5. sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
  7. rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Download the latest hubble release:

  1. export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
  2. HUBBLE_ARCH=amd64
  3. if [ "$(uname -m)" = "arm64" ]; then HUBBLE_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-darwin-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
  5. shasum -a 256 -c hubble-darwin-${HUBBLE_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC hubble-darwin-${HUBBLE_ARCH}.tar.gz /usr/local/bin
  7. rm hubble-darwin-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Download the latest hubble release:

  1. curl -LO "https://raw.githubusercontent.com/cilium/hubble/master/stable.txt"
  2. set /p HUBBLE_VERSION=<stable.txt
  3. curl -L --fail -O "https://github.com/cilium/hubble/releases/download/%HUBBLE_VERSION%/hubble-windows-amd64.tar.gz"
  4. curl -L --fail -O "https://github.com/cilium/hubble/releases/download/%HUBBLE_VERSION%/hubble-windows-amd64.tar.gz.sha256sum"
  5. certutil -hashfile hubble-windows-amd64.tar.gz SHA256
  6. type hubble-windows-amd64.tar.gz.sha256sum
  7. :: verify that the checksum from the two commands above match
  8. tar zxf hubble-windows-amd64.tar.gz

and move the hubble.exe CLI to a directory listed in the %PATH% environment variable after extracting it from the tarball.

Validate Hubble API Access

In order to access the Hubble API, create a port forward to the Hubble service from your local machine. This will allow you to connect the Hubble client to the local port 4245 and access the Hubble Relay service in your Kubernetes cluster. For more information on this method, see Use Port Forwarding to Access Application in a Cluster.

  1. $ cilium hubble port-forward&
  2. Forwarding from 0.0.0.0:4245 -> 4245
  3. Forwarding from [::]:4245 -> 4245

Now you can validate that you can access the Hubble API via the installed CLI:

  1. $ hubble status
  2. Healthcheck (via localhost:4245): Ok
  3. Current/Max Flows: 11917/12288 (96.98%)
  4. Flows/s: 11.74
  5. Connected Nodes: 3/3

You can also query the flow API and look for flows:

  1. $ hubble observe

Note

If you port forward to a port other than 4245, make sure to use the --server flag or HUBBLE_SERVER environment variable to set the Hubble server address (default: localhost:4245). For more information, check out Hubble CLI’s help message by running hubble help status or hubble help observe as well as hubble config for configuring Hubble CLI.

Next Steps