L7-Aware Traffic Management

Cilium provides a way to control L7 traffic via CRDs (e.g. CiliumEnvoyConfig and CiliumClusterwideEnvoyConfig).

Prerequisites

  • Cilium must be configured with kubeProxyReplacement as partial or strict. Please refer to kube-proxy replacement for more details.

  • The minimum supported Kubernetes version for Ingress is 1.19.

Installation

HelmCilium CLI

Cilium Ingress Controller can be enabled with helm flag ingressController.enabled set as true. Please refer to Installation using Helm for a fresh installation.

  1. $ helm upgrade cilium cilium/cilium --version 1.12.0 \
  2.     --namespace kube-system \
  3.     --reuse-values \
  4.     --set ingressController.enabled=true
  5. $ kubectl -n kube-system rollout restart deployment/cilium-operator
  6. $ kubectl -n kube-system rollout restart ds/cilium

If you only want to use envoy traffic management feature without Ingress support, you should only enable --enable-envoy-config flag.

  1. $ helm upgrade cilium cilium/cilium --version 1.12.0 \
  2.     --namespace kube-system \
  3.     --reuse-values \
  4.     --set-string extraConfig.enable-envoy-config=true
  5. $ kubectl -n kube-system rollout restart deployment/cilium-operator
  6. $ kubectl -n kube-system rollout restart ds/cilium

Next you can check the status of the Cilium agent and operator:

  1. $ cilium status

Install the latest version of the Cilium CLI. The Cilium CLI can be used to install Cilium, inspect the state of a Cilium installation, and enable/disable various features (e.g. clustermesh, Hubble).

LinuxmacOSOther

  1. CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
  2. CLI_ARCH=amd64
  3. if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
  5. sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
  7. rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
  1. CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
  2. CLI_ARCH=amd64
  3. if [ "$(uname -m)" = "arm64" ]; then CLI_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-darwin-${CLI_ARCH}.tar.gz{,.sha256sum}
  5. shasum -a 256 -c cilium-darwin-${CLI_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC cilium-darwin-${CLI_ARCH}.tar.gz /usr/local/bin
  7. rm cilium-darwin-${CLI_ARCH}.tar.gz{,.sha256sum}

See the full page of releases.

Install the latest version of the Cilium CLI. The Cilium CLI can be used to install Cilium, inspect the state of a Cilium installation, and enable/disable various features (e.g. clustermesh, Hubble).

LinuxmacOSOther

  1. CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
  2. CLI_ARCH=amd64
  3. if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
  5. sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
  7. rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
  1. CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
  2. CLI_ARCH=amd64
  3. if [ "$(uname -m)" = "arm64" ]; then CLI_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-darwin-${CLI_ARCH}.tar.gz{,.sha256sum}
  5. shasum -a 256 -c cilium-darwin-${CLI_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC cilium-darwin-${CLI_ARCH}.tar.gz /usr/local/bin
  7. rm cilium-darwin-${CLI_ARCH}.tar.gz{,.sha256sum}

See the full page of releases.

Cilium Ingress Controller can be enabled with the below command

  1. $ cilium install \\
  2.     --kube-proxy-replacement=strict \\
  3.     --helm-set ingressController.enabled=true

If you only want to use envoy traffic management feature without Ingress support, you should only enable --enable-envoy-config flag.

  1. $ cilium install \\
  2.     --kube-proxy-replacement=strict \\
  3.     --helm-set-string extraConfig.enable-envoy-config=true

Next you can check the status of the Cilium agent and operator:

  1. $ cilium status

Hubble CLI is also used to observe the traffic in later steps.

LinuxMacOSWindows

Download the latest hubble release:

  1. export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
  2. HUBBLE_ARCH=amd64
  3. if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
  5. sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
  7. rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Download the latest hubble release:

  1. export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
  2. HUBBLE_ARCH=amd64
  3. if [ "$(uname -m)" = "arm64" ]; then HUBBLE_ARCH=arm64; fi
  4. curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-darwin-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
  5. shasum -a 256 -c hubble-darwin-${HUBBLE_ARCH}.tar.gz.sha256sum
  6. sudo tar xzvfC hubble-darwin-${HUBBLE_ARCH}.tar.gz /usr/local/bin
  7. rm hubble-darwin-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Download the latest hubble release:

  1. curl -LO "https://raw.githubusercontent.com/cilium/hubble/master/stable.txt"
  2. set /p HUBBLE_VERSION=<stable.txt
  3. curl -L --fail -O "https://github.com/cilium/hubble/releases/download/%HUBBLE_VERSION%/hubble-windows-amd64.tar.gz"
  4. curl -L --fail -O "https://github.com/cilium/hubble/releases/download/%HUBBLE_VERSION%/hubble-windows-amd64.tar.gz.sha256sum"
  5. certutil -hashfile hubble-windows-amd64.tar.gz SHA256
  6. type hubble-windows-amd64.tar.gz.sha256sum
  7. :: verify that the checksum from the two commands above match
  8. tar zxf hubble-windows-amd64.tar.gz

and move the hubble.exe CLI to a directory listed in the %PATH% environment variable after extracting it from the tarball.

Supported Envoy API Versions

As of now only the Envoy API v3 is supported.

Supported Envoy Extension Resource Types

Envoy extensions are resource types that may or may not be built in to an Envoy build. The standard types referred to in Envoy documentation, such as type.googleapis.com/envoy.config.listener.v3.Listener, and type.googleapis.com/envoy.config.route.v3.RouteConfiguration, are always available.

Cilium nodes deploy an Envoy image to support Cilium HTTP policy enforcement and observability. This build of Envoy has been optimized for the needs of the Cilium Agent and does not contain many of the Envoy extensions available in the Envoy code base.

To see which Envoy extensions are available, please have a look at the Envoy extensions configuration file. Only the extensions that have not been commented out with # are built in to the Cilium Envoy image. Currently this contains the following extensions:

  • envoy.clusters.dynamic_forward_proxy

  • envoy.filters.http.dynamic_forward_proxy

  • envoy.filters.http.ext_authz

  • envoy.filters.http.local_ratelimit

  • envoy.filters.http.ratelimit

  • envoy.filters.http.router

  • envoy.filters.http.set_metadata

  • envoy.filters.listener.tls_inspector

  • envoy.filters.network.connection_limit

  • envoy.filters.network.ext_authz

  • envoy.filters.network.http_connection_manager

  • envoy.filters.network.local_ratelimit

  • envoy.filters.network.mongo_proxy

  • envoy.filters.network.mysql_proxy

  • envoy.filters.network.ratelimit

  • envoy.filters.network.tcp_proxy

  • envoy.filters.network.sni_cluster

  • envoy.filters.network.sni_dynamic_forward_proxy

  • envoy.stat_sinks.metrics_service

  • envoy.transport_sockets.raw_buffer

  • envoy.upstreams.http.http

  • envoy.upstreams.http.tcp

We will evolve the list of built-in extensions based on user feedback.

Examples

Please refer to one of the below examples on how to use and leverage Cilium’s Ingress features: