System Requirements

Before installing Cilium, please ensure that your system meets the minimum requirements below. Most modern Linux distributions already do.

Summary

When running Cilium using the container image cilium/cilium, the host system must meet these requirements:

• Linux kernel >= 4.9.17

When running Cilium as a native process on your host (i.e. not running the cilium/cilium container image) these additional requirements must be met:

• clang+LLVM >= 10.0

• iproute2 with eBPF templating patches 1

When running Cilium without Kubernetes these additional requirements must be met:

• Key-Value store etcd >= 3.1.0

1(1,2)

Requires support for eBPF templating as documented below.

Linux Distribution Compatibility Matrix

The following table lists Linux distributions that are known to work well with Cilium.

2

CentOS 7 requires a third-party kernel provided by ElRepo whereas CentOS 8 ships with a supported kernel.

Note

The above list is based on feedback by users. If you find an unlisted Linux distribution that works well, please let us know by opening a GitHub issue or by creating a pull request that updates this guide.

Note

Systemd 245 and above (systemctl --version) overrides rp_filter setting of Cilium network interfaces. This introduces connectivity issues (see GitHub issue 10645 for details). To avoid that, configure rp_filter in systemd using the following commands:

echo 'net.ipv4.conf.lxc*.rp_filter = 0' > /etc/sysctl.d/99-override_cilium_rp_filter.conf
systemctl restart systemd-sysctl

Linux Kernel

Base Requirements

Cilium leverages and builds on the kernel eBPF functionality as well as various subsystems which integrate with eBPF. Therefore, host systems are required to run Linux kernel version 4.9.17 or later to run a Cilium agent. More recent kernels may provide additional eBPF functionality that Cilium will automatically detect and use on agent start.

In order for the eBPF feature to be enabled properly, the following kernel configuration options must be enabled. This is typically the case with distribution kernels. When an option can be built as a module or statically linked, either choice is valid.

CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_NET_CLS_BPF=y
CONFIG_BPF_JIT=y
CONFIG_NET_CLS_ACT=y
CONFIG_NET_SCH_INGRESS=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_BPF=y

Note

Users running Linux 4.10 or earlier with Cilium CIDR policies may face Restrictions on unique prefix lengths for CIDR policy rules.

Requirements for Iptables-based Masquerading

If you are not using BPF for masquerading (enable-bpf-masquerade=false, the default value), then you will need the following kernel configuration options.

CONFIG_NETFILTER_XT_SET=m
CONFIG_IP_SET=m
CONFIG_IP_SET_HASH_IP=m

Requirements for L7 and FQDN Policies

L7 proxy redirection currently uses TPROXY iptables actions as well as socket matches. For L7 redirection to work as intended kernel configuration must include the following modules:

CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_CT=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m

When xt_socket kernel module is missing the forwarding of redirected L7 traffic does not work in non-tunneled datapath modes. Since some notable kernels (e.g., COS) are shipping without xt_socket module, Cilium implements a fallback compatibility mode to allow L7 policies and visibility to be used with those kernels. Currently this fallback disables ip_early_demux kernel feature in non-tunneled datapath modes, which may decrease system networking performance. This guarantees HTTP and Kafka redirection works as intended. However, if HTTP or Kafka enforcement policies or visibility annotations are never used, this behavior can be turned off by adding the following to the helm configuration command line:

helm install cilium cilium/cilium --version 1.12.0 \
  ...
  --set enableXTSocketFallback=false

Requirements for IPsec

The IPsec Transparent Encryption feature requires a lot of kernel configuration options, most of which to enable the actual encryption. Note that the specific options required depend on the algorithm. The list below corresponds to requirements for GMC-128-AES.

CONFIG_XFRM=y
CONFIG_XFRM_OFFLOAD=y
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_ALGO=m
CONFIG_XFRM_USER=m
CONFIG_INET{,6}_ESP=m
CONFIG_INET{,6}_IPCOMP=m
CONFIG_INET{,6}_XFRM_TUNNEL=m
CONFIG_INET{,6}_TUNNEL=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_CRYPTO_AEAD=m
CONFIG_CRYPTO_AEAD2=m
CONFIG_CRYPTO_GCM=m
CONFIG_CRYPTO_SEQIV=m
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_HMAC=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_AES=m

Requirements for the Bandwidth Manager

The Bandwidth Manager requires the following kernel configuration option to change the packet scheduling algorithm.

CONFIG_NET_SCH_FQ=m

Required Kernel Versions for Advanced Features

Cilium requires Linux kernel 4.9.17 or higher; however, development on additional kernel features continues to progress in the Linux community. Some of Cilium’s features are dependent on newer kernel versions and are thus enabled by upgrading to more recent kernel versions as detailed below.

Key-Value store

Cilium optionally uses a distributed Key-Value store to manage, synchronize and distribute security identities across all cluster nodes. The following Key-Value stores are currently supported:

• etcd >= 3.1.0

Cilium can be used without a Key-Value store when CRD-based state management is used with Kubernetes. This is the default for new Cilium installations. Larger clusters will perform better with a Key-Value store backed identity management instead, see Quick Installation for more details.

See Key-Value Store for details on how to configure the cilium-agent to use a Key-Value store.

clang+LLVM

Note

This requirement is only needed if you run cilium-agent natively. If you are using the Cilium container image cilium/cilium, clang+LLVM is included in the container image.

LLVM is the compiler suite that Cilium uses to generate eBPF bytecode programs to be loaded into the Linux kernel. The minimum supported version of LLVM available to cilium-agent should be >=5.0. The version of clang installed must be compiled with the eBPF backend enabled.

See https://releases.llvm.org/ for information on how to download and install LLVM.

iproute2

Note

iproute2 is only needed if you run cilium-agent directly on the host machine. iproute2 is included in the cilium/cilium container image.

iproute2 is a low level tool used to configure various networking related subsystems of the Linux kernel. Cilium uses iproute2 to configure networking and tc, which is part of iproute2, to load eBPF programs into the kernel.

The version of iproute2 must include the eBPF templating patches. Also, it depends on Cilium’s libbpf fork. See Cilium iproute2 source for more details.

Firewall Rules

If you are running Cilium in an environment that requires firewall rules to enable connectivity, you will have to add the following rules to ensure Cilium works properly.

It is recommended but optional that all nodes running Cilium in a given cluster must be able to ping each other so cilium-health can report and monitor connectivity among nodes. This requires ICMP Type 0/8, Code 0 open among all nodes. TCP 4240 should also be open among all nodes for cilium-health monitoring. Note that it is also an option to only use one of these two methods to enable health monitoring. If the firewall does not permit either of these methods, Cilium will still operate fine but will not be able to provide health information.

For IPSec enabled Cilium deployments, you need to ensure that the firewall allows ESP traffic through. For example, AWS Security Groups doesn’t allow ESP traffic by default.

If you are using VXLAN overlay network mode, Cilium uses Linux’s default VXLAN port 8472 over UDP, unless Linux has been configured otherwise. In this case, UDP 8472 must be open among all nodes to enable VXLAN overlay mode. The same applies to Geneve overlay network mode, except the port is UDP 6081.

If you are running in direct routing mode, your network must allow routing of pod IPs.

As an example, if you are running on AWS with VXLAN overlay networking, here is a minimum set of AWS Security Group (SG) rules. It assumes a separation between the SG on the master nodes, master-sg, and the worker nodes, worker-sg. It also assumes etcd is running on the master nodes.

Master Nodes (master-sg) Rules:

Worker Nodes (worker-sg):

Note

If you use a shared SG for the masters and workers, you can condense these rules into ingress/egress to self. If you are using Direct Routing mode, you can condense all rules into ingress/egress ANY port/protocol to/from self.

The following ports should also be available on each node:

Mounted eBPF filesystem

Note

Some distributions mount the bpf filesystem automatically. Check if the bpf filesystem is mounted by running the command.

# mount | grep /sys/fs/bpf
$ # if present should output, e.g. "none on /sys/fs/bpf type bpf"...

If the eBPF filesystem is not mounted in the host filesystem, Cilium will automatically mount the filesystem.

Mounting this BPF filesystem allows the cilium-agent to persist eBPF resources across restarts of the agent so that the datapath can continue to operate while the agent is subsequently restarted or upgraded.

Optionally it is also possible to mount the eBPF filesystem before Cilium is deployed in the cluster, the following command must be run in the host mount namespace. The command must only be run once during the boot process of the machine.

# mount bpffs /sys/fs/bpf -t bpf

A portable way to achieve this with persistence is to add the following line to /etc/fstab and then run mount /sys/fs/bpf. This will cause the filesystem to be automatically mounted when the node boots.

bpffs                      /sys/fs/bpf             bpf     defaults 0 0

If you are using systemd to manage the kubelet, see the section Mounting BPFFS with systemd.

Privileges

The following privileges are required to run Cilium. When running the standard Kubernetes DaemonSet, the privileges are automatically granted to Cilium.

• Cilium interacts with the Linux kernel to install eBPF program which will then perform networking tasks and implement security rules. In order to install eBPF programs system-wide, CAP_SYS_ADMIN privileges are required. These privileges must be granted to cilium-agent.

  The quickest way to meet the requirement is to run cilium-agent as root and/or as privileged container.

• Cilium requires access to the host networking namespace. For this purpose, the Cilium pod is scheduled to run in the host networking namespace directly.