WireGuard Transparent Encryption

This guide explains how to configure Cilium with transparent encryption of traffic between Cilium-managed endpoints using WireGuard®.

When WireGuard is enabled in Cilium, the agent running on each cluster node will establish a secure WireGuard tunnel between it and all other known nodes in the cluster. Each node automatically creates its own encryption key-pair and distributes its public key via the io.cilium.network.wg-pub-key annotation in the Kubernetes CiliumNode custom resource object. Each node’s public key is then used by other nodes to decrypt and encrypt traffic from and to Cilium-managed endpoints running on that node.

Packets are not encrypted when they are destined to the same node from which they were sent. This behavior is intended. Encryption would provide no benefits in that case, given that the raw traffic can be observed on the node anyway.

The WireGuard tunnel endpoint is exposed on UDP port 51871 on each node. If you run Cilium in an environment that requires firewall rules to enable connectivity, you will have to ensure that all Cilium cluster nodes can reach each other via that port.

Note

When running in the tunneling mode (i.e. with VXLAN or Geneve), pod to pod traffic will be sent only over the WireGuard tunnel which means that the packets will bypass the other tunnel, and thus they will be encapsulated only once.

Enable WireGuard in Cilium

Before you enable WireGuard in Cilium, please ensure that the Linux distribution running on your cluster nodes has support for WireGuard in kernel mode (i.e. CONFIG_WIREGUARD=m on Linux 5.6 and newer, or via the out-of-tree WireGuard module on older kernels).

Cilium CLI

Helm

If you are deploying Cilium with the Cilium CLI, pass the following options:

  1. cilium install --encryption wireguard

If you are deploying Cilium with Helm by following Installation using Helm, pass the following options:

  1. helm install cilium cilium/cilium --version 1.10.2 \
  2.   --namespace kube-system \
  3.   --set l7Proxy=false \
  4.   --set encryption.enabled=true \
  5.   --set encryption.type=wireguard

WireGuard may also be enabled manually by setting setting the enable-wireguard: true option in the Cilium ConfigMap and restarting each Cilium agent instance.

Validate the Setup

Run a bash shell in one of the Cilium pods with kubectl -n kube-system exec -ti ds/cilium -- bash and execute the following commands:

  1. Check that WireGuard has been enabled (number of peers should correspond to a number of nodes subtracted by one):

    1. cilium status | grep Encryption
    3. Encryption: Wireguard [cilium_wg0 (Pubkey: <..>, Port: 51871, Peers: 2)]

  2. Install tcpdump

    1. apt-get update
    2. apt-get -y install tcpdump

  3. Check that traffic is sent via the cilium_wg0 tunnel device:

    1. tcpdump -n -i cilium_wg0
    3. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    4. listening on cilium_wg0, link-type RAW (Raw IP), capture size 262144 bytes
    5. 15:05:24.643427 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [S], seq 476474887, win 64860, options [mss 1410,sackOK,TS val 648097391 ecr 0,nop,wscale 7], length 0
    6. 15:05:24.644185 IP 10.244.3.78.8080 > 10.244.1.35.51116: Flags [S.], seq 4032860634, ack 476474888, win 64308, options [mss 1410,sackOK,TS val 4004186138 ecr 648097391,nop,wscale 7], length 0
    7. 15:05:24.644238 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [.], ack 1, win 507, options [nop,nop,TS val 648097391 ecr 4004186138], length 0
    8. 15:05:24.644277 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [P.], seq 1:81, ack 1, win 507, options [nop,nop,TS val 648097392 ecr 4004186138], length 80: HTTP: GET / HTTP/1.1
    9. 15:05:24.644370 IP 10.244.3.78.8080 > 10.244.1.35.51116: Flags [.], ack 81, win 502, options [nop,nop,TS val 4004186139 ecr 648097392], length 0
    10. 15:05:24.645536 IP 10.244.3.78.8080 > 10.244.1.35.51116: Flags [.], seq 1:1369, ack 81, win 502, options [nop,nop,TS val 4004186140 ecr 648097392], length 1368: HTTP: HTTP/1.1 200 OK
    11. 15:05:24.645569 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [.], ack 1369, win 502, options [nop,nop,TS val 648097393 ecr 4004186140], length 0
    12. 15:05:24.645578 IP 10.244.3.78.8080 > 10.244.1.35.51116: Flags [P.], seq 1369:2422, ack 81, win 502, options [nop,nop,TS val 4004186140 ecr 648097392], length 1053: HTTP
    13. 15:05:24.645644 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [.], ack 2422, win 494, options [nop,nop,TS val 648097393 ecr 4004186140], length 0
    14. 15:05:24.645752 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [F.], seq 81, ack 2422, win 502, options [nop,nop,TS val 648097393 ecr 4004186140], length 0
    15. 15:05:24.646431 IP 10.244.3.78.8080 > 10.244.1.35.51116: Flags [F.], seq 2422, ack 82, win 502, options [nop,nop,TS val 4004186141 ecr 648097393], length 0
    16. 15:05:24.646484 IP 10.244.1.35.51116 > 10.244.3.78.8080: Flags [.], ack 2423, win 502, options [nop,nop,TS val 648097394 ecr 4004186141], length 0

Troubleshooting

When troubleshooting dropped or unencrypted packets between pods, the following commands can be helpful:

  1. # From node A:
  2. cilium debuginfo --output json | jq .encryption
  3. {
  4.   "wireguard": {
  5.     "interfaces": [
  6.       {
  7.         "listen-port": 51871,
  8.         "name": "cilium_wg0",
  9.         "peer-count": 1,
  10.         "peers": [
  11.           {
  12.             "allowed-ips": [
  13.               "10.154.1.107/32",
  14.               "10.154.1.195/32"
  15.             ],
  16.             "endpoint": "192.168.34.12:51871",
  17.             "last-handshake-time": "2021-05-05T12:31:24.418Z",
  18.             "public-key": "RcYfs/GEkcnnv6moK5A1pKnd+YYUue21jO9I08Bv0zo="
  19.           }
  20.         ],
  21.         "public-key": "DrAc2EloK45yqAcjhxerQKwoYUbLDjyrWgt9UXImbEY="
  22.       }
  23.     ]
  24.   }
  25. }
  26. # From node B:
  27. cilium debuginfo --output json | jq .encryption
  28. {
  29.   "wireguard": {
  30.     "interfaces": [
  31.       {
  32.         "listen-port": 51871,
  33.         "name": "cilium_wg0",
  34.         "peer-count": 1,
  35.         "peers": [
  36.           {
  37.             "allowed-ips": [
  38.               "10.154.2.103/32",
  39.               "10.154.2.142/32"
  40.             ],
  41.             "endpoint": "192.168.34.11:51871",
  42.             "last-handshake-time": "2021-05-05T12:31:24.631Z",
  43.             "public-key": "DrAc2EloK45yqAcjhxerQKwoYUbLDjyrWgt9UXImbEY="
  44.           }
  45.         ],
  46.         "public-key": "RcYfs/GEkcnnv6moK5A1pKnd+YYUue21jO9I08Bv0zo="
  47.       }
  48.     ]
  49.   }
  50. }

For pod to pod packets to be successfully encrypted and decrypted, the following must hold:

  • WireGuard public key of a remote node in the peers[*].public-key section matches the actual public key of the remote node (public-key retrieved via the same command on the remote node).
  • peers[*].allowed-ips should contain a list of pod IP addresses running on the remote.

Limitations

WireGuard support in Cilium currently lacks the following features, which may be resolved in upcoming Cilium releases:

  • Host-level encryption. Only traffic between two Cilium-managed endpoints (i.e. pod to pod traffic) is encrypted. Traffic between a Cilium-managed pod and a remote host, or traffic between two hosts running the Cilium agent will currently not be encrypted.
  • L7 policy enforcement and visibility
  • eBPF-based host routing
  • Support for older kernels via user-mode WireGuard

The current status of these limitations is tracked in GitHub issue 15462.

Legal

“WireGuard” is a registered trademark of Jason A. Donenfeld.