Encrypt in-cluster pod traffic

Big picture

Enable WireGuard to secure on-the-wire, in-cluster pod traffic in a Calico cluster.

Value

When this feature is enabled, Calico automatically creates and manages WireGuard tunnels between nodes providing transport-level security for on-the-wire, in-cluster pod traffic. WireGuard provides formally verified secure and performant tunnels without any specialized hardware. For a deep dive in to WireGuard implementation, see this whitepaper.

Concepts

About WireGuard

WireGuard supports both host-to-host encryption for pod traffic and direct node-to-node communication. Because Calico is not implemented using a sidecar, traffic is not encrypted for the full journey from one pod to another; traffic is only encrypted on the host-to-host portion of the journey.

Calico supports WireGuard encryption for both IPv4 and IPv6 traffic. You can enable traffic independently using parameters in the FelixConfiguration resource:

  • wireguardEnabled - enables encrypting IPv4 traffic over an IPv4 underlay network
  • wireguardEnabledV6 - enables encrypting IPv6 traffic over an IPv6 underlay network

Features

This how-to guide uses the following Calico features:

  • Felix configuration resource with WireGuard configuration parameters

Before you begin…

Unsupported

  • GKE
  • Using your own custom keys to encrypt traffic

Limitations

  • IPv4 only
  • EKS, only with AWS CNI
  • AKS, only with Azure CNI

Supported encryption

  • Pod-to-pod traffic
  • Encryption for direct node-to-node communication is supported only on managed clusters deployed on EKS (AWS CNI) and AKS (Azure CNI)

Required

  • On all nodes in the cluster that you want to participate in Calico encryption, verify that the operating system(s) on the nodes are installed with WireGuard.

    Encrypt in-cluster pod traffic - 图1note

    Some node operating systems do not support Wireguard, or do not have it installed by default. Enabling Calico Wireguard encryption does not require all nodes to be installed with Wireguard. However, traffic to or from a node that does not have Wireguard installed will not be encrypted.

  • IP addresses for every node in the cluster. This is required to establish secure tunnels between the nodes. Calico can automatically do this using IP autodetection methods.

How to

Install WireGuard

WireGuard is included in Linux 5.6+ kernels, and has been backported to earlier Linux kernels in some Linux distributions.

Install WireGuard on cluster nodes using instructions for your operating system. Note that you may need to reboot your nodes after installing WireGuard to make the kernel modules available on your system.

Use the following instructions for these platforms that are not listed on the WireGuard installation page, before proceeding to enabling WireGuard.

  • EKS
  • AKS
  • OpenShift

To install WireGuard on the default Amazon Machine Image (AMI):

  1.    sudo yum install kernel-devel-`uname -r` -y
  2.    sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm -y
  3.    sudo curl -o /etc/yum.repos.d/jdoss-wireguard-epel-7.repo https://copr.fedorainfracloud.org/coprs/jdoss/wireguard/repo/epel-7/jdoss-wireguard-epel-7.repo
  4.    sudo yum install wireguard-dkms wireguard-tools -y

AKS cluster nodes run Ubuntu with a kernel that has WireGuard installed already, so there is no manual installation required.

To install WireGuard for OpenShift v4.8:

  1. Install requirements:

  2. Download and configure the tools needed for kmods.
  1. FAKEROOT=$(mktemp -d)
  2. git clone https://github.com/tigera/kmods-via-containers
  3. cd kmods-via-containers
  4. make install FAKEROOT=${FAKEROOT}
  5. cd ..
  6. git clone https://github.com/tigera/kvc-wireguard-kmod
  7. cd kvc-wireguard-kmod
  8. make install FAKEROOT=${FAKEROOT}
  9. cd ..

  1. Configure/edit ${FAKEROOT}/root/etc/kvc/wireguard-kmod.conf.

    a. You must then set the URLs for the KERNEL_CORE_RPM, KERNEL_DEVEL_RPM and KERNEL_MODULES_RPM packages in the conf file $FAKEROOT/etc/kvc/wireguard-kmod.conf. Obtain copies for kernel-core, kernel-devel, and kernel-modules rpms from RedHat Access and host it in an http file server that is reachable by your OCP workers.

    b. For help configuring kvc-wireguard-kmod/wireguard-kmod.conf and Wireguard version to kernel version compatibility, see the kvc-wireguard-kmod README file.

  2. Get RHEL Entitlement data from your own RHEL8 system from a host in your cluster.

    1. tar -czf subs.tar.gz /etc/pki/entitlement/ /etc/rhsm/ /etc/yum.repos.d/redhat.repo

    Please refer to Openshift documentation for more information about these entitlement files.

  3. Copy the subs.tar.gz file to your workspace and then extract the contents using the following command.

    1. tar -x -C ${FAKEROOT}/root -f subs.tar.gz

  4. Transpile your machine config using CoreOS Butane.

    1. cd kvc-wireguard-kmod
    2. make ignition FAKEROOT=${FAKEROOT} > mc-wg.yaml

  5. With the KUBECONFIG set for your cluster, run the following command to apply the MachineConfig which will install WireGuard across your cluster.

    1. oc create -f mc-wg.yaml

Enable WireGuard for a cluster

  • Operator
  • Manifest

Enable IPv4 WireGuard encryption across all the nodes using the following command.

  1. kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true}}'

Enable IPv6 WireGuard encryption across all the nodes using the following command.

  1. kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabledV6":true}}'

To enable both IPv4 and IPv6 WireGuard encryption across all the nodes, use the following command.

  1. kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true,"wireguardEnabledV6":true}}'

Enable IPv4 WireGuard encryption across all the nodes using the following command.

  1. calicoctl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true}}'

Enable IPv6 WireGuard encryption across all the nodes using the following command.

  1. calicoctl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabledV6":true}}'

To enable both IPv4 and IPv6 WireGuard encryption across all the nodes, use the following command.

  1. calicoctl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true,"wireguardEnabledV6":true}}'

Perform the next step for EKS and AKS clusters only, and only if your cluster is using the cloud provider CNI plugin and not Calico CNI. Enable WireGuard encryption for direct node-to-node communications using the following command.

  1. calicoctl patch felixconfiguration default --type='merge' -p '{"spec": {"wireguardHostEncryptionEnabled": true}}'

For OpenShift, add the Felix configuration with WireGuard enabled under custom resources.

Encrypt in-cluster pod traffic - 图2note

The above command can be used to change other WireGuard attributes. For a list of other WireGuard parameters and configuration evaluation, see the Felix configuration.

Encrypt in-cluster pod traffic - 图3note

natOutgoing: true is set for the default IPv4 IP pool, but not so for IPv6. Wireguard requires natOutgoing to be enabled in both IPv4 and IPv6, so enable NAT outgoing for the IPv6 IP pools when using IPv6 Wireguard.

We recommend that you review and modify the MTU used by Calico networking when WireGuard is enabled to increase network performance. Follow the instructions in the Configure MTU to maximize network performance guide to set the MTU to a value appropriate for your network.

Disable WireGuard for an individual node

To disable WireGuard on a specific node with WireGuard installed, modify the node-specific Felix configuration. e.g., to turn off encryption for pod traffic on node my-node, use the following command. This command disables WireGuard for both IPv4 and IPv6, modify it accordingly if disabling only either IP version:

  1. cat <<EOF | kubectl apply -f -
  2. apiVersion: projectcalico.org/v3
  3. kind: FelixConfiguration
  4. metadata:
  5.   name: node.my-node
  6. spec:
  7.   logSeverityScreen: Info
  8.   reportingInterval: 0s
  9.   wireguardEnabled: false
  10.   wireguardEnabledV6: false
  11. EOF

With the above command, Calico will not encrypt any of the pod traffic to or from node my-node.

To enable encryption for IPv4 and IPv6 pod traffic on node my-node again, patch this node’s FelixConfiguration (modify accordingly if only dealing with IPv4 or IPv6):

  1. calicoctl patch felixconfiguration node.my-node --type='merge' -p '{"spec":{"wireguardEnabled":true,"wireguardEnabledV6":true}}'

Verify configuration

To verify that the nodes are configured for WireGuard encryption, check the node status set by Felix using calicoctl. For example:

  1.    $ calicoctl get node <NODE-NAME> -o yaml
  2.    ...
  3.    status:
  4.      ...
  5.      wireguardPublicKey: jlkVyQYooZYzI2wFfNhSZez5eWh44yfq1wKVjLvSXgY=
  6.      wireguardPublicKeyV6: hTnWXGM4qk/Z8fQgyGFdpPd4qM9QGR2ey30s31yC6g4=
  7.      ...

Disable WireGuard for a cluster

To disable WireGuard on all nodes modify the default Felix configuration. For example:

  1.   calicoctl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":false,"wireguardEnabledV6":false}}'

Above and beyond