Self-managed Kubernetes in Amazon Web Services (AWS)

Big picture

Use Calico with a self-managed Kubernetes cluster in Amazon Web Services (AWS).

Value

Managing your own Kubernetes cluster (as opposed to using a managed-Kubernetes service like EKS), gives you the most flexibility in configuring Calico and Kubernetes. Calico combines flexible networking capabilities with “run-anywhere” security enforcement to provide a solution with native Linux kernel performance and true cloud-native scalability.

Concepts

Kubernetes Operations (kops) is a cluster management tool that handles provisioning cluster VMs and installing Kubernetes. It has built-in support for using Calico as the Kubernetes networking provider.

Before you begin…

Self-managed Kubernetes in Amazon Web Services (AWS) - 图1note

Calico makes use of the Kubernetes Container Storage Interface (CSI) to support various types of volumes. The necessary drivers required for CSI to function correctly in AWS clusters using EBS volumes may no longer be present by default in clusters running Kubernetes 1.23. Please check the documentation for the installer being used to ensure the necessary CSI drivers are installed.

If using Kubernetes Operations (kops) as further down on this page please use the relevant linked kops documentation to ensure your cluster has the necessary configuration.

How to

There are many ways to install and manage Kubernetes in AWS. Using Kubernetes Operations (kops) is a good default choice for most people, as it gives you access to all of Calico’s flexible and powerful networking features. However, there are other options that may work better for your environment.

Kubernetes Operations for Calico networking and network policy

To use kops to create a cluster with Calico networking and network policy:

  1. Install kops on your workstation.

  2. Set up your environment for AWS .

  3. Be sure to set up an S3 state store and export its name:

    1. export KOPS_STATE_STORE=s3://name-of-your-state-store-bucket

  4. Verify CSI driver installation configuration as per your particular cluster and volumes

  5. Configure kops to use Calico for networking. The easiest way to do this is to pass --networking calico to kops when creating the cluster. For example:

    1. kops create cluster \
    2.  --zones us-west-2a \
    3.  --networking calico \
    4.  name-of-your-cluster

    Or, you you can add calico to your cluster config. Run kops edit cluster and set the following networking configuration.

    1. networking:
    2.   calico: {}

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

You can further customize the Calico install with options listed in the kops documentation.

Other options and tools

Amazon VPC CNI plugin

As an alternative to Calico for both networking and network policy, you can use Amazon’s VPC CNI plugin for networking, and Calico for network policy. The advantage of this approach is that pods are assigned IP addresses associated with Elastic Network Interfaces on worker nodes. The IPs come from the VPC network pool and therefore do not require NAT to access resources outside the Kubernetes cluster.

Set your kops cluster configuration to:

  1. networking:
  2.   amazonvpc: {}

Then install Calico for network policy only after the cluster is up and ready.

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

Kubespray

Kubespray is a tool for provisioning and managing Kubernetes clusters with support for multiple clouds including Amazon Web Services. Calico is the default networking provider, or you can set the kube_network_plugin variable to calico. See the Kubespray docs for more details.

Next steps

Required

Recommended