Install Calico networking and network policy for on-premises deployments

Big picture

Install Calico to provide both networking and network policy for self-managed on-premises deployments.

Value

Calico networking and network policy are a powerful choice for a CaaS implementation. If you have the networking infrastructure and resources to manage Kubernetes on-premises, installing the full Calico product provides the most customization and control.

Concepts

Calico is installed by an operator which manages the installation, upgrade, and general lifecycle of a Calico cluster. The operator is installed directly on the cluster as a Deployment, and is configured through one or more custom Kubernetes API resources.

Calico manifests

Calico can also be installed using raw manifests as an alternative to the operator. The manifests contain the necessary resources for installing Calico on each node in your Kubernetes cluster. Using manifests is not recommended as they cannot automatically manage the lifecycle of the Calico as the operator does. However, manifests may be useful for clusters that require highly specific modifications to the underlying Kubernetes resources.

Before you begin…

How to

Install Calico

  • Operator
  • Manifest

  1. Install the operator on your cluster.

    1. kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.4/manifests/tigera-operator.yaml

  2. Download the custom resources necessary to configure Calico.

    1. curl https://raw.githubusercontent.com/projectcalico/calico/v3.27.4/manifests/custom-resources.yaml -O

    If you wish to customize the Calico install, customize the downloaded custom-resources.yaml manifest locally.

  3. Create the manifest to install Calico.

    1. kubectl create -f custom-resources.yaml

  4. Verify Calico installation in your cluster.

    1. watch kubectl get pods -n calico-system

    You should see a result similar to the below.

    1. NAMESPACE     NAME                READY   STATUS                  RESTARTS         AGE
    2. kube-system   calico-node-txngh   1/1     Running                   0              54s
PolicyIPAMCNIOverlayRoutingDatastore

Based on your datastore and number of nodes, select a link below to install Calico.

Install Calico networking and network policy for on-premises deployments - 图1note

The option, Kubernetes API datastore, more than 50 nodes provides scaling using Typha daemon. Typha is not included for etcd because etcd already handles many clients so using Typha is redundant and not recommended.

Install Calico with Kubernetes API datastore, 50 nodes or less

Install Calico networking and network policy for on-premises deployments - 图2note

This option is maintained for upgrade compatibility, but we recommend that new clusters use the operator, which will automatically configure Calico correctly for your cluster size (including deploying the Typha scale-out proxy and securing it when necessary).

  1. Download the Calico networking manifest for the Kubernetes API datastore.

    1. curl https://raw.githubusercontent.com/projectcalico/calico/v3.27.4/manifests/calico.yaml -O

  2. If you are using pod CIDR 192.168.0.0/16, skip to the next step. If you are using a different pod CIDR with kubeadm, no changes are required — Calico will automatically detect the CIDR based on the running configuration. For other platforms, make sure you uncomment the CALICO_IPV4POOL_CIDR variable in the manifest and set it to the same value as your chosen pod CIDR.

  3. Customize the manifest as necessary.

  4. Apply the manifest using the following command.

    1. kubectl apply -f calico.yaml

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

Install Calico with Kubernetes API datastore, more than 50 nodes

Install Calico networking and network policy for on-premises deployments - 图3note

This option is maintained for upgrade compatibility but we recommend that new clusters use the operator to deploy Typha (the extra scale-out component included in this manifest) instead of using this method. The operator deploys Typha, autoscales it, and auto-configures mTLS between the per-host agent and Typha for maximum security. This manifest option leaves scaling up to you and, by default, it does not secure Typha’s port.

  1. Download the Calico networking manifest for the Kubernetes API datastore.

    1. curl https://raw.githubusercontent.com/projectcalico/calico/v3.27.4/manifests/calico-typha.yaml -o calico.yaml

  2. If you are using pod CIDR 192.168.0.0/16, skip to the next step. If you are using a different pod CIDR with kubeadm, no changes are required — Calico will automatically detect the CIDR based on the running configuration. For other platforms, make sure you uncomment the CALICO_IPV4POOL_CIDR variable in the manifest and set it to the same value as your chosen pod CIDR.

  3. Modify the replica count to the desired number in the Deployment named, calico-typha.

    1. apiVersion: apps/v1beta1
    2. kind: Deployment
    3. metadata:
    4.   name: calico-typha
    5.   ...
    6. spec:
    7.   ...
    8.   replicas: <number of replicas>

    We recommend at least one replica for every 200 nodes, and no more than 20 replicas. In production, we recommend a minimum of three replicas to reduce the impact of rolling upgrades and failures. The number of replicas should always be less than the number of nodes, otherwise rolling upgrades will stall. In addition, Typha only helps with scale if there are fewer Typha instances than there are nodes.

    Install Calico networking and network policy for on-premises deployments - 图4note

    If you set typha_service_name and set the Typha deployment replica count to 0, Felix will not start.

  4. Customize the manifest if desired.

  5. Apply the manifest.

    1. kubectl apply -f calico.yaml

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

Install Calico with etcd datastore

Install Calico networking and network policy for on-premises deployments - 图5note

The etcd datastore is not recommended for new Kubernetes installs:

  • etcd is another component to manage and maintain.
  • Some newer Kubernetes-targeted features (such as service matches in policy) are not supported with the etcd datastore.
  • eBPF dataplane mode is not supported with the etcd datastore. It relies on watching services to implement some features, which is not supported with the etcd datastore.
  • The Cloud/Enterprise versions of Calico do not support etcd mode at all so using this manifest prevents you from upgrading.

However, it is the only option that supports running both OpenStack and Kubernetes nodes in the same cluster.

  1. Download the Calico networking manifest for etcd.

    1. curl https://raw.githubusercontent.com/projectcalico/calico/v3.27.4/manifests/calico-etcd.yaml -o calico.yaml

  2. If you are using pod CIDR 192.168.0.0/16, skip to the next step. If you are using a different pod CIDR with kubeadm, no changes are required — Calico will automatically detect the CIDR based on the running configuration. For other platforms, make sure you uncomment the CALICO_IPV4POOL_CIDR variable in the manifest and set it to the same value as your chosen pod CIDR.

  3. In the ConfigMap named, calico-config, set the value of etcd_endpoints to the IP address and port of your etcd server.

    Install Calico networking and network policy for on-premises deployments - 图6note

    You can specify more than one etcd_endpoint using commas as delimiters.

  4. Customize the manifest if desired.

  5. Apply the manifest using the following command.

    1. kubectl apply -f calico.yaml

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

Next steps

Required

Recommended - Networking

  • If you are using the default BGP networking with full-mesh node-to-node peering with no encapsulation, go to Configure BGP peering to get traffic flowing between pods.
  • If you are unsure about networking options, or want to implement encapsulation (overlay networking), see Determine best networking option.

Recommended - Security