Self-managed Kubernetes in Microsoft Azure

Big picture

Use Calico with a self-managed Kubernetes cluster in Microsoft Azure.

Value

Managing your own Kubernetes cluster (as opposed to using a managed-Kubernetes service like AKS), gives you the most flexibility in configuring Calico and Kubernetes. Calico combines flexible networking capabilities with “run-anywhere” security enforcement to provide a solution with native Linux kernel performance and true cloud-native scalability.

Concepts

aks-engine is an open-source tool for creating and managing Kubernetes clusters in Microsoft Azure. It is the core technology for Microsoft’s Azure Kubernetes Service (AKS), but allows you to manage the cluster yourself.

Before you begin…

How to

There are many ways to install and manage Kubernetes in Azure. This guide shows how to use aks-engine to deploy a cluster with Azure’s CNI plugin for networking and Calico for network policy enforcement. The advantage of this approach is that pods are assigned IP addresses associated with Azure Network Interfaces on worker nodes. The IPs come from the VNET network pool and therefore do not require NAT to access resources outside the Kubernetes cluster. However, there are other options that may work better for your environment.

aks-engine for Azure networking and Calico network policy

Install aks-engine on your workstation.

Before deploying, customize your cluster definition to use Calico for network policy. Add or modify the kubernetesConfig section to include the following (see the aks-engine documentation for other Kubernetes configuration settings).

  1. "kubernetesConfig": {
  2.    "networkPlugin": "azure",
  3.    "networkPolicy": "calico"
  4.  }

Or, start with this example cluster definition with these value already set, and customize to meet your needs.

Then, follow the ask-engine documentation to deploy your cluster, passing your cluster definition to aks-engine deploy via the -m flag.

The geeky details of what you get:

PolicyIPAMCNIOverlayRoutingDatastore

Other options and tools

Calico networking

You can also deploy Calico for both networking and policy enforcement. In this mode, Calico uses a VXLAN-based overlay network that masks the IP addresses of the pods from the underlying Azure VNET. This can be useful in large deployments or when running multiple clusters and IP address space is a big concern.

Unfortunately, aks-engine does not support this mode, so you must use a different tool chain to install and manage the cluster. Some options:

Terraform

Terraform is a tool for automating infrastructure provisioning using declarative configurations. You can also go as far as automating the install of Docker, kubeadm, and Kubernetes using Terraform “provisioners.” See the Terraform documentation for more details.

kubeadm

kubeadm is a command line tool for bootstrapping a Kubernetes cluster on top of already-provisioned compute resources, like VMs in a cloud or bare metal hosts. Unlike aks-engine which handles provisioning cloud resources, installing Kubernetes, and installing Calico, kubeadm only handles the second step of installing Kubernetes. You should proceed to install Calico after completing kubeadm install.

Kubespray

Kubespray is a tool for provisioning and managing Kubernetes clusters with support for multiple clouds including Azure. Calico is the default networking provider, or you can set the kube_network_plugin variable to calico. See the Kubespray docs for more details.

Next steps

Required

Recommended