Cloud Controller Manager

FEATURE STATE: Kubernetes v1.11 [beta]

Cloud infrastructure technologies let you run Kubernetes on public, private, and hybrid clouds. Kubernetes believes in automated, API-driven infrastructure without tight coupling between components.

The cloud-controller-manager is a Kubernetes control plane component that embeds cloud-specific control logic. The cloud controller manager lets you link your cluster into your cloud provider’s API, and separates out the components that interact with that cloud platform from components that only interact with your cluster.

By decoupling the interoperability logic between Kubernetes and the underlying cloud infrastructure, the cloud-controller-manager component enables cloud providers to release features at a different pace compared to the main Kubernetes project.

The cloud-controller-manager is structured using a plugin mechanism that allows different cloud providers to integrate their platforms with Kubernetes.

Design

Kubernetes components

The cloud controller manager runs in the control plane as a replicated set of processes (usually, these are containers in Pods). Each cloud-controller-manager implements multiple controllers in a single process.

Note: You can also run the cloud controller manager as a Kubernetes addon rather than as part of the control plane.

Cloud controller manager functions

The controllers inside the cloud controller manager include:

Node controller

The node controller is responsible for updating Node objects when new servers are created in your cloud infrastructure. The node controller obtains information about the hosts running inside your tenancy with the cloud provider. The node controller performs the following functions:

  1. Update a Node object with the corresponding server’s unique identifier obtained from the cloud provider API.
  2. Annotating and labelling the Node object with cloud-specific information, such as the region the node is deployed into and the resources (CPU, memory, etc) that it has available.
  3. Obtain the node’s hostname and network addresses.
  4. Verifying the node’s health. In case a node becomes unresponsive, this controller checks with your cloud provider’s API to see if the server has been deactivated / deleted / terminated. If the node has been deleted from the cloud, the controller deletes the Node object from your Kubernetes cluster.

Some cloud provider implementations split this into a node controller and a separate node lifecycle controller.

Route controller

The route controller is responsible for configuring routes in the cloud appropriately so that containers on different nodes in your Kubernetes cluster can communicate with each other.

Depending on the cloud provider, the route controller might also allocate blocks of IP addresses for the Pod network.

Service controller

Services integrate with cloud infrastructure components such as managed load balancers, IP addresses, network packet filtering, and target health checking. The service controller interacts with your cloud provider’s APIs to set up load balancers and other infrastructure components when you declare a Service resource that requires them.

Authorization

This section breaks down the access that the cloud controller managers requires on various API objects, in order to perform its operations.

Node controller

The Node controller only works with Node objects. It requires full access to read and modify Node objects.

v1/Node:

  • Get
  • List
  • Create
  • Update
  • Patch
  • Watch
  • Delete

Route controller

The route controller listens to Node object creation and configures routes appropriately. It requires Get access to Node objects.

v1/Node:

  • Get

Service controller

The service controller listens to Service object Create, Update and Delete events and then configures Endpoints for those Services appropriately.

To access Services, it requires List, and Watch access. To update Services, it requires Patch and Update access.

To set up Endpoints resources for the Services, it requires access to Create, List, Get, Watch, and Update.

v1/Service:

  • List
  • Get
  • Watch
  • Patch
  • Update

Others

The implementation of the core of the cloud controller manager requires access to create Event objects, and to ensure secure operation, it requires access to create ServiceAccounts.

v1/Event:

  • Create
  • Patch
  • Update

v1/ServiceAccount:

  • Create

The RBAC ClusterRole for the cloud controller manager looks like:

  1. apiVersion: rbac.authorization.k8s.io/v1
  2. kind: ClusterRole
  3. metadata:
  4.   name: cloud-controller-manager
  5. rules:
  6. - apiGroups:
  7.   - ""
  8.   resources:
  9.   - events
  10.   verbs:
  11.   - create
  12.   - patch
  13.   - update
  14. - apiGroups:
  15.   - ""
  16.   resources:
  17.   - nodes
  18.   verbs:
  19.   - '*'
  20. - apiGroups:
  21.   - ""
  22.   resources:
  23.   - nodes/status
  24.   verbs:
  25.   - patch
  26. - apiGroups:
  27.   - ""
  28.   resources:
  29.   - services
  30.   verbs:
  31.   - list
  32.   - patch
  33.   - update
  34.   - watch
  35. - apiGroups:
  36.   - ""
  37.   resources:
  38.   - serviceaccounts
  39.   verbs:
  40.   - create
  41. - apiGroups:
  42.   - ""
  43.   resources:
  44.   - persistentvolumes
  45.   verbs:
  46.   - get
  47.   - list
  48.   - update
  49.   - watch
  50. - apiGroups:
  51.   - ""
  52.   resources:
  53.   - endpoints
  54.   verbs:
  55.   - create
  56.   - get
  57.   - list
  58.   - watch
  59.   - update

What’s next

Cloud Controller Manager Administration has instructions on running and managing the cloud controller manager.

To upgrade a HA control plane to use the cloud controller manager, see Migrate Replicated Control Plane To Use Cloud Controller Manager.

Want to know how to implement your own cloud controller manager, or extend an existing project?

The cloud controller manager uses Go interfaces to allow implementations from any cloud to be plugged in. Specifically, it uses the CloudProvider interface defined in cloud.go from kubernetes/cloud-provider.

The implementation of the shared controllers highlighted in this document (Node, Route, and Service), and some scaffolding along with the shared cloudprovider interface, is part of the Kubernetes core. Implementations specific to cloud providers are outside the core of Kubernetes and implement the CloudProvider interface.

For more information about developing plugins, see Developing Cloud Controller Manager.