OpenStack Cloud Controller Manager reference guide

The OpenStack Cloud Controller Manager

In OKD 4.12, clusters that run on OpenStack are switched from the legacy OpenStack cloud provider to the external OpenStack Cloud Controller Manager (CCM). This change follows the move in Kubernetes from in-tree, legacy cloud providers to external cloud providers that are implemented by using the Cloud Controller Manager.

To preserve user-defined configurations for the legacy cloud provider, existing configurations are mapped to new ones as part of the migration process. It searches for a configuration called cloud-provider-config in the openshift-config namespace.

The config map name cloud-provider-config is not statically configured. It is derived from the spec.cloudConfig.name value in the infrastructure/cluster CRD.

Found configurations are synchronized to the cloud-conf config map in the openshift-cloud-controller-manager namespace.

As part of this synchronization, the OpenStack CCM Operator alters the new config map such that its properties are compatible with the external cloud provider. The file is changed in the following ways:

  • The [Global] secret-name, [Global] secret-namespace, and [Global] kubeconfig-path options are removed. They do not apply to the external cloud provider.

  • The [Global] use-clouds, [Global] clouds-file, and [Global] cloud options are added.

  • The entire [BlockStorage] section is removed. External cloud providers no longer perform storage operations. Block storage configuration is managed by the Cinder CSI driver.

Additionally, the CCM Operator enforces a number of default options. Values for these options are always overriden as follows:

  1. [Global]
  2. use-clouds = true
  3. clouds-file = /etc/openstack/secret/clouds.yaml
  4. cloud = openstack
  5. ...
  7. [LoadBalancer]
  8. use-octavia = true
  9. enabled = true (1)
1If the network is configured to use Kuryr, this value is false.

The clouds-value value, /etc/openstack/secret/clouds.yaml, is mapped to the openstack-cloud-credentials config in the openshift-cloud-controller-manager namespace. You can modify the OpenStack cloud in this file as you do any other clouds.yaml file.

The OpenStack Cloud Controller Manager (CCM) config map

An OpenStack CCM config map defines how your cluster interacts with your OpenStack cloud. By default, this configuration is stored under the cloud.conf key in the cloud-conf config map in the openshift-cloud-controller-manager namespace.

The cloud-conf config map is generated from the cloud-provider-config config map in the openshift-config namespace.

To change the settings that are described by the cloud-conf config map, modify the cloud-provider-config config map.

As part of this synchronization, the CCM Operator overrides some options. For more information, see “The OpenStack Cloud Controller Manager”.

For example:

An example cloud-conf config map

  1. apiVersion: v1
  2. data:
  3.   cloud.conf: |
  4.     [Global] (1)
  5.     secret-name = openstack-credentials
  6.     secret-namespace = kube-system
  7.     region = regionOne
  8.     [LoadBalancer]
  9.     use-octavia = True
  10. kind: ConfigMap
  11. metadata:
  12.   creationTimestamp: "2022-12-20T17:01:08Z"
  13.   name: cloud-conf
  14.   namespace: openshift-cloud-controller-manager
  15.   resourceVersion: "2519"
  16.   uid: cbbeedaf-41ed-41c2-9f37-4885732d3677
1Set global options by using a clouds.yaml file rather than modifying the config map.

The following options are present in the config map. Except when indicated otherwise, they are mandatory for clusters that run on OpenStack.

Load balancer options

CCM supports several load balancer options for deployments that use Octavia.

Neutron-LBaaS support is deprecated.

OptionDescription

enabled

Whether or not to enable the LoadBalancer type of services integration. The default value is true.

floating-network-id

Optional. The external network used to create floating IP addresses for load balancer virtual IP addresses (VIPs). If there are multiple external networks in the cloud, this option must be set or the user must specify loadbalancer.openstack.org/floating-network-id in the service annotation.

floating-subnet-id

Optional. The external network subnet used to create floating IP addresses for the load balancer VIP. Can be overridden by the service annotation loadbalancer.openstack.org/floating-subnet-id.

floating-subnet

Optional. A name pattern (glob or regular expression if starting with ~) for the external network subnet used to create floating IP addresses for the load balancer VIP. Can be overridden by the service annotation loadbalancer.openstack.org/floating-subnet. If multiple subnets match the pattern, the first one with available IP addresses is used.

floating-subnet-tags

Optional. Tags for the external network subnet used to create floating IP addresses for the load balancer VIP. Can be overridden by the service annotation loadbalancer.openstack.org/floating-subnet-tags. If multiple subnets match these tags, the first one with available IP addresses is used.

If the OpenStack network is configured with sharing disabled, for example, with the —no-share flag used during creation, this option is unsupported. Set the network to share to use this option.

lb-method

The load balancing algorithm used to create the load balancer pool. For the Amphora provider the value can be ROUND_ROBIN, LEAST_CONNECTIONS, or SOURCE_IP. The default value is ROUND_ROBIN.

For the OVN provider, only the SOURCE_IP_PORT algorithm is supported.

For the Amphora provider, if using the LEAST_CONNECTIONS or SOURCE_IP methods, configure the create-monitor option as true in the cloud-provider-config config map on the openshift-config namespace and ETP:Local on the load-balancer type service to allow balancing algorithm enforcement in the client to service endpoint connections.

lb-provider

Optional. Used to specify the provider of the load balancer, for example, amphora or octavia. Only the Amphora and Octavia providers are supported.

lb-version

Optional. The load balancer API version. Only “v2” is supported.

subnet-id

The ID of the Networking service subnet on which load balancer VIPs are created.

network-id

The ID of the Networking service network on which load balancer VIPs are created. Unnecessary if subnet-id is set.

create-monitor

Whether or not to create a health monitor for the service load balancer. A health monitor is required for services that declare externalTrafficPolicy: Local. The default value is false.

This option is unsupported if you use OpenStack earlier than version 17 with the ovn provider.

monitor-delay

The interval in seconds by which probes are sent to members of the load balancer. The default value is 5.

monitor-max-retries

The number of successful checks that are required to change the operating status of a load balancer member to ONLINE. The valid range is 1 to 10, and the default value is 1.

monitor-timeout

The time in seconds that a monitor waits to connect to the back end before it times out. The default value is 3.

internal-lb

Whether or not to create an internal load balancer without floating IP addresses. The default value is false.

LoadBalancerClass “ClassName”

This is a config section that comprises a set of options:

  • floating-network-id

  • floating-subnet-id

  • floating-subnet

  • floating-subnet-tags

  • network-id

  • subnet-id

The behavior of these options is the same as that of the identically named options in the load balancer section of the CCM config file.

You can set the ClassName value by specifying the service annotation loadbalancer.openstack.org/class.

max-shared-lb

The maximum number of services that can share a load balancer. The default value is 2.

Options that the Operator overrides

The CCM Operator overrides the following options, which you might recognize from configuring OpenStack. Do not configure them yourself. They are included in this document for informational purposes only.

OptionDescription

auth-url

The OpenStack Identity service URL. For example, http://128.110.154.166/identity.

os-endpoint-type

The type of endpoint to use from the service catalog.

username

The Identity service user name.

password

The Identity service user password.

domain-id

The Identity service user domain ID.

domain-name

The Identity service user domain name.

tenant-id

The Identity service project ID. Leave this option unset if you are using Identity service application credentials.

In version 3 of the Identity API, which changed the identifier tenant to project, the value of tenant-id is automatically mapped to the project construct in the API.

tenant-name

The Identity service project name.

tenant-domain-id

The Identity service project domain ID.

tenant-domain-name

The Identity service project domain name.

user-domain-id

The Identity service user domain ID.

user-domain-name

The Identity service user domain name.

use-clouds

Whether or not to fetch authorization credentials from a clouds.yaml file. Options set in this section are prioritized over values read from the clouds.yaml file.

CCM searches for the file in the following places:

  1. The value of the clouds-file option.

  2. A file path stored in the environment variable OS_CLIENT_CONFIG_FILE.

  3. The directory pkg/openstack.

  4. The directory ~/.config/openstack.

  5. The directory /etc/openstack.

clouds-file

The file path of a clouds.yaml file. It is used if the use-clouds option is set to true.

cloud

The named cloud in the clouds.yaml file that you want to use. It is used if the use-clouds option is set to true.