About the Cloud Credential Operator

About the Cloud Credential Operator

The Cloud Credential Operator (CCO) manages cloud provider credentials as custom resource definitions (CRDs). The CCO syncs on CredentialsRequest custom resources (CRs) to allow OKD components to request cloud provider credentials with the specific permissions that are required for the cluster to run.

By setting different values for the credentialsMode parameter in the install-config.yaml file, the CCO can be configured to operate in several different modes. If no mode is specified, or the credentialsMode parameter is set to an empty string (""), the CCO operates in its default mode.

Modes

By setting different values for the credentialsMode parameter in the install-config.yaml file, the CCO can be configured to operate in mint, passthrough, or manual mode. These options provide transparency and flexibility in how the CCO uses cloud credentials to process CredentialsRequest CRs in the cluster, and allow the CCO to be configured to suit the security requirements of your organization. Not all CCO modes are supported for all cloud providers.

Mint: In mint mode, the CCO uses the provided admin-level cloud credential to create new credentials for components in the cluster with only the specific permissions that are required.
Passthrough: In passthrough mode, the CCO passes the provided cloud credential to the components that request cloud credentials.
Manual: In manual mode, a user manages cloud credentials instead of the CCO.
- Manual with AWS Security Token Service: In manual mode, you can configure an AWS cluster to use Amazon Web Services Security Token Service (AWS STS). With this configuration, the CCO uses temporary credentials for different components.
- Manual with GCP Workload Identity: In manual mode, you can configure a GCP cluster to use GCP Workload Identity. With this configuration, the CCO uses temporary credentials for different components.

Table 1. CCO mode support matrix
Cloud provider	Mint	Passthrough	Manual
Alibaba Cloud			X
Amazon Web Services (AWS)	X	X	X
Microsoft Azure		X ^[1]	X
Google Cloud Platform (GCP)	X	X	X
IBM Cloud			X
OpenStack		X
oVirt		X
VMware vSphere		X

Manual mode is the only supported CCO configuration for Microsoft Azure Stack Hub.

Determining the Cloud Credential Operator mode

For platforms that support using the CCO in multiple modes, you can determine what mode the CCO is configured to use by using the web console or the CLI.

Determining the Cloud Credential Operator mode by using the web console

You can determine what mode the Cloud Credential Operator (CCO) is configured to use by using the web console.

Only Amazon Web Services (AWS), global Microsoft Azure, and Google Cloud Platform (GCP) clusters support multiple CCO modes.

Prerequisites

You have access to an OKD account with cluster administrator permissions.

Procedure

Log in to the OKD web console as a user with the cluster-admin role.
Navigate to Administration → Cluster Settings.
On the Cluster Settings page, select the Configuration tab.
Under Configuration resource, select CloudCredential.
On the CloudCredential details page, select the YAML tab.

In the YAML block, check the value of spec.credentialsMode. The following values are possible, though not all are supported on all platforms:

'': The CCO is operating in the default mode. In this configuration, the CCO operates in mint or passthrough mode, depending on the credentials provided during installation.
Mint: The CCO is operating in mint mode.
Passthrough: The CCO is operating in passthrough mode.
Manual: The CCO is operating in manual mode.

To determine the specific configuration of an AWS or GCP cluster that has a spec.credentialsMode of ‘’, Mint, or Manual, you must investigate further.

AWS and GCP clusters support using mint mode with the root secret deleted.

An AWS or GCP cluster that uses manual mode might be configured to create and manage cloud credentials from outside of the cluster using the AWS Security Token Service (STS) or GCP Workload Identity. You can determine whether your cluster uses this strategy by examining the cluster Authentication object.

AWS or GCP clusters that use the default ('') only: To determine whether the cluster is operating in mint or passthrough mode, inspect the annotations on the cluster root secret:
1. Navigate to Workloads → Secrets and look for the root secret for your cloud provider.
  
  Ensure that the Project dropdown is set to All Projects.
  
  Platform Secret name
  AWS
  aws-creds
  GCP
  gcp-credentials
2. To view the CCO mode that the cluster is using, click 1 annotation under Annotations, and check the value field. The following values are possible:
  - Mint: The CCO is operating in mint mode.
  - Passthrough: The CCO is operating in passthrough mode.
  If your cluster uses mint mode, you can also determine whether the cluster is operating without the root secret.
AWS or GCP clusters that use mint mode only: To determine whether the cluster is operating without the root secret, navigate to Workloads → Secrets and look for the root secret for your cloud provider.

Ensure that the Project dropdown is set to All Projects.

Platform Secret name
AWS
aws-creds
GCP
gcp-credentials
- If you see one of these values, your cluster is using mint or passthrough mode with the root secret present.
- If you do not see these values, your cluster is using the CCO in mint mode with the root secret removed.
AWS or GCP clusters that use manual mode only: To determine whether the cluster is configured to create and manage cloud credentials from outside of the cluster, you must check the cluster Authentication object YAML values.
1. Navigate to Administration → Cluster Settings.
2. On the Cluster Settings page, select the Configuration tab.
3. Under Configuration resource, select Authentication.
4. On the Authentication details page, select the YAML tab.
5. In the YAML block, check the value of the .spec.serviceAccountIssuer parameter.
  - A value that contains a URL that is associated with your cloud provider indicates that the CCO is using manual mode with AWS STS or GCP Workload Identity to create and manage cloud credentials from outside of the cluster. These clusters are configured using the ccoctl utility.
  - An empty value ('') indicates that the cluster is using the CCO in manual mode but was not configured using the ccoctl utility.

Platform	Secret name
AWS	`aws-creds`
GCP	`gcp-credentials`

Platform	Secret name
AWS	`aws-creds`
GCP	`gcp-credentials`

Determining the Cloud Credential Operator mode by using the CLI

You can determine what mode the Cloud Credential Operator (CCO) is configured to use by using the CLI.

Only Amazon Web Services (AWS), global Microsoft Azure, and Google Cloud Platform (GCP) clusters support multiple CCO modes.

Prerequisites

You have access to an OKD account with cluster administrator permissions.
You have installed the OpenShift CLI (oc).

Procedure

To determine the mode that the CCO is configured to use, enter the following command:

$ oc get cloudcredentials cluster \
  -o=jsonpath={.spec.credentialsMode}

The following output values are possible, though not all are supported on all platforms:

'': The CCO is operating in the default mode. In this configuration, the CCO operates in mint or passthrough mode, depending on the credentials provided during installation.
Mint: The CCO is operating in mint mode.
Passthrough: The CCO is operating in passthrough mode.
Manual: The CCO is operating in manual mode.

To determine the specific configuration of an AWS or GCP cluster that has a spec.credentialsMode of ‘’, Mint, or Manual, you must investigate further.

AWS and GCP clusters support using mint mode with the root secret deleted.

AWS or GCP clusters that use the default ('') only: To determine whether the cluster is operating in mint or passthrough mode, run the following command:
```
$ oc get secret <secret_name> \
  -n kube-system \
  -o jsonpath \
  --template '{ .metadata.annotations }'
```
where <secret_name> is aws-creds for AWS or gcp-credentials for GCP.

This command displays the value of the .metadata.annotations parameter in the cluster root secret object. The following output values are possible:
- Mint: The CCO is operating in mint mode.
- Passthrough: The CCO is operating in passthrough mode.
If your cluster uses mint mode, you can also determine whether the cluster is operating without the root secret.
AWS or GCP clusters that use mint mode only: To determine whether the cluster is operating without the root secret, run the following command:
```
$ oc get secret <secret_name> \
  -n=kube-system
```
where <secret_name> is aws-creds for AWS or gcp-credentials for GCP.

If the root secret is present, the output of this command returns information about the secret. An error indicates that the root secret is not present on the cluster.
AWS or GCP clusters that use manual mode only: To determine whether the cluster is configured to create and manage cloud credentials from outside of the cluster, run the following command:
```
$ oc get authentication cluster \
  -o jsonpath \
  --template='{ .spec.serviceAccountIssuer }'
```
This command displays the value of the .spec.serviceAccountIssuer parameter in the cluster Authentication object.
- An output of a URL that is associated with your cloud provider indicates that the CCO is using manual mode with AWS STS or GCP Workload Identity to create and manage cloud credentials from outside of the cluster. These clusters are configured using the ccoctl utility.
- An empty output indicates that the cluster is using the CCO in manual mode but was not configured using the ccoctl utility.

Default behavior

For platforms on which multiple modes are supported (AWS, Azure, and GCP), when the CCO operates in its default mode, it checks the provided credentials dynamically to determine for which mode they are sufficient to process CredentialsRequest CRs.

By default, the CCO determines whether the credentials are sufficient for mint mode, which is the preferred mode of operation, and uses those credentials to create appropriate credentials for components in the cluster. If the credentials are not sufficient for mint mode, it determines whether they are sufficient for passthrough mode. If the credentials are not sufficient for passthrough mode, the CCO cannot adequately process CredentialsRequest CRs.

If the provided credentials are determined to be insufficient during installation, the installation fails. For AWS, the installer fails early in the process and indicates which required permissions are missing. Other providers might not provide specific information about the cause of the error until errors are encountered.

If the credentials are changed after a successful installation and the CCO determines that the new credentials are insufficient, the CCO puts conditions on any new CredentialsRequest CRs to indicate that it cannot process them because of the insufficient credentials.

To resolve insufficient credentials issues, provide a credential with sufficient permissions. If an error occurred during installation, try installing again. For issues with new CredentialsRequest CRs, wait for the CCO to try to process the CR again. As an alternative, you can manually create IAM for AWS, Azure, and GCP.

Additional resources

Cluster Operators reference page for the Cloud Credential Operator