External Account Configuration

You can manage some Spinnaker configurations externally, which allows you to load them dynamically.

This page describes Spinnaker’s external configuration feature. Using external configuration, you can manage account configuration (either complete account configuration, or only secrets such as account passwords) externally from Spinnaker.

External configuration uses the open-source Spring Cloud Config project, which includes a configuration server and client libraries to support centralized external configuration in a distributed system. The Config Server connects to a remote configuration repository and serves stored configuration properties to client applications.

Configuration properties can be organized by Spring profile and application name and can be stored in any of a number of backends, including GitHub, HashiCorp Vault, AWS S3, JDBC, CredHub, and others. For more information about Spring Cloud Config, see the documentation .

The following table lists the Spinnaker services that currently incorporate external configuration.

ServiceAccount TypesNotes
ClouddriverCloud provider, artifactAutomatic configuration refreshing is supported for Cloud Foundry and Kubernetes cloud provider accounts only.
EchoPub/Sub
IgorCI system, SCM (e.g. Git)

For Clouddriver, support for external configuration was added in Spinnaker 1.15. For Echo and Igor, support was added in Spinnaker 1.16.

Enabling external configuration

To enable external configuration via the Config Server, add Spring Cloud Config configuration properties, which are under spring.cloud.config.server, and include settings for the backend you wish to use.

If you are deploying Spinnaker using Halyard, you can place this file under the ~/.hal/$DEPLOYMENT/profiles directory in the machine on which Halyard is installed, as described in Custom Profiles ($DEPLOYMENT is typically default). If you are deploying Spinnaker manually, you might place this configuration in a YAML file called spinnakerconfig.yml, alongside your spinnaker.yml file.

Using a Git backend

To use a Git backend, configure the settings under spring.cloud.config.server.git. Your configuration might look like the following example:

  1. spring:
  2.   profiles:
  3.     include: git
  4.   cloud:
  5.     config:
  6.       server:
  7.         git:
  8.           uri: https://github.com/example/spinnaker-config
  9.           basedir: /tmp/config-repo
  10.           refresh-rate: 10

The repository will be cloned to the directory specified by the git.basedir property. If using multiple Git repositories, you must give each repository a unique value for basedir.

Using a Vault backend

To use a HashiCorp Vault backend, configure the settings under spring.cloud.config.server.vault. Your configuration might look like the following example:

  1. spring:
  2.   profiles:
  3.     include: vault
  4.   cloud:
  5.     config:
  6.       server:
  7.         vault:
  8.           host: config-vault.example.com
  9.           port: 8200
  10.           backend: secret
  11.           kvVersion: 2
  12.           defaultKey: clouddriver
  13.           token: [vault access token]

The above example configures authentication for Vault using using a static authentication token, via the setting of the spring.cloud.config.server.vault.token property. Starting with Spinnaker 1.18, you can use other methods of configuring authentication for Vault. See the relevant Spring Cloud Config Server documentation for more information.

Using an S3 backend

To use an AWS S3 backend, enable the awss3 profile and configure the settings under spring.cloud.config.server.awss3. Your configuration might look like the following example:

  1. spring:
  2.   profiles:
  3.     active: awss3
  4.   cloud:
  5.     config:
  6.       server:
  7.         awss3:
  8.           region: us-west-2
  9.           bucket: my-bucket

To configure authentication, you must use AWS’s Default Credential Provider Chain. See the AWS SDK documentation for more information.

For information about configuring other supported Config Server backends, see the Spring Cloud Config Server documentation .

Managing external configuration

You can use the Config Server to manage all of your account configuration (bypassing Halyard), or to manage secrets used by Halyard account configuration. You can also use the Config Server to retrieve configuration files or encrypted configuration properties.

Complete account configuration

If you wish to use external configuration instead of Halyard to store complete configuration for your accounts, you can move the account configuration to files stored in a Config Server backend, such as a Git repository or HashiCorp Vault server.

To store complete account configuration in Git, create a Git repository and within it, place a file named spinnaker.yml. The file should contain the account configuration, as in the following example of Echo configuration for a Pub/Sub account:

  1. pubsub:
  2.   enabled: true
  3.   google:
  4.     enabled: true
  5.     pubsubType: GOOGLE
  6.     subscriptions:
  7.     - name: my-gcs-subscription
  8.       project: my-project
  9.       subscriptionName: my-gcs-subscription
  10.       jsonPath: /path/to/my-gce-account.json
  11.       ackDeadlineSeconds: 10
  12.       messageFormat: GCS
  13.     publishers: []

To store complete account configuration in HashiCorp Vault, create a JSON file (for example, echo.json) containing the account configuration, as in the following example:

  1. {
  2.    "pubsub": {
  3.       "enabled": true,
  4.       "google": {
  5.          "enabled": true,
  6.          "pubsubType": "GOOGLE",
  7.          "subscriptions": [
  8.             {
  9.                "name": "my-gcs",
  10.                "project": "My-Project",
  11.                "subscriptionName": "my-gcs-subscription",
  12.                "jsonPath": "/home/spinnaker/.hal/default/staging/dependencies/my-gce-account.json",
  13.                "ackDeadlineSeconds": 10,
  14.                "messageFormat": "GCS"
  15.             }
  16.          ],
  17.          "publishers": []
  18.       }
  19.    }
  20. }

Then use the vault command-line interface tool to store this JSON in Vault:

  1. $ vault kv put secret/echo @echo.json

Secrets

If you wish to use Halyard to manage account configuration but store secrets (such as account passwords) externally, you can move the account secrets into files stored in a Config Server backend, such as a Git repository or HashiCorp Vault server. You can then use Spring property placeholders to reference the secret values.

If you are configuring a Jenkins account in Igor, your spinnaker.yml file might look like the following example:

  1. jenkins:
  2.   enabled: true
  3.   masters:
  4.   - name: my-jenkins-master
  5.     permissions: {}
  6.     address: http://my-jenkins.ci.example.com
  7.     username: myuser
  8.     password: ${ci.my-jenkins.password}

You could add this configuration using the hal CLI, as in the following example:

  1. $ hal config provider jenkins account add \
  2.    my-jenkins-master \
  3.    --address http://my-jenkins.ci.example.com \
  4.    --username myuser \
  5.    --password ${ci.my-jenkins.password}

In the Config Server backend (such as a Git repository), the account secrets file might look like the following example:

  1. ci:
  2.   my-jenkins:
  3.     password: 'secret'

Configuration files

Some account configuration, including configuration for the Kubernetes, Google Compute Engine, and Google App Engine cloud providers in Clouddriver and Google Cloud Pub/Sub accounts in Echo, allows credentials to be loaded from files that are separate from the YAML configuration. You can load these separate files from an external source using the Config Server’s Resource abstraction. In the service YAML configuration, prefix a file path with configserver: to indicate that the file should be retrieved by the Config Server.

External configuration files (such as kubeconfig files and GCP JSON files) can only be loaded from S3 or Git (not from Vault).

If you are configuring a Kubernetes cloud provider account and want to load an external kubeconfig.yml file using Config Server, your account configuration might look like the following example:

  1. kubernetes:
  2.   enabled: true
  3.   accounts:
  4.     - name: default
  5.       kubeconfigFile: configserver:kubeconfig.yml
  6.       dockerRegistries:
  7.         - accountName: dockerhub
  8.           namespaces: []
  9.       context: default

Encrypted values

The Config Server can store encrypted secrets in its backend and decrypt the secret values when serving them to clients. You can use this to store encrypted Spinnaker account secrets in a Config Server backend (such as a Git repository).

To enable Config Server decryption, configure the encryption key in your spinnaker-config-local.yml, along with the Config Server backend configuration properties. You can configure the encryption key using properties under encrypt.

When using Config Server encryption or decryption, your configuration might look like the following example:

  1. spring:
  2.   profiles:
  3.     include: git
  4.   cloud:
  5.     config:
  6.       server:
  7.         git:
  8.           uri: https://github.com/example/spinnaker-config
  9.           basedir: /tmp/config-repo
  10.           refresh-rate: 10
  11. encrypt:
  12.   key: mykey

For more information about using the Config Server’s encryption and decryption features, see the Spring Cloud Config Server documentation .

External configuration refresh

If you have enabled caching , Spinnaker will refresh its external configuration automatically on the same time interval as the Clouddriver agent caches. When using the default in-memory caching, this interval defaults to 60 seconds. When using Redis caching , you can configure this interval using the redis.poll.intervalSeconds property.

Last modified February 22, 2022: Update link to doc (#182) (10d8e8c)