Front50: Cassandra to Object Store (S3, Azure, or GCS)

Migrate from Cassandra to cloud-based persistent storage.

Cassandra is no longer an actively maintained or supported persistence store.

A migration to S3, GCS, or AZS is recommended.

1. Create a bucket and folder

Make up a bucket name (or account name if using Azure) that is consistent with the naming policies for the underlying storage service. For purposes of this document, we’ll pick ${USER}-spinnaker since many storage services (including Amazon S3, Google GCS, and Azure AZS) require globally unique names.

Make up a folder name within the bucket. The default is “front50”. Spinnaker will store all the objects within this folder. The folder name will be used for configuration and managed by Spinnaker. You do not need to physically create the folder.

Note that currently only Amazon Simple Storage Service (S3), Google Cloud Storage (GCS), or Azure Storage (AZS) are supported, and they are mutually exclusive. Pick only one. This is independent of where you are actually running Spinnaker.

A. Create the bucket in S3

If you wish to use S3 as the storage service, create the bucket and underlying root folder.

See Amazon’s Documentation for more information.

To enable versioning on an existing bucket, follow these steps.

B. Create the bucket in GCS

If you wish to use GCS as the storage service, Spinnaker can automatically create the bucket (with versioning) for you if it has the right OAuth scopes (Storage Admin). To create it yourself you are best off using the gsutil tool . Using gsutil, also turn on versioning within the bucket.

  1. gsutil mb ${USER}-spinnaker
  2. gsutil versioning set on ${USER}-spinnaker

C. Create the storage account in AZS

If you wish to use Azure Storage, Spinnaker can automatically create the container and root folder (with versioning). Follow instructions here to create a storage account and make sure to note the storage account name and a key.

2. Disable Cassandra in spinnaker-local.yml

  1. services:
  2.   front50:
  3.     cassandra:
  4.       enabled: false

3. Enable the object store in spinnaker-local.yml

A. Enable S3

If you wish to use S3 as the storage service, set the following:

  1. services:
  2.   front50:
  3.     storage_bucket: YOUR_S3_BUCKET_NAME (From Step #1)
  4.     bucket_root: YOUR_S3_FOLDER_NAME (From Step #1)
  5.     s3:
  6.       enabled: true

B. Enable GCS

If you wish to use GCS as the storage service, set the following YAML properties. The default project name is the ${providers.google.primaryCredentials.project} from spinnaker-local.yml. It is needed only if the bucket does not exist and Spinnaker is going to create the bucket for you. The jsonPath is used to convey the credentials to use in order to access GCS. The default is ${providers.google.primaryCredentials.jsonPath} from spinnaker-local.yml however if you created the bucket in a different project (e.g. in the project that is running Spinnaker as opposed to the project that Spinnaker may be managing) then you may need to supply different credentials.

For more information about credentials and obtaining a JSON credentials file, see the discussion on Service Accounts

  1. services:
  2.   front50:
  3.     storage_bucket: YOUR_GCS_BUCKET_NAME (From Step #1)
  4.     bucket_root: YOUR_GCS_FOLDER_NAME (From Step #1)
  5.     gcs:
  6.       enabled: true
  7.       project: SEE NOTE
  8.       jsonPath: SEE NOTE

C. Enable AZS

If you wish to use Azure Storage, set the following:

  1. services:
  2.   front50:
  3.     azs:
  4.       enabled: true
  5.       storageAccountName: YOUR_STORAGE_ACCOUNT_NAME (From Step #1)
  6.       storageAccountKey: YOUR_STORAGE_ACCOUNT_KEY (From Step #1)
  7.       storageContainerName: front50

4. Export existing applications, pipelines, strategies, notifications and projects

Replace “FRONT50_HOSTNAME” and “FRONT50_PORT” and run the following (“localhost” and “8080” are the defaults):

  1. #!/bin/sh
  3. rm applications.json
  4. curl http://FRONT50_HOSTNAME:FRONT50_PORT/v2/applications | json_pp > applications.json
  6. rm pipelines.json
  7. curl http://FRONT50_HOSTNAME:FRONT50_PORT/pipelines | json_pp > pipelines.json
  9. rm strategies.json
  10. curl http://FRONT50_HOSTNAME:FRONT50_PORT/strategies | json_pp > strategies.json
  12. rm notifications.json
  13. curl http://FRONT50_HOSTNAME:FRONT50_PORT/notifications | json_pp > notifications.json
  15. rm projects.json
  16. curl http://FRONT50_HOSTNAME:FRONT50_PORT/v2/projects | json_pp > projects.json

5. Deploy new Front50

Run sudo restart front50

6. Import Applications, Pipelines, Strategies, Notifications and Projects

Replace “FRONT50_HOSTNAME” and “FRONT50_PORT” and run the following (“localhost” and “8080” are the defaults):

  1. #!/bin/sh
  3. curl -X POST -H "Content-type: application/json" --data-binary @"notifications.json" http://FRONT50_HOSTNAME:FRONT50_PORT/notifications/batchUpdate
  4. curl -X POST -H "Content-type: application/json" --data-binary @"strategies.json" http://FRONT50_HOSTNAME:FRONT50_PORT/strategies/batchUpdate
  5. curl -X POST -H "Content-type: application/json" --data-binary @"pipelines.json" http://FRONT50_HOSTNAME:FRONT50_PORT/pipelines/batchUpdate
  6. curl -X POST -H "Content-type: application/json" --data-binary @"applications.json" http://FRONT50_HOSTNAME:FRONT50_PORT/v2/applications/batch/applications
  7. curl -X POST -H "Content-type: application/json" --data-binary @"projects.json" http://FRONT50_HOSTNAME:FRONT50_PORT/v2/projects/batchUpdate

Last modified October 8, 2020: docs(headers): shorten linkTitle and description where applicable (68701bc)