ACL Token Migration

Consul 1.4.0 introduces a new ACL system with improvements for the security and management of ACL tokens and policies. This guide documents how to upgrade existing (now called “legacy”) tokens after upgrading to 1.4.0.

Since the policy syntax changed to be more precise and flexible to manage, it’s necessary to manually translate old tokens into new ones to take advantage of the new ACL system features. Tooling is provided to help automate this and this guide describes the overall process.

Note: Before starting the token migration process all Consul agents, servers and clients, must be running at least version 1.4.0. Additionally, you must ensure the cluster is in a healthy state including a functioning leader. Once the leader has determined that all servers in the cluster are capable of using the new ACL system, the leader will transition itself. Then, the other servers will transition themselves to the new system, followed by the client agents. You can use consul info to investigate the cluster health.

Consul 1.4.0 retains full support for “legacy” ACL tokens so upgrades from Consul 1.3.0 are safe. Existing tokens will continue to work in the same way for at least two “major” releases (1.5.x, 1.6.x, etc; note HashiCorp does not use SemVer for our products).

This document will briefly describes the high-level migration process and provides some specific examples of migration strategies.

Migration Process

While “legacy” tokens will continue to work for several major releases, it’s advisable to plan on migrating existing tokens as soon as is convenient. Migrating also enables using the new policy management improvements, stricter policy syntax rules and other features of the new system without re-issuing all the secrets in use.

The high-level process for migrating a legacy token is as follows:

  1. Create a new policy or policies that grant the required access
  2. Update the existing token to use those policies

Prerequisites

This process assumes that the 1.4.0 upgrade is complete including all legacy ACLs having their accessor IDs populated. This might take up to several minutes after the servers upgrade in the primary datacenter. You can tell if this is the case by using consul acl token list and checking that no tokens exist with a blank AccessorID.

In addition, it is assumed that all clients that might create ACL tokens (e.g. Vault’s Consul secrets engine) have been updated to use the new ACL APIs.

Specifically if you are using Vault’s Consul secrets engine you need to be running Vault 1.0.0 or higher, and you must update all roles defined in Vault to specify a list of policy names rather than an inline policy (which causes Vault to use the legacy API).

Note: if you have systems still creating “legacy” tokens with the old APIs, the migration steps below will still work, however you’ll have to keep re-running them until nothing is creating legacy tokens to ensure all tokens are migrated.

Creating Policies

There are a range of different strategies for creating new policies from existing tokens. Two high-level strategies are described here although others or a mixture of these may be most appropriate depending on the ACL tokens you already have.

Strategy 1: Simple Policy Mapping

The simplest and most automatic strategy is to create one new policy for every existing token. This is easy to automate, but may result in a lot of policies with exactly the same rules and with non-human-readable names which will make managing policies harder. This approach can be accomplished using the consul acl policy create command with -from-token option.

ProsCons
✅ Simple❌ May leave many duplicated policies
✅ Easy to automate❌ Policy names not human-readable

A detailed example of using this approach is given below.

Strategy 2: Combining Policies

This strategy takes a more manual approach to create a more manageable set of policies. There are a spectrum of options for how to do this which tradeoff increasing human involvement for increasing clarity and re-usability of the resulting policies.

For example you could use hashes of the policy rules to de-duplicate identical token policies automatically, however naming them something meaningful for humans would likely still need manual intervention.

Toward the other end of the spectrum it might be beneficial for security to translate prefix matches into exact matches. This however requires the operator knowing that clients using the token really doesn’t rely on the prefix matching semantics of the old ACL system.

To assist with this approach, there is a CLI tool and corresponding API that can translate a legacy ACL token’s rules into a new ACL policy that is exactly equivalent. See consul acl translate-rules.

ProsCons
✅ Clearer, more manageable policies❌ Requires more manual effort
✅ Policies can be re-used by new ACL tokens❌ May take longer for large or complex existing policy sets

A detailed example of using this approach is given below.

Updating Existing Tokens

Once you have created one or more policies that adequately express the rules needed for a legacy token, you can update the token via the CLI or API to use those policies.

After updating, the token is no longer considered “legacy” and will have all the properties of a new token, however it keeps it’s SecretID (the secret part of the token used in API calls) so clients already using that token will continue to work. It is assumed that the policies you attach continue to grant the necessary access for existing clients; this is up to the operator to ensure.

Update via API

Use the PUT /v1/acl/token/:AccessorID endpoint. Specifically, ensure that the Rules field is omitted or empty. Empty Rules indicates that this is now treated as a new token.

Update via CLI

Use the consul acl token update command to update the token. Specifically you need to use -upgrade-legacy which will ensure that legacy rules are removed as well as the new policies added.

Migration Examples

Below are two detailed examples of the two high-level strategies for creating policies discussed above. It should be noted these are intended to clarify the concrete steps you might take. We don’t recommend you perform production migrations with ad-hoc terminal commands. Combining these or something similar into a script might be appropriate.

Simple Policy Mapping

This strategy uses the CLI to create a new policy for every existing legacy token with exactly equivalent rules. It’s easy to automate and clients will see no change in behavior for their tokens, but it does leave you with a lot of potentially identical policies to manage or clean up later.

Create Policies

You can get the AccessorID of every legacy token from the API. For example, using curl and jq in bash:

  1. $ LEGACY_IDS=$(curl -sH "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  2.    'localhost:8500/v1/acl/tokens' | jq -r '.[] | select (.Legacy) | .AccessorID')
  3. $ echo "$LEGACY_IDS"
  4. 621cbd12-dde7-de06-9be0-e28d067b5b7f
  5. 65cecc86-eb5b-ced5-92dc-f861cf7636fe
  6. ba464aa8-d857-3d26-472c-4d49c3bdae72

To create a policy for each one we can use something like:

  1. for id in $LEGACY_IDS; do \
  2.   consul acl policy create -name "migrated-$id" -from-token $id \
  3.     -description "Migrated from legacy ACL token"; \
  4. done

Each policy now has an identical set of rules to the original token. You can inspect these:

  1. $ consul acl policy read -name migrated-621cbd12-dde7-de06-9be0-e28d067b5b7f
  2. ID:           573d84bd-8b08-3061-e391-d2602e1b4947
  3. Name:         migrated-621cbd12-dde7-de06-9be0-e28d067b5b7f
  4. Description:  Migrated from legacy ACL token
  5. Datacenters:
  6. Rules:
  7. service_prefix "" {
  8.   policy = "write"
  9. }

Notice how the policy here is service_prefix and not service since the old ACL syntax was an implicit prefix match. This ensures any clients relying on prefix matching behavior will still work.

Update Tokens

With the policies created as above, we can automatically upgrade all legacy tokens.

  1. for id in $LEGACY_IDS; do \
  2.   consul acl token update -id $id -policy-name "migrated-$id" -upgrade-legacy; \
  3. done

The update is now complete, all legacy tokens are now new tokens with identical secrets and enforcement rules.

Combining Policies

This strategy has more manual elements but results in a cleaner and more manageable set of policies than the fully automatic solutions. Note that this is just an example to illustrate a few ways you may choose to merge or manipulate policies.

Find All Unique Policies

You can get the AccessorID of every legacy token from the API. For example, using curl and jq in bash:

  1. $ LEGACY_IDS=$(curl -sH "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  2.    'localhost:8500/v1/acl/tokens' | jq -r '.[] | select (.Legacy) | .AccessorID')
  3. $ echo "$LEGACY_IDS"
  4. 8b65fdf9-303e-0894-9f87-e71b3273600c
  5. d9deb39b-1b30-e100-b9c5-04aba3f593a1
  6. f2bce42e-cdcc-848d-28ca-cfd0556a22e3

Now we want to read the actual policy for each legacy token and de-duplicate them. We can use the translate-rules helper sub-command which will read the token’s policy and return a new ACL policy that is exactly equivalent.

  1. $ for id in $LEGACY_IDS; do \
  2.   echo "Policy for $id:"
  3.   consul acl translate-rules -token-accessor "$id"; \
  4. done
  5. Policy for 8b65fdf9-303e-0894-9f87-e71b3273600c:
  6. service_prefix "bar" {
  7.   policy = "write"
  8. }
  9. Policy for d9deb39b-1b30-e100-b9c5-04aba3f593a1:
  10. service_prefix "foo" {
  11.   policy = "write"
  12. }
  13. Policy for f2bce42e-cdcc-848d-28ca-cfd0556a22e3:
  14. service_prefix "bar" {
  15.   policy = "write"
  16. }

Notice that two policies are the same and one different.

We can change the loop above to take a hash of this policy definition to de-duplicate the policies into a set of files locally. This example uses command available on macOS but equivalents for other platforms should be easy to find.

  1. $ mkdir policies
  2. $ for id in $LEGACY_IDS; do \
  3.   # Fetch the equivalent new policy rules based on the legacy token rules
  4.   NEW_POLICY=$(consul acl translate-rules -token-accessor "$id"); \
  5.   # Sha1 hash the rules
  6.   HASH=$(echo -n "$NEW_POLICY" | shasum | awk '{ print $1 }'); \
  7.   # Write rules to a policy file named with the hash to de-duplicated
  8.   echo "$NEW_POLICY" > policies/$HASH.hcl; \
  9. done
  10. $ tree policies
  11. policies
  12. ├── 024ce11f26f59436c518fb31f0999d1400485c17.hcl
  13. └── 501b787c9444fbd62f346ab257eeb27197be2444.hcl

Cleaning Up Policies

You can now manually inspect and potentially edit these policies. For example we could rename them according to their intended use. In this case we maintain the hash as it will allow us to match tokens to policies later.

  1. $ cat policies/024ce11f26f59436c518fb31f0999d1400485c17.hcl
  2. service_prefix "bar" {
  3.   policy = "write"
  4. }
  5. $ # Add human-readable suffix to the file name so policies end up clearly named
  6. $ mv policies/024ce11f26f59436c518fb31f0999d1400485c17.hcl \
  7.   policies/024ce11f26f59436c518fb31f0999d1400485c17-bar-service.hcl

You might also choose to tighten up the rules, for example if you know you never rely on prefix-matching the service name foo you might choose to modify the policy to use exact match.

  1. $ cat policies/501b787c9444fbd62f346ab257eeb27197be2444.hcl
  2. service_prefix "foo" {
  3.   policy = "write"
  4. }
  5. $ echo 'service "foo" { policy = "write" }' > policies/501b787c9444fbd62f346ab257eeb27197be2444.hcl
  6. $ # Add human-readable suffix to the file name so policies end up clearly named
  7. $ mv policies/501b787c9444fbd62f346ab257eeb27197be2444.hcl \
  8.   policies/501b787c9444fbd62f346ab257eeb27197be2444-foo-service.hcl

Creating Policies

We now have a minimal set of policies to create, with human-readable names. We can create each one with something like the following.

  1. $ for p in $(ls policies | grep ".hcl"); do \
  2.   # Extract the hash part of the file name
  3.   HASH=$(echo "$p" | cut -d - -f 1); \
  4.   # Extract the name suffix without .hcl
  5.   NAME=$(echo "$p" | cut -d - -f 2- | cut -d . -f 1); \
  6.   # Create new policy based on the rules in the file and the name we gave
  7.   consul acl policy create -name $NAME \
  8.     -rules "@policies/$p" \
  9.     -description "Migrated from legacy token"; \
  10. done
  11. ID:           da2a9f9b-4e44-13f8-e308-76ce7a8dcb21
  12. Name:         bar-service
  13. Description:  Migrated from legacy token
  14. Datacenters:
  15. Rules:
  16. service_prefix "bar" {
  17.   policy = "write"
  18. }
  20. ID:           9fbded86-9140-efe4-b661-c8bd07b6c584
  21. Name:         foo-service
  22. Description:  Migrated from legacy token
  23. Datacenters:
  24. Rules:
  25. service "foo" { policy = "write" }

Upgrading Tokens

Finally we can map our existing tokens to those policies using the hash in the policy file names. The -upgrade-legacy flag removes the token’s legacy embedded rules at the same time as associating them with the new policies created from those rules.

  1. $ for id in $LEGACY_IDS; do \
  2.   NEW_POLICY=$(consul acl translate-rules -token-accessor "$id"); \
  3.   HASH=$(echo -n "$NEW_POLICY" | shasum | awk '{ print $1 }'); \
  4.   # Lookup the hash->new policy mapping from the policy file names
  5.   POLICY_FILE=$(ls policies | grep "^$HASH"); \
  6.   POLICY_NAME=$(echo "$POLICY_FILE" | cut -d - -f 2- | cut -d . -f 1); \
  7.   echo "==> Mapping token $id to policy $POLICY_NAME"; \
  8.   consul acl token update -id $id -policy-name $POLICY_NAME -upgrade-legacy; \
  9. done
  10. ==> Mapping token 8b65fdf9-303e-0894-9f87-e71b3273600c to policy bar-service
  11. Token updated successfully.
  12. AccessorID:   8b65fdf9-303e-0894-9f87-e71b3273600c
  13. SecretID:     3dbb3981-7654-733a-3475-5ce20fc5a7b9
  14. Description:
  15. Local:        false
  16. Create Time:  0001-01-01 00:00:00 +0000 UTC
  17. Policies:
  18.    da2a9f9b-4e44-13f8-e308-76ce7a8dcb21 - bar-service
  19. ==> Mapping token d9deb39b-1b30-e100-b9c5-04aba3f593a1 to policy foo-service
  20. Token updated successfully.
  21. AccessorID:   d9deb39b-1b30-e100-b9c5-04aba3f593a1
  22. SecretID:     5f54733b-4c76-eb74-8781-3550c20f4969
  23. Description:
  24. Local:        false
  25. Create Time:  0001-01-01 00:00:00 +0000 UTC
  26. Policies:
  27.    9fbded86-9140-efe4-b661-c8bd07b6c584 - foo-service
  28. ==> Mapping token f2bce42e-cdcc-848d-28ca-cfd0556a22e3 to policy bar-service
  29. Token updated successfully.
  30. AccessorID:   f2bce42e-cdcc-848d-28ca-cfd0556a22e3
  31. SecretID:     f3aaa3e2-2c6f-cf3c-1e86-454de728e8ab
  32. Description:
  33. Local:        false
  34. Create Time:  0001-01-01 00:00:00 +0000 UTC
  35. Policies:
  36.    da2a9f9b-4e44-13f8-e308-76ce7a8dcb21 - bar-service

At this point all tokens are upgraded and can use new ACL features while retaining the same secret clients are already using.