HashiCorp Vault Integration

HashiCorp Vault can be used as a secure key management service forServer-Side Encryption (SSE-KMS).

HashiCorp Vault Integration - 图1

Some examples below use the Vault command line utility to interact withVault. You may need to set the following environment variable with the correctaddress of your Vault server to use this utility:

  1. export VAULT_ADDR='http://vault-server:8200'

Vault secrets engines

Vault provides several secrets engines, which can store, generate, and encryptdata. Currently, the Object Gateway supports:

KV secrets engine

The KV secrets engine is used to store arbitrary key/value secrets in Vault. Toenable the KV engine version 2 in Vault, use the following command:

  1. vault secrets enable kv-v2

The Object Gateway can be configured to use the KV engine version 2 with thefollowing setting:

  1. rgw crypt vault secret engine = kv

Transit secrets engine

The transit engine handles cryptographic functions on data in-transit. To enableit in Vault, use the following command:

  1. vault secrets enable transit

The Object Gateway can be configured to use the transit engine with thefollowing setting:

  1. rgw crypt vault secret engine = transit

Vault authentication

Vault supports several authentication mechanisms. Currently, the ObjectGateway can be configured to authenticate to Vault using theToken authentication method or a Vault agent.

Token authentication

Note

Token authentication is not recommended for production environments.

The token authentication method expects a Vault token to be present in aplaintext file. The Object Gateway can be configured to use token authenticationwith the following settings:

  1. rgw crypt vault auth = token
  2. rgw crypt vault token file = /etc/ceph/vault.token
  3. rgw crypt vault addr = http://vault-server:8200

For security reasons, the token file must be readable by the Object Gatewayonly. Also, the Object Gateway should be given a Vault token with a restrictedpolicy that allows it to fetch keyrings from a specific path only. Such a policycan be created in Vault using the command line utility as in the followingexamples:

  1. vault policy write rgw-kv-policy -<<EOF
  2.   path "secret/data/*" {
  3.     capabilities = ["read"]
  4.   }
  5. EOF
  6.  
  7. vault policy write rgw-transit-policy -<<EOF
  8.   path "transit/export/encryption-key/*" {
  9.     capabilities = ["read"]
  10.   }
  11. EOF

Once the policy is created, a token can be generated by a Vault administrator:

  1. vault token create -policy=rgw-kv-policy

Sample output:

  1. Key                  Value
  2. ---                  -----
  3. token                s.72KuPujbc065OdWB71poOmIq
  4. token_accessor       jv95ZYBUFv6Ss84x7SCSy6lZ
  5. token_duration       768h
  6. token_renewable      true
  7. token_policies       ["default" "rgw-kv-policy"]
  8. identity_policies    []
  9. policies             ["default" "rgw-kv-policy"]

The actual token, displayed in the Value column of the first line of theoutput, must be saved in a file as plaintext.

Vault agent

The Vault agent is a client daemon that provides authentication to Vault andmanages token renewal and caching. It typically runs on the same host as theObject Gateway. With a Vault agent, it is possible to use other Vaultauthentication mechanism such as AppRole, AWS, Certs, JWT, and Azure.

The Object Gateway can be configured to use a Vault agent with the followingsettings:

  1. rgw crypt vault auth = agent
  2. rgw crypt vault addr = http://localhost:8100

Vault namespaces

In the Enterprise version, Vault supports the concept of namespaces, whichallows centralized management for teams within an organization while ensuringthat those teams operate within isolated environments known as tenants.

The Object Gateway can be configured to access Vault within a particularnamespace using the following configuration setting:

  1. rgw crypt vault namespace = tenant1

Create a key in Vault

Note

Keys for server-side encryption must be 256-bit long and base-64encoded.

Using the KV engine

A key for server-side encryption can be created in the KV version 2 engine usingthe command line utility, as in the following example:

  1. vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32)

Sample output:

  1. ====== Metadata ======
  2. Key              Value
  3. ---              -----
  4. created_time     2019-08-29T17:01:09.095824999Z
  5. deletion_time    n/a
  6. destroyed        false
  7. version          1

Note that in the KV secrets engine, secrets are stored as key-value pairs, andthe Gateway expects the key name to be key, i.e. the secret must be in theform key=<secret key>.

Using the Transit engine

Keys created with the Transit engine must be exportable in order to be used forserver-side encryption with the Object Gateway. An exportable key can be createdwith the command line utility as follows:

  1. vault write -f transit/keys/mybucketkey exportable=true

The command above creates a keyring, which contains a key of typeaes256-gcm96 by default. To verify that the key was correctly created, usethe following command:

  1. vault read transit/export/encryption-key/mybucketkey/1

Sample output:

  1. Key     Value
  2. ---     -----
  3. keys    map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=]
  4. name    mybucketkey
  5. type    aes256-gcm96

Note that in order to read the key created with the Transit engine, the fullpath must be provided including the key version.

Configure the Ceph Object Gateway

Edit the Ceph configuration file to enable Vault as a KMS backend forserver-side encryption:

  1. rgw crypt s3 kms backend = vault

Choose the Vault authentication method, e.g.:

  1. rgw crypt vault auth = token
  2. rgw crypt vault token file = /etc/ceph/vault.token
  3. rgw crypt vault addr = http://vault-server:8200

Or:

  1. rgw crypt vault auth = agent
  2. rgw crypt vault addr = http://localhost:8100

Choose the secrets engine:

  1. rgw crypt vault secret engine = kv

Or:

  1. rgw crypt vault secret engine = transit

Optionally, set the Vault namespace where encryption keys will be fetched from:

  1. rgw crypt vault namespace = tenant1

Finally, the URLs where the Gateway will retrieve encryption keys from Vault canbe restricted by setting a path prefix. For instance, the Gateway can berestricted to fetch KV keys as follows:

  1. rgw crypt vault prefix = /v1/secret/data

Or, in the case of exportable transit keys:

  1. rgw crypt vault prefix = /v1/transit/export/encryption-key

In the example above, the Gateway would only fetch transit encryption keys underhttp://vault-server:8200/v1/transit/export/encryption-key.

Upload object

When uploading an object to the Gateway, provide the SSE key ID in the request.As an example, using the AWS command-line client:

  1. aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey

The Object Gateway will fetch the key from Vault, encrypt the object and storeit in the bucket. Any request to download the object will make the Gatewayautomatically retrieve the correspondent key from Vault and decrypt the object.

Note that the secret will be fetched from Vault using a URL constructed byconcatenating the base address (rgw crypt vault addr), the (optional)URL prefix (rgw crypt vault prefix), and finally the key ID. In the exampleabove, the Gateway would fetch the secret from:

  1. http://vaultserver:8200/v1/secret/data/myproject/mybucketkey