Azure Cosmos DB (SQL API)

Detailed information on the Azure Cosmos DB (SQL API) state store component

Component format

To setup Azure Cosmos DB state store create a component of type state.azure.cosmosdb. See this guide on how to create and apply a state store configuration.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: <NAME>
  5. spec:
  6.   type: state.azure.cosmosdb
  7.   version: v1
  8.   metadata:
  9.   - name: url
  10.     value: <REPLACE-WITH-URL>
  11.   - name: masterKey
  12.     value: <REPLACE-WITH-MASTER-KEY>
  13.   - name: database
  14.     value: <REPLACE-WITH-DATABASE>
  15.   - name: collection
  16.     value: <REPLACE-WITH-COLLECTION>

Warning

The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described here.

If you wish to use Cosmos DB as an actor store, append the following to the yaml.

  1.   - name: actorStateStore
  2.     value: "true"

Spec metadata fields

FieldRequiredDetailsExample
urlYThe Cosmos DB url“https://******.documents.azure.com:443/“.
masterKeyY*The key to authenticate to the Cosmos DB account. Only required when not using Azure AD authentication.“key”
databaseYThe name of the database“db”
collectionYThe name of the collection (container)“collection”
actorStateStoreNConsider this state store for actors. Defaults to “false”“true”, “false”

Azure Active Directory (Azure AD) authentication

The Azure Cosmos DB state store component supports authentication using all Azure Active Directory mechanisms. For further information and the relevant component metadata fields to provide depending on the choice of Azure AD authentication mechanism, see the docs for authenticating to Azure.

You can read additional information for setting up Cosmos DB with Azure AD authentication in the section below.

Setup Azure Cosmos DB

Follow the instructions from the Azure documentation on how to create an Azure Cosmos DB account. The database and collection must be created in Cosmos DB before Dapr can use it.

Important: The partition key for the collection must be named /partitionKey (note: this is case-sensitive).

In order to setup Cosmos DB as a state store, you need the following properties:

  • URL: the Cosmos DB url. for example: https://******.documents.azure.com:443/
  • Master Key: The key to authenticate to the Cosmos DB account. Skip this if using Azure AD authentication.
  • Database: The name of the database
  • Collection: The name of the collection (or container)

Best Practices for Production Use

Azure Cosmos DB shares a strict metadata request rate limit across all databases in a single Azure Cosmos DB account. New connections to Azure Cosmos DB assume a large percentage of the allowable request rate limit. (See the Cosmos DB documentation)

Therefore several strategies must be applied to avoid simultaneous new connections to Azure Cosmos DB:

  • Ensure sidecars of applications only load the Azure Cosmos DB component when they require it to avoid unnecessary database connections. This can be done by scoping your components to specific applications.
  • Choose deployment strategies that sequentially deploy or start your applications to minimize bursts in new connections to your Azure Cosmos DB accounts.
  • Avoid reusing the same Azure Cosmos DB account for unrelated databases or systems (even outside of Dapr). Distinct Azure Cosmos DB accounts have distinct rate limits.
  • Increase the initTimeout value to allow the component to retry connecting to Azure Cosmos DB during side car initialization for up to 5 minutes. The default value is 5s and should be increased. When using Kubernetes, increasing this value may also require an update to your Readiness and Liveness probes.
  1. spec:
  2.   type: state.azure.cosmosdb
  3.   version: v1
  4.   initTimeout: 5m
  5.   metadata:

Data format

To use the Cosmos DB state store, your data must be sent to Dapr in JSON-serialized format. Having it just JSON serializable will not work.

If you are using the Dapr SDKs (for example the .NET SDK), the SDK automatically serializes your data to JSON.

If you want to invoke Dapr’s HTTP endpoint directly, take a look at the examples (using curl) in the Partition keys section below.

Partition keys

For non-actor state operations, the Azure Cosmos DB state store will use the key property provided in the requests to the Dapr API to determine the Cosmos DB partition key. This can be overridden by specifying a metadata field in the request with a key of partitionKey and a value of the desired partition.

The following operation uses nihilus as the partition key value sent to Cosmos DB:

  1. curl -X POST http://localhost:3500/v1.0/state/<store_name> \
  2.   -H "Content-Type: application/json"
  3.   -d '[
  4.         {
  5.           "key": "nihilus",
  6.           "value": "darth"
  7.         }
  8.       ]'

For non-actor state operations, if you want to control the Cosmos DB partition, you can specify it in metadata. Reusing the example above, here’s how to put it under the mypartition partition

  1. curl -X POST http://localhost:3500/v1.0/state/<store_name> \
  2.   -H "Content-Type: application/json"
  3.   -d '[
  4.         {
  5.           "key": "nihilus",
  6.           "value": "darth",
  7.           "metadata": {
  8.             "partitionKey": "mypartition"
  9.           }
  10.         }
  11.       ]'

For actor state operations, the partition key is generated by Dapr using the appId, the actor type, and the actor id, such that data for the same actor always ends up under the same partition (you do not need to specify it). This is because actor state operations must use transactions, and in Cosmos DB the items in a transaction must be on the same partition.

Setting up Cosmos DB for authenticating with Azure AD

When using the Dapr Cosmos DB state store and authenticating with Azure AD, you need to perform a few additional steps to set up your environment.

Prerequisites:

  • You need a Service Principal created as per the instructions in the authenticating to Azure page. You need the ID of the Service Principal for the commands below (note that this is different from the client ID of your application, or the value you use for azureClientId in the metadata).
  • Azure CLI
  • jq
  • The scripts below are optimized for a bash or zsh shell

Granting your Azure AD application access to Cosmos DB

You can find more information on the official documentation, including instructions to assign more granular permissions.

In order to grant your application permissions to access data stored in Cosmos DB, you need to assign it a custom role for the Cosmos DB data plane. In this example you’re going to use a built-in role, “Cosmos DB Built-in Data Contributor”, which grants your application full read-write access to the data; you can optionally create custom, fine-tuned roles following the instructions in the official docs.

  1. # Name of the Resource Group that contains your Cosmos DB
  2. RESOURCE_GROUP="..."
  3. # Name of your Cosmos DB account
  4. ACCOUNT_NAME="..."
  5. # ID of your Service Principal object
  6. PRINCIPAL_ID="..."
  7. # ID of the "Cosmos DB Built-in Data Contributor" role
  8. # You can also use the ID of a custom role
  9. ROLE_ID="00000000-0000-0000-0000-000000000002"
  11. az cosmosdb sql role assignment create \
  12.   --account-name "$ACCOUNT_NAME" \
  13.   --resource-group "$RESOURCE_GROUP" \
  14.   --scope "/" \
  15.   --principal-id "$PRINCIPAL_ID" \
  16.   --role-definition-id "$ROLE_ID"

Creating the stored procedures for Dapr

When using Cosmos DB as a state store for Dapr, we need to create two stored procedures in your collection. When you configure the state store using a “master key”, Dapr creates those for you, automatically. However, when your state store authenticates with Cosmos DB using Azure AD, because of limitations in the platform we are not able to do it automatically.

If you are using Azure AD to authenticate your Cosmos DB state store and have not created the stored procedures (or if you are using an outdated version of them), your Dapr sidecar will fail to start and you will see an error similar to this in your logs:

  1. Dapr requires stored procedures created in Cosmos DB before it can be used as state store. Those stored procedures are currently not existing or are using a different version than expected. When you authenticate using Azure AD we cannot automatically create them for you: please start this state store with a Cosmos DB master key just once so we can create the stored procedures for you; otherwise, you can check our docs to learn how to create them yourself: https://aka.ms/dapr/cosmosdb-aad

To fix this issue, you have two options:

  1. Configure your component to authenticate with the “master key” just once, to have Dapr automatically initialize the stored procedures for you. While you need to use a “master key” the first time you launch your application, you should be able to remove that and use Azure AD credentials (including Managed Identities) after.
  2. Alternatively, you can follow the steps below to create the stored procedures manually. These steps must be performed before you can start your application the first time.

To create the stored procedures manually, you can use the commands below.

First, download the code of the stored procedures for the version of Dapr that you’re using. This will create two .js files in your working directory:

  1. # Set this to the version of Dapr that you're using
  2. DAPR_VERSION="release-1.8"
  3. curl -LfO "https://raw.githubusercontent.com/dapr/components-contrib/${DAPR_VERSION}/state/azure/cosmosdb/storedprocedures/__daprver__.js"
  4. curl -LfO "https://raw.githubusercontent.com/dapr/components-contrib/${DAPR_VERSION}/state/azure/cosmosdb/storedprocedures/__dapr_v2__.js"

You won’t need to update the code for the stored procedures every time you update Dapr. Although the code for the stored procedures doesn’t change often, sometimes we may make updates to that: when that happens, if you’re using Azure AD authentication your Dapr sidecar will fail to launch until you update the stored procedures, re-running the commands above.

Then, using the Azure CLI create the stored procedures in Cosmos DB, for your account, database, and collection (or container):

  1. # Name of the Resource Group that contains your Cosmos DB
  2. RESOURCE_GROUP="..."
  3. # Name of your Cosmos DB account
  4. ACCOUNT_NAME="..."
  5. # Name of your database in the Cosmos DB account
  6. DATABASE_NAME="..."
  7. # Name of the container (collection) in your database
  8. CONTAINER_NAME="..."
  10. az cosmosdb sql stored-procedure create \
  11.   --resource-group "$RESOURCE_GROUP" \
  12.   --account-name "$ACCOUNT_NAME" \
  13.   --database-name "$DATABASE_NAME" \
  14.   --container-name "$CONTAINER_NAME" \
  15.   --name "__daprver__" \
  16.   --body @__daprver__.js
  17. az cosmosdb sql stored-procedure create \
  18.   --resource-group "$RESOURCE_GROUP" \
  19.   --account-name "$ACCOUNT_NAME" \
  20.   --database-name "$DATABASE_NAME" \
  21.   --container-name "$CONTAINER_NAME" \
  22.   --name "__dapr_v2__" \
  23.   --body @__dapr_v2__.js

Last modified July 27, 2022: Remove namespace element from component examples (#2647) (ff9de5c8)