How-To: Query state

API for querying state stores

alpha

The state query API is in alpha stage.

Introduction

The state query API provides a way of querying the key/value data stored in state store components. This query API is not a replacement for a complete query language, and is focused on retrieving, filtering and sorting key/value data that you have saved through the state management APIs.

Even though the state store is a key/value store, the value might be a JSON document with its own hierarchy, keys, and values. The query API allows you to use those keys and values to retrive corresponding documents.

Limitations

The state query API has the following limitations:

  • The API does not support querying of actor state stored in a state store. For that you need to use the query API for the specific database. See querying actor state.
  • The API does not work with Dapr encrypted state stores capability. Since the encryption is done by the Dapr runtime and stored as encrypted data, then this effectively prevents server side querying.

You can find additional information in the related links section.

Querying the state

You submit query requests via HTTP POST/PUT or gRPC. The body of the request is the JSON map with 3 entries: filter, sort, and page.

The filter is an optional section. It specifies the query conditions in the form of a tree of key/value operations, where the key is the operator and the value is the operands.

The following operations are supported:

OperatorOperandsDescription
EQkey:valuekey == value
INkey:[]valuekey == value[0] OR key == value[1] OR … OR key == value[n]
AND[]operationoperation[0] AND operation[1] AND … AND operation[n]
OR[]operationoperation[0] OR operation[1] OR … OR operation[n]

If filter section is omitted, the query returns all entries.

The sort is an optional section and is an ordered array of key:order pairs, where key is a key in the state store, and the order is an optional string indicating sorting order: "ASC" for ascending and "DESC" for descending. If omitted, ascending order is the default.

The page is an optional section containing limit and token parameters. limit sets the page size. token is an iteration token returned by the component, and is used in subsequent queries.

For some background understanding, this query request is translated into the native query language and executed by the state store component.

Example data and query

Let’s look at some real examples, starting with simple and progressing towards more complex ones.

As a dataset, let’s consider a collection of with employee records containing employee ID, organization, state, and city. Notice that this dataset is an array of key/value pairs where key is the unique ID, and the value is the JSON object with employee record. To better illustrate functionality, let’s have organization name (org) and employee ID (id) as a nested JSON person object.

First, you need to create an instance of MongoDB, which is your state store.

  1. docker run -d --rm -p 27017:27017 --name mongodb mongo:5

Next is to start a Dapr application. Refer to this component configuration file, which instructs Dapr to use MongoDB as its state store.

  1. dapr run --app-id demo --dapr-http-port 3500 --components-path query-api-examples/components

Now populate the state store with the employee dataset, so you can then query it later.

  1. curl -X POST -H "Content-Type: application/json" -d @query-api-examples/dataset.json http://localhost:3500/v1.0/state/statestore

Once populated, you can examine the data in the state store. The image below a section of the MongoDB UI displaying employee records.

Sample dataset

Each entry has the _id member as a concatenated object key, and the value member containing the JSON record.

The query API allows you to select records from this JSON structure.

Now you can run the queries.

Example 1

First, let’s find all employees in the state of California and sort them by their employee ID in descending order.

This is the query:

  1. {
  2.     "filter": {
  3.         "EQ": { "value.state": "CA" }
  4.     },
  5.     "sort": [
  6.         {
  7.             "key": "value.person.id",
  8.             "order": "DESC"
  9.         }
  10.     ]
  11. }

An equivalent of this query in SQL is:

  1. SELECT * FROM c WHERE
  2.   value.state = "CA"
  3. ORDER BY
  4.   value.person.id DESC

Execute the query with the following command:

  1. curl -s -X POST -H "Content-Type: application/json" -d @query-api-examples/query1.json http://localhost:3500/v1.0-alpha1/state/statestore/query | jq .
  1. Invoke-RestMethod -Method Post -ContentType 'application/json' -InFile query-api-examples/query1.json -Uri 'http://localhost:3500/v1.0-alpha1/state/statestore/query'

The query result is an array of matching key/value pairs in the requested order:

  1. {
  2.   "results": [
  3.     {
  4.       "key": "3",
  5.       "data": {
  6.         "person": {
  7.           "org": "Finance",
  8.           "id": 1071
  9.         },
  10.         "city": "Sacramento",
  11.         "state": "CA"
  12.       },
  13.       "etag": "44723d41-deb1-4c23-940e-3e6896c3b6f7"
  14.     },
  15.     {
  16.       "key": "7",
  17.       "data": {
  18.         "city": "San Francisco",
  19.         "state": "CA",
  20.         "person": {
  21.           "id": 1015,
  22.           "org": "Dev Ops"
  23.         }
  24.       },
  25.       "etag": "0e69e69f-3dbc-423a-9db8-26767fcd2220"
  26.     },
  27.     {
  28.       "key": "5",
  29.       "data": {
  30.         "state": "CA",
  31.         "person": {
  32.           "org": "Hardware",
  33.           "id": 1007
  34.         },
  35.         "city": "Los Angeles"
  36.       },
  37.       "etag": "f87478fa-e5c5-4be0-afa5-f9f9d75713d8"
  38.     },
  39.     {
  40.       "key": "9",
  41.       "data": {
  42.         "person": {
  43.           "org": "Finance",
  44.           "id": 1002
  45.         },
  46.         "city": "San Diego",
  47.         "state": "CA"
  48.       },
  49.       "etag": "f5cf05cd-fb43-4154-a2ec-445c66d5f2f8"
  50.     }
  51.   ]
  52. }

Example 2

Let’s now find all employees from the “Dev Ops” and “Hardware” organizations.

This is the query:

  1. {
  2.     "filter": {
  3.         "IN": { "value.person.org": [ "Dev Ops", "Hardware" ] }
  4.     }
  5. }

An equivalent of this query in SQL is:

  1. SELECT * FROM c WHERE
  2.   value.person.org IN ("Dev Ops", "Hardware")

Execute the query with the following command:

  1. curl -s -X POST -H "Content-Type: application/json" -d @query-api-examples/query2.json http://localhost:3500/v1.0-alpha1/state/statestore/query | jq .
  1. Invoke-RestMethod -Method Post -ContentType 'application/json' -InFile query-api-examples/query2.json -Uri 'http://localhost:3500/v1.0-alpha1/state/statestore/query'

Similar to the previous example, the result is an array of matching key/value pairs.

Example 3

In this example let’s find all employees from the “Dev Ops” department and those employees from the “Finance” departing residing in the states of Washington and California.

In addition, let’s sort the results first by state in descending alphabetical order, and then by employee ID in ascending order. Also, let’s process up to 3 records at a time.

This is the query:

  1. {
  2.     "filter": {
  3.         "OR": [
  4.             {
  5.                 "EQ": { "value.person.org": "Dev Ops" }
  6.             },
  7.             {
  8.                 "AND": [
  9.                     {
  10.                         "EQ": { "value.person.org": "Finance" }
  11.                     },
  12.                     {
  13.                         "IN": { "value.state": [ "CA", "WA" ] }
  14.                     }
  15.                 ]
  16.             }
  17.         ]
  18.     },
  19.     "sort": [
  20.         {
  21.             "key": "value.state",
  22.             "order": "DESC"
  23.         },
  24.         {
  25.             "key": "value.person.id"
  26.         }
  27.     ],
  28.     "page": {
  29.         "limit": 3
  30.     }
  31. }

An equivalent of this query in SQL is:

  1. SELECT * FROM c WHERE
  2.   value.person.org = "Dev Ops" OR
  3.   (value.person.org = "Finance" AND value.state IN ("CA", "WA"))
  4. ORDER BY
  5.   value.state DESC,
  6.   value.person.id ASC
  7. LIMIT 3

Execute the query with the following command:

  1. curl -s -X POST -H "Content-Type: application/json" -d @query-api-examples/query3.json http://localhost:3500/v1.0-alpha1/state/statestore/query | jq .
  1. Invoke-RestMethod -Method Post -ContentType 'application/json' -InFile query-api-examples/query3.json -Uri 'http://localhost:3500/v1.0-alpha1/state/statestore/query'

Upon successful execution, the state store returns a JSON object with a list of matching records and the pagination token:

  1. {
  2.   "results": [
  3.     {
  4.       "key": "1",
  5.       "data": {
  6.         "person": {
  7.           "org": "Dev Ops",
  8.           "id": 1036
  9.         },
  10.         "city": "Seattle",
  11.         "state": "WA"
  12.       },
  13.       "etag": "6f54ad94-dfb9-46f0-a371-e42d550adb7d"
  14.     },
  15.     {
  16.       "key": "4",
  17.       "data": {
  18.         "person": {
  19.           "org": "Dev Ops",
  20.           "id": 1042
  21.         },
  22.         "city": "Spokane",
  23.         "state": "WA"
  24.       },
  25.       "etag": "7415707b-82ce-44d0-bf15-6dc6305af3b1"
  26.     },
  27.     {
  28.       "key": "10",
  29.       "data": {
  30.         "person": {
  31.           "org": "Dev Ops",
  32.           "id": 1054
  33.         },
  34.         "city": "New York",
  35.         "state": "NY"
  36.       },
  37.       "etag": "26bbba88-9461-48d1-8a35-db07c374e5aa"
  38.     }
  39.   ],
  40.   "token": "3"
  41. }

The pagination token is used “as is” in the subsequent query to get the next batch of records:

  1. {
  2.     "filter": {
  3.         "OR": [
  4.             {
  5.                 "EQ": { "value.person.org": "Dev Ops" }
  6.             },
  7.             {
  8.                 "AND": [
  9.                     {
  10.                         "EQ": { "value.person.org": "Finance" }
  11.                     },
  12.                     {
  13.                         "IN": { "value.state": [ "CA", "WA" ] }
  14.                     }
  15.                 ]
  16.             }
  17.         ]
  18.     },
  19.     "sort": [
  20.         {
  21.             "key": "value.state",
  22.             "order": "DESC"
  23.         },
  24.         {
  25.             "key": "value.person.id"
  26.         }
  27.     ],
  28.     "page": {
  29.         "limit": 3,
  30.         "token": "3"
  31.     }
  32. }
  1. curl -s -X POST -H "Content-Type: application/json" -d @query-api-examples/query3-token.json http://localhost:3500/v1.0-alpha1/state/statestore/query | jq .
  1. Invoke-RestMethod -Method Post -ContentType 'application/json' -InFile query-api-examples/query3-token.json -Uri 'http://localhost:3500/v1.0-alpha1/state/statestore/query'

And the result of this query is:

  1. {
  2.   "results": [
  3.     {
  4.       "key": "9",
  5.       "data": {
  6.         "person": {
  7.           "org": "Finance",
  8.           "id": 1002
  9.         },
  10.         "city": "San Diego",
  11.         "state": "CA"
  12.       },
  13.       "etag": "f5cf05cd-fb43-4154-a2ec-445c66d5f2f8"
  14.     },
  15.     {
  16.       "key": "7",
  17.       "data": {
  18.         "city": "San Francisco",
  19.         "state": "CA",
  20.         "person": {
  21.           "id": 1015,
  22.           "org": "Dev Ops"
  23.         }
  24.       },
  25.       "etag": "0e69e69f-3dbc-423a-9db8-26767fcd2220"
  26.     },
  27.     {
  28.       "key": "3",
  29.       "data": {
  30.         "person": {
  31.           "org": "Finance",
  32.           "id": 1071
  33.         },
  34.         "city": "Sacramento",
  35.         "state": "CA"
  36.       },
  37.       "etag": "44723d41-deb1-4c23-940e-3e6896c3b6f7"
  38.     }
  39.   ],
  40.   "token": "6"
  41. }

That way you can update the pagination token in the query and iterate through the results until no more records are returned.

Last modified February 18, 2022: Update setup-jetstream.md (#2200) (428d8c2)