Redis Streams

Detailed documentation on the Redis Streams pubsub component

Component format

To setup Redis Streams pubsub create a component of type pubsub.redis. See this guide on how to create and apply a pubsub configuration.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: redis-pubsub
  5.   namespace: default
  6. spec:
  7.   type: pubsub.redis
  8.   version: v1
  9.   metadata:
  10.   - name: redisHost
  11.     value: localhost:6379
  12.   - name: redisPassword
  13.     value: "KeFg23!"
  14.   - name: consumerID
  15.     value: "myGroup"
  16.   - name: enableTLS
  17.     value: "false"

Warning

The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described here.

Spec metadata fields

FieldRequiredDetailsExample
redisHostYConnection-string for the redis host. If “redisType” is “cluster” it can be multiple hosts separated by commas or just a single hostlocalhost:6379, redis-master.default.svc.cluster.local:6379
redisPasswordYPassword for Redis host. No Default. Can be secretKeyRef to use a secret reference“”, “KeFg23!”
redisUsernameNUsername for Redis host. Defaults to empty. Make sure your redis server version is 6 or above, and have created acl rule correctly.“”, “default”
consumerIDNThe consumer group ID“myGroup”
enableTLSNIf the Redis instance supports TLS with public certificates, can be configured to be enabled or disabled. Defaults to “false”“true”, “false”
redeliverIntervalNThe interval between checking for pending messages to redelivery. Defaults to “60s”. “0” disables redelivery.“30s”
processingTimeoutNThe amount time a message must be pending before attempting to redeliver it. Defaults to “15s”. “0” disables redelivery.“30s”
queueDepthNThe size of the message queue for processing. Defaults to “100”.“1000”
concurrencyNThe number of concurrent workers that are processing messages. Defaults to “10”.“15”
redisTypeNThe type of redis. There are two valid values, one is “node” for single node mode, the other is “cluster” for redis cluster mode. Defaults to “node”.“cluster”
redisDBNDatabase selected after connecting to redis. If “redisType” is “cluster” this option is ignored. Defaults to “0”.“0”
redisMaxRetriesNMaximum number of times to retry commands before giving up. Default is to not retry failed commands.“5”
redisMinRetryIntervalNMinimum backoff for redis commands between each retry. Default is “8ms”; “-1” disables backoff.“8ms”
redisMaxRetryIntervalNMaximum backoff for redis commands between each retry. Default is “512ms”;“-1” disables backoff.“5s”
dialTimeoutNDial timeout for establishing new connections. Defaults to “5s”.“5s”
readTimeoutNTimeout for socket reads. If reached, redis commands will fail with a timeout instead of blocking. Defaults to “3s”, “-1” for no timeout.“3s”
writeTimeoutNTimeout for socket writes. If reached, redis commands will fail with a timeout instead of blocking. Defaults is readTimeout.“3s”
poolSizeNMaximum number of socket connections. Default is 10 connections per every CPU as reported by runtime.NumCPU.“20”
poolTimeoutNAmount of time client waits for a connection if all connections are busy before returning an error. Default is readTimeout + 1 second.“5s”
maxConnAgeNConnection age at which the client retires (closes) the connection. Default is to not close aged connections.“30m”
minIdleConnsNMinimum number of idle connections to keep open in order to avoid the performance degradation associated with creating new connections. Defaults to “0”.“2”
idleCheckFrequencyNFrequency of idle checks made by idle connections reaper. Default is “1m”. “-1” disables idle connections reaper.“-1”
idleTimeoutNAmount of time after which the client closes idle connections. Should be less than server’s timeout. Default is “5m”. “-1” disables idle timeout check.“10m”
failoverNProperty to enabled failover configuration. Needs sentinalMasterName to be set. Defaults to “false”“true”, “false”
sentinelMasterNameNThe sentinel master name. See Redis Sentinel Documentation“”, “127.0.0.1:6379”
maxLenApproxNMaximum number of items inside a stream.The old entries are automatically evicted when the specified length is reached, so that the stream is left at a constant size. Defaults to unlimited.“10000”

Create a Redis instance

Dapr can use any Redis instance - containerized, running on your local dev machine, or a managed cloud service, provided the version of Redis is 5.0.0 or later.

The Dapr CLI will automatically create and setup a Redis Streams instance for you. The Redis instance will be installed via Docker when you run dapr init, and the component file will be created in default directory. ($HOME/.dapr/components directory (Mac/Linux) or %USERPROFILE%\.dapr\components on Windows).

You can use Helm to quickly create a Redis instance in our Kubernetes cluster. This approach requires Installing Helm.

  1. Install Redis into your cluster.

    1. helm repo add bitnami https://charts.bitnami.com/bitnami
    2. helm install redis bitnami/redis

  2. Run kubectl get pods to see the Redis containers now running in your cluster.

  3. Add redis-master:6379 as the redisHost in your redis.yaml file. For example:

    1.     metadata:
    2.     - name: redisHost
    3.       value: redis-master:6379

  4. Next, we’ll get our Redis password, which is slightly different depending on the OS we’re using:

    • Windows: Run kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" > encoded.b64, which will create a file with your encoded password. Next, run certutil -decode encoded.b64 password.txt, which will put your redis password in a text file called password.txt. Copy the password and delete the two files.

    • Linux/MacOS: Run kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" | base64 --decode and copy the outputted password.

    Add this password as the redisPassword value in your redis.yaml file. For example:

    1.     - name: redisPassword
    2.       value: "lhDOkwTlp0"

AWS Redis

GCP Cloud MemoryStore

Azure Redis

Note

The Dapr CLI automatically deploys a local redis instance in self hosted mode as part of the dapr init command.

Last modified February 18, 2022: Update setup-jetstream.md (#2200) (428d8c2)