How-To: Control concurrency and rate limit applications

Control how many requests and events will invoke your application simultaneously

A common scenario in distributed computing is to only allow for a given number of requests to execute concurrently. Using Dapr, you can control how many requests and events will invoke your application simultaneously.

Note that this rate limiting is guaranteed for every event that’s coming from Dapr, meaning Pub/Sub events, direct invocation from other services, bindings events etc. Dapr can’t enforce the concurrency policy on requests that are coming to your app externally.

*Note that rate limiting per second can be achieved by using the middleware.http.ratelimit middleware. However, there is an imporant difference between the two approaches. The rate limit middlware is time bound and limits the number of requests per second, while the app-max-concurrency flag specifies the number of concurrent requests (and events) at any point of time. See Rate limit middleware. *

Watch this video on how to control concurrency and rate limiting “.

Setting app-max-concurrency

Without using Dapr, a developer would need to create some sort of a semaphore in the application and take care of acquiring and releasing it. Using Dapr, there are no code changes needed to an app.

Setting app-max-concurrency in Kubernetes

To set app-max-concurrency in Kubernetes, add the following annotation to your pod:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: nodesubscriber
  5.   namespace: default
  6.   labels:
  7.     app: nodesubscriber
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: nodesubscriber
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: nodesubscriber
  17.       annotations:
  18.         dapr.io/enabled: "true"
  19.         dapr.io/app-id: "nodesubscriber"
  20.         dapr.io/app-port: "3000"
  21.         dapr.io/app-max-concurrency: "1"
  22. ...

Setting app-max-concurrency using the Dapr CLI

To set app-max-concurrency with the Dapr CLI for running on your local dev machine, add the app-max-concurrency flag:

  1. dapr run --app-max-concurrency 1 --app-port 5000 python ./app.py

The above examples will effectively turn your app into a single concurrent service.

Last modified October 12, 2023: Update config.toml (#3826) (0ffc2e7)