How-To: Control concurrency and rate limit applications

Learn how to control how many requests and events can invoke your application simultaneously

Typically, in distributed computing, you may only want to allow for a given number of requests to execute concurrently. Using Dapr’s app-max-concurrency, you can control how many requests and events can invoke your application simultaneously.

Default app-max-concurreny is set to -1, meaning no concurrency.

Different approaches

While this guide focuses on app-max-concurrency, you can also limit request rate per second using the middleware.http.ratelimit middleware. However, it’s important to understand the difference between the two approaches:

  • middleware.http.ratelimit: Time bound and limits the number of requests per second
  • app-max-concurrency: Specifies the number of concurrent requests (and events) at any point of time.

See Rate limit middleware for more information about that approach.

Demo

Watch this video on how to control concurrency and rate limiting.

Configure app-max-concurrency

Without using Dapr, you would need to create some sort of a semaphore in the application and take care of acquiring and releasing it.

Using Dapr, you don’t need to make any code changes to your application.

Select how you’d like to configure app-max-concurrency.

To set concurrency limits with the Dapr CLI for running on your local dev machine, add the app-max-concurrency flag:

  1. dapr run --app-max-concurrency 1 --app-port 5000 python ./app.py

The above example effectively turns your app into a single concurrent service.

To configure concurrency limits in Kubernetes, add the following annotation to your pod:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: nodesubscriber
  5.   namespace: default
  6.   labels:
  7.     app: nodesubscriber
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: nodesubscriber
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: nodesubscriber
  17.       annotations:
  18.         dapr.io/enabled: "true"
  19.         dapr.io/app-id: "nodesubscriber"
  20.         dapr.io/app-port: "3000"
  21.         dapr.io/app-max-concurrency: "1"
  22. #...

Limitations

Controlling concurrency on external requests

Rate limiting is guaranteed for every event coming from Dapr, including pub/sub events, direct invocation from other services, bindings events, etc. However, Dapr can’t enforce the concurrency policy on requests that are coming to your app externally.

Arguments and annotations

Next steps

Limit secret store access

Last modified October 11, 2024: Fixed typo (#4389) (fe17926)