Configure KEDA Autoscaling of Knative Kafka Resources

All of the Knative Kafka components which dispatch events (source, channel broker) support scaling the dispatcher data plane with KEDA.

Important

This feature is Alpha

Enable Autoscaling of Kafka components

Important

Note that enabling autoscaling will enable scaling for all KafkaSources, Triggers referencing Kafka Brokers, and Subscriptions referencing KafkaChannels

To enable the feature, you need to modify the config-kafka-features configmap and set the controller-autoscaler-keda flag to enabled. Note that in order for this feature to work, you must also install KEDA into your cluster.

After your changes your configmap should look like:

  1. apiVersion: v1
  2. kind: ConfigMap
  3. metadata:
  4.   name: config-kafka-features
  5.   namespace: knative-eventing
  6. data:
  7.   controller-autoscaler-keda: enabled
  8.   # other features and config options...

Configure Autoscaling for a Resource

If you want to customize how KEDA scales a KafkaSource, trigger, or subscription you can set annotations on the resource:

  1. apiVersion: <eventing|messaging|sources>.knative.dev/v1
  2. kind: <KafkaSource|Subscription|Trigger>
  3. metadata:
  4.   annotations:
  5.     # The minimum number of replicas to scale down to
  6.     autoscaling.eventing.knative.dev/min-scale: "0"
  7.     # The maximum number of replicas to scale up to
  8.     autoscaling.eventing.knative.dev/max-scale: "50"
  9.     # The interval in seconds the autoscaler uses to poll metrics in order to inform its scaling decisions
  10.     autoscaling.eventing.knative.dev/polling-interval: "10"
  11.     # The period in seconds the autoscaler waits until it scales down
  12.     autoscaling.eventing.knative.dev/cooldown-period: "30"
  13.     # The lag that is used for scaling (1<->N)
  14.     autoscaling.eventing.knative.dev/lag-threshold: "100"
  15.     # The lag that is used for activation (0<->1)
  16.     autoscaling.eventing.knative.dev: "0"
  17. spec:
  18.   # Spec fields for the resource...

Disable Autoscaling for a Resource

If you want to disable autoscaling for a KafkaSource, trigger, or subscription, you need to set an annotation on the resource:

  1. apiVersion: <eventing|messaging|sources>.knative.dev/v1
  2. kind: <KafkaSource|Subscription|Trigger>
  3. metadata:
  4.   annotations:
  5.     autoscaling.eventing.knative.dev/class: disabled
  6. spec:
  7.   # Spec fields for the resource...