Azure Event Hubs

Detailed documentation on the Azure Event Hubs pubsub component

Component format

To setup Azure Event Hubs pubsub create a component of type pubsub.azure.eventhubs. See this guide on how to create and apply a pubsub configuration.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: eventhubs-pubsub
  5.   namespace: default
  6. spec:
  7.   type: pubsub.azure.eventhubs
  8.   version: v1
  9.   metadata:
  10.   - name: connectionString
  11.     value: "Endpoint=sb://{EventHubNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={EventHub}"
  12.   - name: storageAccountName
  13.     value: "myeventhubstorage"
  14.   - name: storageAccountKey
  15.     value: "112233445566778899"
  16.   - name: storageContainerName
  17.     value: "myeventhubstoragecontainer"

Warning

The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described here.

Spec metadata fields

FieldRequiredDetailsExample
connectionStringYConnection-string for the Event Hubs“Endpoint=sb://{EventHubNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={EventHub}”
storageAccountNameYStorage account name to use for the EventProcessorHost“myeventhubstorage”
storageAccountKeyYStorage account key to use for the EventProcessorHost. Can be secretKeyRef to use a secret reference“112233445566778899”
storageContainerNameYStorage container name for the storage account name.“myeventhubstoragecontainer”

Create an Azure Event Hub

Follow the instructions here on setting up Azure Event Hubs. Since this implementation uses the Event Processor Host, you will also need an Azure Storage Account. Follow the instructions here to manage the storage account access keys.

See here on how to get the Event Hubs connection string. Note this is not the Event Hubs namespace.

Create consumer groups for each subscriber

For every Dapr app that wants to subscribe to events, create an Event Hubs consumer group with the name of the dapr id. For example, a Dapr app running on Kubernetes with dapr.io/app-id: "myapp" will need an Event Hubs consumer group named myapp.

Note: Dapr passes the name of the Consumer group to the EventHub and so this is not supplied in the metadata.

Subscribing to Azure IoT Hub Events

Azure IoT Hub provides an endpoint that is compatible with Event Hubs, so the Azure Event Hubs pubsub component can also be used to subscribe to Azure IoT Hub events.

The device-to-cloud events created by Azure IoT Hub devices will contain additional IoT Hub System Properties, and the Azure Event Hubs pubsub component for Dapr will return the following as part of the response metadata:

System Property NameDescription
iothub-connection-auth-generation-idThe connectionDeviceGenerationId of the device that sent the message. See IoT Hub device identity properties.
iothub-connection-auth-methodThe authentication method used to authenticate the device that sent the message.
iothub-connection-device-idThe deviceId of the device that sent the message. See IoT Hub device identity properties.
iothub-connection-module-idThe moduleId of the device that sent the message. See IoT Hub device identity properties.
iothub-enqueuedtimeThe date and time in RFC3339 format that the device-to-cloud message was received by IoT Hub.

For example, the headers of a delivered HTTP subscription message would contain:

  1. {
  2.   'user-agent': 'fasthttp',
  3.   'host': '127.0.0.1:3000',
  4.   'content-type': 'application/json',
  5.   'content-length': '120',
  6.   'iothub-connection-device-id': 'my-test-device',
  7.   'iothub-connection-auth-generation-id': '637618061680407492',
  8.   'iothub-connection-auth-method': '{"scope":"module","type":"sas","issuer":"iothub","acceptingIpFilterRule":null}',
  9.   'iothub-connection-module-id': 'my-test-module-a',
  10.   'iothub-enqueuedtime': '2021-07-13T22:08:09Z',
  11.   'x-opt-sequence-number': '35',
  12.   'x-opt-enqueued-time': '2021-07-13T22:08:09Z',
  13.   'x-opt-offset': '21560',
  14.   'traceparent': '00-4655608164bc48b985b42d39865f3834-ed6cf3697c86e7bd-01'
  15. }

Last modified September 20, 2021 : Merge pull request #1800 from greenie-msft/gRPC_proxying_video (36dff3c)