Azure OpenAI binding spec

Detailed documentation on the Azure OpenAI binding component

Component format

To setup an Azure OpenAI binding create a component of type bindings.azure.openai. See this guide on how to create and apply a binding configuration. See this for the documentation for Azure OpenAI Service.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: <NAME>
  5. spec:
  6.   type: bindings.azure.openai
  7.   version: v1
  8.   metadata:
  9.   - name: apiKey # Required
  10.     value: "1234567890abcdef"
  11.   - name: endpoint # Required
  12.     value: "https://myopenai.openai.azure.com"

Warning

The above example uses apiKey as a plain string. It is recommended to use a secret store for the secrets as described here.

Spec metadata fields

FieldRequiredBinding supportDetailsExample
endpointYOutputAzure OpenAI service endpoint URL.“https://myopenai.openai.azure.com“
apiKeyYOutputThe access key of the Azure OpenAI service. Only required when not using Microsoft Entra ID authentication.“1234567890abcdef”
azureTenantIdYInputThe tenant ID of the Azure OpenAI resource. Only required when apiKey is not provided.“tenentID”
azureClientIdYInputThe client ID that should be used by the binding to create or update the Azure OpenAI Subscription and to authenticate incoming messages. Only required when apiKey is not provided.“clientId”
azureClientSecretYInputThe client secret that should be used by the binding to create or update the Azure OpenAI Subscription and to authenticate incoming messages. Only required when apiKey is not provided.“clientSecret”

Microsoft Entra ID authentication

The Azure OpenAI binding component supports authentication using all Microsoft Entra ID mechanisms. For further information and the relevant component metadata fields to provide depending on the choice of Microsoft Entra ID authentication mechanism, see the docs for authenticating to Azure.

Example Configuration

  1. apiVersion: dapr.io/v1alpha1
  2. kind: component
  3. metadata:
  4.   name: <NAME>
  5. spec:
  6.   type: bindings.azure.openai
  7.   version: v1
  8.   metadata:
  9.   - name: endpoint
  10.     value: "https://myopenai.openai.azure.com"
  11.   - name: azureTenantId
  12.     value: "***"
  13.   - name: azureClientId
  14.     value: "***"
  15.   - name: azureClientSecret
  16.     value: "***"

Binding support

This component supports output binding with the following operations:

Completion API

To call the completion API with a prompt, invoke the Azure OpenAI binding with a POST method and the following JSON body:

  1. {
  2.   "operation": "completion",
  3.   "data": {
  4.     "deploymentId": "my-model",
  5.     "prompt": "A dog is",
  6.     "maxTokens":5
  7.     }
  8. }

The data parameters are:

  • deploymentId - string that specifies the model deployment ID to use.
  • prompt - string that specifies the prompt to generate completions for.
  • maxTokens - (optional) defines the max number of tokens to generate. Defaults to 16 for completion API.
  • temperature - (optional) defines the sampling temperature between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. Defaults to 1.0 for completion API.
  • topP - (optional) defines the sampling temperature. Defaults to 1.0 for completion API.
  • n - (optional) defines the number of completions to generate. Defaults to 1 for completion API.
  • presencePenalty - (optional) Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Defaults to 0.0 for completion API.
  • frequencyPenalty - (optional) Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Defaults to 0.0 for completion API.

Read more about the importance and usage of these parameters in the Azure OpenAI API documentation.

示例

  1. curl -d '{ "data": {"deploymentId: "my-model" , "prompt": "A dog is ", "maxTokens":15}, "operation": "completion" }' \
  2.       http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

The response body contains the following JSON:

  1. [
  2.   {
  3.     "finish_reason": "length",
  4.     "index": 0,
  5.     "text": " a pig in a dress.\n\nSun, Oct 20, 2013"
  6.   },
  7.   {
  8.     "finish_reason": "length",
  9.     "index": 1,
  10.     "text": " the only thing on earth that loves you\n\nmore than he loves himself.\"\n\n"
  11.   }
  12. ]

Chat Completion API

To perform a chat-completion operation, invoke the Azure OpenAI binding with a POST method and the following JSON body:

  1. {
  2.     "operation": "chat-completion",
  3.     "data": {
  4.         "deploymentId": "my-model",
  5.         "messages": [
  6.             {
  7.                 "role": "system",
  8.                 "message": "You are a bot that gives really short replies"
  9.             },
  10.             {
  11.                 "role": "user",
  12.                 "message": "Tell me a joke"
  13.             }
  14.         ],
  15.         "n": 2,
  16.         "maxTokens": 30,
  17.         "temperature": 1.2
  18.     }
  19. }

The data parameters are:

  • deploymentId - string that specifies the model deployment ID to use.
  • messages - array of messages that will be used to generate chat completions. Each message is of the form:
    • role - string that specifies the role of the message. Can be either user, system or assistant.
    • message - string that specifies the conversation message for the role.
  • maxTokens - (optional) defines the max number of tokens to generate. Defaults to 16 for the chat completion API.
  • temperature - (optional) defines the sampling temperature between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. Defaults to 1.0 for the chat completion API.
  • topP - (optional) defines the sampling temperature. Defaults to 1.0 for the chat completion API.
  • n - (optional) defines the number of completions to generate. Defaults to 1 for the chat completion API.
  • presencePenalty - (optional) Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Defaults to 0.0 for the chat completion API.
  • frequencyPenalty - (optional) Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Defaults to 0.0 for the chat completion API.

Example

  1. curl -d '{
  2.   "data": {
  3.       "deploymentId": "my-model",
  4.       "messages": [
  5.           {
  6.               "role": "system",
  7.               "message": "You are a bot that gives really short replies"
  8.           },
  9.           {
  10.               "role": "user",
  11.               "message": "Tell me a joke"
  12.           }
  13.       ],
  14.       "n": 2,
  15.       "maxTokens": 30,
  16.       "temperature": 1.2
  17.   },
  18.   "operation": "chat-completion"
  19. }' \
  20. http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

The response body contains the following JSON:

  1. [
  2.   {
  3.     "finish_reason": "stop",
  4.     "index": 0,
  5.     "message": {
  6.       "content": "Why was the math book sad? Because it had too many problems.",
  7.       "role": "assistant"
  8.     }
  9.   },
  10.   {
  11.     "finish_reason": "stop",
  12.     "index": 1,
  13.     "message": {
  14.       "content": "Why did the tomato turn red? Because it saw the salad dressing!",
  15.       "role": "assistant"
  16.     }
  17.   }
  18. ]

Get Embedding API

The get-embedding operation returns a vector representation of a given input that can be easily consumed by machine learning models and other algorithms. To perform a get-embedding operation, invoke the Azure OpenAI binding with a POST method and the following JSON body:

  1. {
  2.     "operation": "get-embedding",
  3.     "data": {
  4.         "deploymentId": "my-model",
  5.         "message": "The capital of France is Paris."
  6.     }
  7. }

The data parameters are:

  • deploymentId - string that specifies the model deployment ID to use.
  • message - string that specifies the text to embed.

Example

  1. curl -d '{
  2.   "data": {
  3.       "deploymentId": "embeddings",
  4.       "message": "The capital of France is Paris."
  5.   },
  6.   "operation": "get-embedding"
  7. }' \
  8. http://localhost:<dapr-port>/v1.0/bindings/<binding-name>

Response

The response body contains the following JSON:

  1. [0.018574921,-0.00023652936,-0.0057790717,.... (1536 floats total for ada)]

Learn more about the Azure OpenAI output binding

Watch the following Community Call presentation to learn more about the Azure OpenAI output binding.