Workflow templates

OpenSearch provides several workflow templates for some common machine learning (ML) use cases. Using a template simplifies complex setups and provides many default values for use cases like semantic or conversational search.

You can specify a workflow template when you call the Create Workflow API:

• To use an OpenSearch-provided workflow template, specify the template use case as the use_case query parameter (see the Example). For a list of OpenSearch-provided templates, see Supported workflow templates.

• To use a custom workflow template, provide the complete template in the request body. For an example of a custom template, see an example JSON template or an example YAML template.

To provision the workflow, specify provision=true as a query parameter.

Example

In this example, you’ll configure the semantic_search_with_cohere_embedding_query_enricher workflow template. The workflow created using this template performs the following configuration steps:

• Deploys an externally hosted Cohere model
• Creates an ingest pipeline using the model
• Creates a sample k-NN index and configures a search pipeline to define the default model ID for that index

Step 1: Create and provision the workflow

Send the following request to create and provision a workflow using the semantic_search_with_cohere_embedding_query_enricher workflow template. The only required request body field for this template is the API key for the Cohere Embed model:

POST /_plugins/_flow_framework/workflow?use_case=semantic_search_with_cohere_embedding_query_enricher&provision=true
{
    "create_connector.credential.key" : "<YOUR API KEY>"
}

copy

OpenSearch responds with a workflow ID for the created workflow:

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50"
}

The workflow in the previous step creates a default k-NN index. The default index name is my-nlp-index:

{
  "create_index.name": "my-nlp-index"
}

For all default parameter values for this workflow template, see Cohere Embed semantic search defaults.

Step 2: Ingest documents into the index

To ingest documents into the index created in the previous step, send the following request:

PUT /my-nlp-index/_doc/1
{
  "passage_text": "Hello world",
  "id": "s1"
}

copy

Step 3: Perform vector search

To perform a vector search on your index, use a neural query clause:

GET /my-nlp-index/_search
{
  "_source": {
    "excludes": [
      "passage_embedding"
    ]
  },
  "query": {
    "neural": {
      "passage_embedding": {
        "query_text": "Hi world",
        "k": 100
      }
    }
  }
}

copy

Parameters

Each workflow template has a defined schema and a set of APIs with predefined default values for each step. For more information about template parameter defaults, see Supported workflow templates.

Overriding default values

To override a template’s default values, provide the new values in the request body when sending a create workflow request. For example, the following request changes the Cohere model, the name of the text_embedding processor output field, and the name of the sparse index of the semantic_search_with_cohere_embedding template:

POST /_plugins/_flow_framework/workflow?use_case=semantic_search_with_cohere_embedding
{
    "create_connector.model" : "embed-multilingual-v3.0",
    "text_embedding.field_map.output": "book_embedding",
    "create_index.name": "sparse-book-index"
}

copy

Viewing workflow resources

The workflow you created provisioned all the necessary resources for semantic search. To view the provisioned resources, call the Get Workflow Status API and provide the workflowID for your workflow:

GET /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_status

copy

Supported workflow templates

The following table lists the supported workflow templates. To use a workflow template, specify it in the use_case query parameter when creating a workflow.