ML inference processor

The ml_inference processor is used to generate inferences from machine learning (ML) models connected to the OpenSearch ML Commons plugin. The inferences are added as new fields to the ingested documents.

PREREQUISITE
Before using the ml_inference processor, you must have an ML model connected to your OpenSearch cluster through the ML Commons plugin. For more information, see Connecting to externally hosted models.

Syntax

The following is the syntax for the ml-inference processor:

  1. {
  2.   "ml_inference": {
  3.     "model_id": "<model_id>",
  4.     "input_map": [
  5.       {
  6.         "<model_input>": "<document_field>"
  7.       }
  8.     ],
  9.     "output_map": [
  10.       {
  11.         "<new_document_field>": "<model_output>"
  12.       }
  13.     ],
  14.     "model_config":{
  15.       "<model_config_field>": "<config_value>"
  16.     }
  17.   }
  18. }

copy

Configuration parameters

The following table lists the required and optional parameters for the ml-inference processor.

ParameterData typeRequired/OptionalDescription
model_idStringRequiredThe ID of the ML model connected to the OpenSearch cluster through the ML Commons plugin.
input_mapArrayOptionalAn array specifying how to map fields from the ingested document to the model’s input. If no input mapping is specified, then all fields from the document are used as the model input by default. The input_map size indicates the number of prediction tasks. In each map, the key represents the model input field name, and the value specifies the document field used to construct the model input.
input_map.<input_key>StringOptionalThe model input field name.
input_map.<input_field>StringOptionalThe name of the field from the ingested document to be used as the model’s input.
output_mapArrayOptionalAn array specifying how to map the model’s output to new fields in the ingested document.
output_map.<output_field>StringOptionalThe name of the new field in the ingested document in which the model’s output (specified by the output_key) is stored. If no output mapping is specified, then all fields from the model prediction output are added to the new document field by default.
output_map.<output_key>StringOptionalThe key representing the model output to be stored in the output_field.
model_configObjectOptionalCustom configuration options for the ML model.
max_prediction_tasksIntegerOptionalThe maximum number of concurrent prediction tasks that can run during document ingestion. Default is 10.
descriptionStringOptionalA brief description of the processor.
tagStringOptionalAn identifier tag for the processor. Useful for debugging to distinguish between processors of the same type.
ignore_missingBooleanOptionalIf true and any of the input fields defined in input_map or output_map are missing, then the missing fields are ignored. Otherwise, a missing field causes a failure. Default is false.
ignore_failureBooleanOptionalIf true and any exceptions occur, then they are ignored and ingestion continues. Otherwise, an exception occurs and ingestion is canceled. Default is false.

Using the processor

Follow these steps to use the processor in a pipeline. You must provide a model ID when creating the processor. Only remote models are currently supported. For more information about creating remote models, see Connecting to externally hosted models.

Step 1: Create a pipeline

The following example creates an ingest pipeline where the model requires an input field, produces a data field for prediction, and converts the passage_text field into text embeddings stored in the passage_embedding field. The input_map and output_map mappings support standard JSON path notation for complex data structures.

  2. PUT /_ingest/pipeline/ml_inference_pipeline
  3. {
  4.   "description": "Generate passage_embedding for ingested documents",
  5.   "processors": [
  6.     {
  7.       "ml_inference": {
  8.         "model_id": "<your model id>",
  9.         "input_map": [
  10.           {
  11.             "input": "passage_text"
  12.           }
  13.         ],
  14.         "output_map": [
  15.           {
  16.             "passage_embedding": "data"
  17.           }
  18.         ]
  19.       }
  20.     }
  21.   ]
  22. }

copy

Step 2 (Optional): Test the pipeline

It is recommended that you test your pipeline before you ingest documents.

To test the pipeline, run the following query:

  1. POST _ingest/pipeline/ml_inference_pipeline/_simulate
  2. {
  3.   "docs": [
  4.     {
  5.       "_index": "testindex1",
  6.       "_id": "1",
  7.       "_source":{
  8.          "passage_text": "hello world"
  9.       }
  10.     }
  11.   ]
  12. }

copy

Response

The response confirms that, in addition to the passage_text field, the processor has generated text embeddings in the passage_embedding field:

  1. {
  2.   "docs" : [
  3.     {
  4.       "doc" : {
  5.         "_index" : "testindex1",
  6.         "_id" : "1",
  7.         "_source" : {
  8.           "passage_embedding" : [
  9.             0.017304314,
  10.             -0.021530833,
  11.             0.050184276,
  12.             0.08962978,
  13.             ...
  14.           ],
  15.           "passage_text" : "hello world"
  16.         },
  17.         "_ingest" : {
  18.           "timestamp" : "2023-10-11T22:35:53.654650086Z"
  19.         }
  20.       }
  21.     }
  22.   ]
  23. }

Once you have created an ingest pipeline, you need to create an index for ingestion and ingest documents into the index.

Limitation

The ml_inference processor currently supports only remote models connected through a connector. Local models uploaded to an OpenSearch cluster are not yet supported. Check the OpenSearch Project Roadmap for updates on upcoming features.