Simulate pipeline

Introduced 1.0

Use the simulate ingest pipeline API operation to run or test the pipeline.

Path and HTTP methods

The following requests simulate the latest ingest pipeline created:

  1. GET _ingest/pipeline/_simulate
  2. POST _ingest/pipeline/_simulate

copy

The following requests simulate a single pipeline based on the pipeline ID:

  1. GET _ingest/pipeline/<pipeline-id>/_simulate
  2. POST _ingest/pipeline/<pipeline-id>/_simulate

copy

Request body fields

The following table lists the request body fields used to run a pipeline.

FieldRequiredTypeDescription
docsRequiredArrayThe documents to be used to test the pipeline.
pipelineOptionalObjectThe pipeline to be simulated. If the pipeline identifier is not included, then the response simulates the latest pipeline created.

The docs field can include subfields listed in the following table.

FieldRequiredTypeDescription
sourceRequiredObjectThe document’s JSON body.
idOptionalStringA unique document identifier. The identifier cannot be used elsewhere in the index.
indexOptionalStringThe index where the document’s transformed data appears.

Query parameters

The following table lists the query parameters for running a pipeline.

ParameterTypeDescription
verboseBooleanVerbose mode. Display data output for each processor in the executed pipeline.

Example: Specify a pipeline in the path

  1. POST /_ingest/pipeline/my-pipeline/_simulate
  2. {
  3.   "docs": [
  4.     {
  5.       "_index": "my-index",
  6.       "_id": "1",
  7.       "_source": {
  8.         "grad_year": 2024,
  9.         "graduated": false,
  10.         "name": "John Doe"
  11.       }
  12.     },
  13.     {
  14.       "_index": "my-index",
  15.       "_id": "2",
  16.       "_source": {
  17.         "grad_year": 2025,
  18.         "graduated": false,
  19.         "name": "Jane Doe"
  20.       }
  21.     }
  22.   ]
  23. }

copy

The request returns the following response:

  1. {
  2.   "docs": [
  3.     {
  4.       "doc": {
  5.         "_index": "my-index",
  6.         "_id": "1",
  7.         "_source": {
  8.           "name": "JOHN DOE",
  9.           "grad_year": 2023,
  10.           "graduated": true
  11.         },
  12.         "_ingest": {
  13.           "timestamp": "2023-06-20T23:19:54.635306588Z"
  14.         }
  15.       }
  16.     },
  17.     {
  18.       "doc": {
  19.         "_index": "my-index",
  20.         "_id": "2",
  21.         "_source": {
  22.           "name": "JANE DOE",
  23.           "grad_year": 2023,
  24.           "graduated": true
  25.         },
  26.         "_ingest": {
  27.           "timestamp": "2023-06-20T23:19:54.635746046Z"
  28.         }
  29.       }
  30.     }
  31.   ]
  32. }

Example: Verbose mode

When the previous request is run with the verbose parameter set to true, the response shows the sequence of transformations for each document. For example, for the document with the ID 1, the response contains the results of applying each processor in the pipeline in sequence:

  1. {
  2.   "docs": [
  3.     {
  4.       "processor_results": [
  5.         {
  6.           "processor_type": "set",
  7.           "status": "success",
  8.           "description": "Sets the graduation year to 2023",
  9.           "doc": {
  10.             "_index": "my-index",
  11.             "_id": "1",
  12.             "_source": {
  13.               "name": "John Doe",
  14.               "grad_year": 2023,
  15.               "graduated": false
  16.             },
  17.             "_ingest": {
  18.               "pipeline": "my-pipeline",
  19.               "timestamp": "2023-06-20T23:23:26.656564631Z"
  20.             }
  21.           }
  22.         },
  23.         {
  24.           "processor_type": "set",
  25.           "status": "success",
  26.           "description": "Sets 'graduated' to true",
  27.           "doc": {
  28.             "_index": "my-index",
  29.             "_id": "1",
  30.             "_source": {
  31.               "name": "John Doe",
  32.               "grad_year": 2023,
  33.               "graduated": true
  34.             },
  35.             "_ingest": {
  36.               "pipeline": "my-pipeline",
  37.               "timestamp": "2023-06-20T23:23:26.656564631Z"
  38.             }
  39.           }
  40.         },
  41.         {
  42.           "processor_type": "uppercase",
  43.           "status": "success",
  44.           "doc": {
  45.             "_index": "my-index",
  46.             "_id": "1",
  47.             "_source": {
  48.               "name": "JOHN DOE",
  49.               "grad_year": 2023,
  50.               "graduated": true
  51.             },
  52.             "_ingest": {
  53.               "pipeline": "my-pipeline",
  54.               "timestamp": "2023-06-20T23:23:26.656564631Z"
  55.             }
  56.           }
  57.         }
  58.       ]
  59.     }
  60.   ]
  61. }

Example: Specify a pipeline in the request body

Alternatively, you can specify a pipeline directly in the request body without first creating a pipeline:

  1. POST /_ingest/pipeline/_simulate
  2. {
  3.   "pipeline" :
  4.   {
  5.     "description": "Splits text on white space characters",
  6.     "processors": [
  7.       {
  8.         "csv" : {
  9.           "field" : "name",
  10.           "separator": ",",
  11.           "target_fields": ["last_name", "first_name"],
  12.           "trim": true
  13.         }
  14.       },
  15.       {
  16.       "uppercase": {
  17.         "field": "last_name"
  18.       }
  19.     }
  20.     ]
  21.   },
  22.   "docs": [
  23.     {
  24.       "_index": "second-index",
  25.       "_id": "1",
  26.       "_source": {
  27.         "name": "Doe,John"
  28.       }
  29.     },
  30.     {
  31.       "_index": "second-index",
  32.       "_id": "2",
  33.       "_source": {
  34.         "name": "Doe, Jane"
  35.       }
  36.     }
  37.   ]
  38. }

copy

Response

The request returns the following response:

  1. {
  2.   "docs": [
  3.     {
  4.       "doc": {
  5.         "_index": "second-index",
  6.         "_id": "1",
  7.         "_source": {
  8.           "name": "Doe,John",
  9.           "last_name": "DOE",
  10.           "first_name": "John"
  11.         },
  12.         "_ingest": {
  13.           "timestamp": "2023-08-24T19:20:44.816219673Z"
  14.         }
  15.       }
  16.     },
  17.     {
  18.       "doc": {
  19.         "_index": "second-index",
  20.         "_id": "2",
  21.         "_source": {
  22.           "name": "Doe, Jane",
  23.           "last_name": "DOE",
  24.           "first_name": "Jane"
  25.         },
  26.         "_ingest": {
  27.           "timestamp": "2023-08-24T19:20:44.816492381Z"
  28.         }
  29.       }
  30.     }
  31.   ]
  32. }