Scripts and search speed
Scripts can’t make use of Elasticsearch’s index structures or related optimizations. This can sometimes result in slower search speeds.
If you often use scripts to transform indexed data, you can speed up search by making these changes during ingest instead. However, that often means slower index speeds.
Example
An index, my_test_scores
, contains two long
fields:
math_score
verbal_score
When running searches, users often use a script to sort results by the sum of these two field’s values.
GET /my_test_scores/_search
{
"query": {
"term": {
"grad_year": "2099"
}
},
"sort": [
{
"_script": {
"type": "number",
"script": {
"source": "doc['math_score'].value + doc['verbal_score'].value"
},
"order": "desc"
}
}
]
}
To speed up search, you can perform this calculation during ingest and index the sum to a field instead.
First, add a new field, total_score
, to the index. The total_score
field will contain sum of the math_score
and verbal_score
field values.
PUT /my_test_scores/_mapping
{
"properties": {
"total_score": {
"type": "long"
}
}
}
Next, use an ingest pipeline containing the script
processor to calculate the sum of math_score
and verbal_score
and index it in the total_score
field.
PUT _ingest/pipeline/my_test_scores_pipeline
{
"description": "Calculates the total test score",
"processors": [
{
"script": {
"source": "ctx.total_score = (ctx.math_score + ctx.verbal_score)"
}
}
]
}
To update existing data, use this pipeline to reindex any documents from my_test_scores
to a new index, my_test_scores_2
.
POST /_reindex
{
"source": {
"index": "my_test_scores"
},
"dest": {
"index": "my_test_scores_2",
"pipeline": "my_test_scores_pipeline"
}
}
Continue using the pipeline to index any new documents to my_test_scores_2
.
POST /my_test_scores_2/_doc/?pipeline=my_test_scores_pipeline
{
"student": "kimchy",
"grad_year": "2099",
"math_score": 800,
"verbal_score": 800
}
These changes may slow indexing but allow for faster searches. Users can now sort searches made on my_test_scores_2
using the total_score
field instead of using a script.
GET /my_test_scores_2/_search
{
"query": {
"term": {
"grad_year": "2099"
}
},
"sort": [
{
"total_score": {
"order": "desc"
}
}
]
}
We recommend testing and benchmarking any indexing changes before deploying them in production.
Script errors
Elasticsearch returns error details when there is a compliation or runtime exception. The contents of this response are useful for tracking down the problem.
This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features.
The contents of position
are experimental and subject to change.