Search using Learning To Rank

Search using Learning To Rank

This feature was introduced in version 8.12.0 and is only available to certain subscription levels. For more information, see https://www.elastic.co/subscriptions.

Learning To Rank as a rescorer

Once your LTR model is trained and deployed in Elasticsearch, it can be used as a rescorer in the search API:

  1. resp = client.search(
  2.     index="my-index",
  3.     query={
  4.         "multi_match": {
  5.             "fields": [
  6.                 "title",
  7.                 "content"
  8.             ],
  9.             "query": "the quick brown fox"
  10.         }
  11.     },
  12.     rescore={
  13.         "learning_to_rank": {
  14.             "model_id": "ltr-model",
  15.             "params": {
  16.                 "query_text": "the quick brown fox"
  17.             }
  18.         },
  19.         "window_size": 100
  20.     },
  21. )
  22. print(resp)
  1. response = client.search(
  2.   index: 'my-index',
  3.   body: {
  4.     query: {
  5.       multi_match: {
  6.         fields: [
  7.           'title',
  8.           'content'
  9.         ],
  10.         query: 'the quick brown fox'
  11.       }
  12.     },
  13.     rescore: {
  14.       learning_to_rank: {
  15.         model_id: 'ltr-model',
  16.         params: {
  17.           query_text: 'the quick brown fox'
  18.         }
  19.       },
  20.       window_size: 100
  21.     }
  22.   }
  23. )
  24. puts response
  1. const response = await client.search({
  2.   index: "my-index",
  3.   query: {
  4.     multi_match: {
  5.       fields: ["title", "content"],
  6.       query: "the quick brown fox",
  7.     },
  8.   },
  9.   rescore: {
  10.     learning_to_rank: {
  11.       model_id: "ltr-model",
  12.       params: {
  13.         query_text: "the quick brown fox",
  14.       },
  15.     },
  16.     window_size: 100,
  17.   },
  18. });
  19. console.log(response);
  1. GET my-index/_search
  2. {
  3.   "query": { 
  4.     "multi_match": {
  5.       "fields": ["title", "content"],
  6.       "query": "the quick brown fox"
  7.     }
  8.   },
  9.   "rescore": {
  10.     "learning_to_rank": {
  11.       "model_id": "ltr-model", 
  12.       "params": { 
  13.         "query_text": "the quick brown fox"
  14.       }
  15.     },
  16.     "window_size": 100 
  17.   }
  18. }

First pass query providing documents to be rescored.

The unique identifier of the trained model uploaded to Elasticsearch.

Named parameters to be passed to the query templates used for feature.

The number of documents that should be examined by the rescorer on each shard.

Known limitations
Rescore window size

Scores returned by LTR models are usually not comparable with the scores issued by the first pass query and can be lower than the non-rescored score. This can cause the non-rescored result document to be ranked higher than the rescored document. To prevent this, the window_size parameter is mandatory for LTR rescorers and should be greater than or equal to from + size.

Pagination

When exposing pagination to users, window_size should remain constant as each page is progressed by passing different from values. Changing the window_size can alter the top hits causing results to confusingly shift as the user steps through pages.

Negative scores

Depending on how your model is trained, it’s possible that the model will return negative scores for documents. While negative scores are not allowed from first-stage retrieval and ranking, it is possible to use them in the LTR rescorer.