Uploading trained models

While model training occurs outside of the Learning to Rank plugin, you can use the plugin for logging feature scores. After you have trained a model, you can upload it to the plugin in the available serialization formats, such as RankLib and XGBoost.

RankLib model training

The feature logging process generates a RankLib-comsumable judgment file. In the following judgment file, the query with ID 1 rambo includes the logged features 1 (a title TF*IDF score) and 2 (a description TF*IDF score) for a set of documents:

  1. 4   qid:1   1:9.8376875 2:12.318446 # 7555  rambo
  2. 3   qid:1   1:10.7808075    2:9.510193 # 1370   rambo
  3. 3   qid:1   1:10.7808075    2:6.8449354 # 1369  rambo
  4. 3   qid:1   1:10.7808075    2:0.0 # 1368    rambo

The RankLib library can be called using the following command:

  1.  cmd = "java -jar RankLib-2.8.jar -ranker %s -train%rs -save %s -frate 1.0" % (whichModel, judgmentsWithFeaturesFile, modelOutput)

The judgmentsWithFeatureFile is the input provided to RankLib for training. Additional parameters can be passed. See the RankLib documentation for more information.

RankLib outputs the model in its own serialization format. As shown in the following example, a LambdaMART model is an ensemble of regression trees:

  1. ## LambdaMART
  2. ## No. of trees = 1000
  3. ## No. of leaves = 10
  4. ## No. of threshold candidates = 256
  5. ## Learning rate = 0.1
  6. ## Stop early = 100
  8.     <ensemble>
  9.        <tree id="1" weight="0.1">
  10.            <split>
  11.                <feature> 2 </feature>
  12.                ...

Within the RankLib model, each tree in the ensemble examines feature values, makes decisions based on these feature values, and outputs the relevance scores. The features are referred to by their ordinal position, starting from 1, which corresponds to the 0th feature in the original feature set. RankLib does not use feature names during model training.

XGBoost model training

Unlike the RankLib model, the XGBoost model is serialized in a format specific to gradient-boosted decision trees, as shown in the following example:

  1.     [  { "nodeid": 0, "depth": 0, "split": "tmdb_multi", "split_condition": 11.2009, "yes": 1, "no": 2, "missing": 1, "children": [
  2.         { "nodeid": 1, "depth": 1, "split": "tmdb_title", "split_condition": 2.20631, "yes": 3, "no": 4, "missing": 3, "children": [
  3.         { "nodeid": 3, "leaf": -0.03125 },
  4.         ...

XGBoost parameters

Optional parameters can be specified for an XGBoost model. These parameters are specified as an object, with the decision trees specified in the splits field. The supported parameters include objective, which defines the model learning objective as described in the XGBoost documentation. This parameter can transform the final model prediction. The supported values include binary:logistic, binary:logitraw, rank:ndcg, rank:map, rank:pairwise, reg:linear, and reg:logistic.

Simple linear models

Machine learning (ML) models, such as Support Vector Machines (SVMs), output linear weights for each feature. The LTR model supports representing these linear weights in a simple format, such as those learned from an SVM or linear regression model. In the following example output, the weights indicate the relative importance of the features in the model’s prediction:

  1. {
  2.     "title_query" : 0.3,
  3.     "body_query" : 0.5,
  4.     "recency" : 0.1
  5. }

Feature normalization

Feature normalization is used to convert feature values to a consistent range, typically between 0 and 1 or -1 and 1. This is done during the training phase to better understand the relative impact of each feature. Some models, especially linear ones such as SVMRank, rely on normalization to function correctly.

Model upload process

After training your model, the next step is to make it available for search operations. This involves uploading the model to the Learning to Rank plugin. When uploading a model, you must provide the following information:

  • Feature set used during training
  • Model type, for example, RankLib or XGBoost
  • Model content

The following example request shows how to upload a RankLib model that was trained using the more_movie_features feature set:

  1.     POST _ltr/_featureset/more_movie_features/_createmodel
  2.     {
  3.         "model": {
  4.             "name": "my_ranklib_model",
  5.             "model": {
  6.                 "type": "model/ranklib",
  7.                 "definition": "## LambdaMART\n
  8.                                 ## No. of trees = 1000
  9.                                 ## No. of leaves = 10
  10.                                 ## No. of threshold candidates = 256
  11.                                 ## Learning rate = 0.1
  12.                                 ## Stop early = 100
  14.                                 <ensemble>
  15.                                     <tree id="1" weight="0.1">
  16.                                         <split>
  17.                                             <feature> 2 </feature>
  18.                                             ...
  19.                             "
  20.             }
  21.         }
  22.     }

The following example request shows how to upload an XGBoost model that was trained using the more_movie_features feature set:

  1.     POST _ltr/_featureset/more_movie_features/_createmodel
  2.     {
  3.         "model": {
  4.             "name": "my_xgboost_model",
  5.             "model": {
  6.                 "type": "model/xgboost+json",
  7.                 "definition": "[  { \"nodeid\": 0, \"depth\": 0, \"split\": \"tmdb_multi\", \"split_condition\": 11.2009, \"yes\": 1, \"no\": 2, \"missing\": 1, \"children\": [
  8.                                     { \"nodeid\": 1, \"depth\": 1, \"split\": \"tmdb_title\", \"split_condition\": 2.20631, \"yes\": 3, \"no\": 4, \"missing\": 3, \"children\": [
  9.                                       { \"nodeid\": 3, \"leaf\": -0.03125 },
  10.                                     ..."
  11.             }
  12.         }
  13.     }

The following example request shows how to upload an XGBoost model that was trained using the more_movie_features feature set with parameters:

  1.     POST _ltr/_featureset/more_movie_features/_createmodel
  2.     {
  3.         "model": {
  4.             "name": "my_xgboost_model",
  5.             "model": {
  6.                 "type": "model/xgboost+json",
  7.                 "definition": "{
  8.                                  \"objective\": \"reg:logistic\",
  9.                                  \"splits\": [  { \"nodeid\": 0, \"depth\": 0, \"split\": \"tmdb_multi\", \"split_condition\": 11.2009, \"yes\": 1, \"no\": 2, \"missing\": 1, \"children\": [
  10.                                                   { \"nodeid\": 1, \"depth\": 1, \"split\": \"tmdb_title\", \"split_condition\": 2.20631, \"yes\": 3, \"no\": 4, \"missing\": 3, \"children\": [
  11.                                                     { \"nodeid\": 3, \"leaf\": -0.03125 },
  12.                                                   ...
  13.                                              ]
  14.                                }"
  15.             }
  16.         }
  17.     }

The following example request shows how to upload a simple linear model that was trained using the more_movie_features feature set:

  1.     POST _ltr/_featureset/more_movie_features/_createmodel
  2.     {
  3.         "model": {
  4.             "name": "my_linear_model",
  5.             "model": {
  6.                 "type": "model/linear",
  7.                 "definition": """
  8.                                 {
  9.                                     "title_query" : 0.3,
  10.                                     "body_query" : 0.5,
  11.                                     "recency" : 0.1
  12.                                 }
  13.                             """
  14.             }
  15.         }
  16.     }

Creating a model with feature normalization

Feature normalization is a crucial preprocessing step that can be applied before model evaluation. LTR supports two types of feature normalization: min-max and standard normalization.

Standard normalization

Standard normalization transforms features as follows:

  • Maps the mean value to 0
  • Maps one standard deviation above the mean to 1
  • Maps one standard deviation below the mean to -1

The following example request shows how to create a model with standard feature normalization:

  1.     POST _ltr/_featureset/more_movie_features/_createmodel
  2.     {
  3.         "model": {
  4.             "name": "my_linear_model",
  5.             "model": {
  6.                 "type": "model/linear",
  7.                 "feature_normalizers": {
  8.                                "release_year": {
  9.                                   "standard": {
  10.                                     "mean": 1970,
  11.                                     "standard_deviation": 30
  12.                                   }
  13.                                }
  14.                             },
  15.                 "definition": """
  16.                                 {
  17.                                     "release_year" : 0.3,
  18.                                     "body_query" : 0.5,
  19.                                     "recency" : 0.1
  20.                                 }
  21.                             """
  22.             }
  23.         }
  24.     }

Min-max normalization

Min-max normalization scales features to a fixed range, typically between 0 and 1. Min-max normalization transforms features as follows:

  • Maps the specified minimum value to 0
  • Maps the specified maximum value to 1
  • Scales the values between 0 and 1 linearly

The following example request shows how to implement min-max normalization:

  1.     "feature_normalizers": {
  2.         "vote_average": {
  3.             "min_max": {
  4.                 "minimum": 0,
  5.                 "maximum": 10
  6.             }
  7.         }
  8.     }

Model independence from feature sets

Models are initially created with reference to a feature set. After their creation, they exist as independent top-level entities.

Accessing models

To retrieve a model, use a GET request:

  1. GET _ltr/_model/my_linear_model

To delete a model, use a DELETE request:

  1. DELETE _ltr/_model/my_linear_model

Model names must be globally unique across all feature sets.

Model persistence

When a model is created, its features are copied. This prevents changes to the original features from affecting existing models or model production. For example, if the feature set used to create the model is deleted, you can still access and use the model.

Model response

When retrieving a model, you receive a response that includes the features used to create it, as shown in the following example:

  1.     {
  2.     "_index": ".ltrstore",
  3.     "_type": "store",
  4.     "_id": "model-my_linear_model",
  5.     "_version": 1,
  6.     "found": true,
  7.     "_source": {
  8.         "name": "my_linear_model",
  9.         "type": "model",
  10.         "model": {
  11.             "name": "my_linear_model",
  12.             "feature_set": {
  13.                 "name": "more_movie_features",
  14.                 "features": [
  15.                 {
  16.                     "name": "body_query",
  17.                     "params": [
  18.                         "keywords"
  19.                         ],
  20.                      "template": {
  21.                         "match": {
  22.                             "overview": ""
  23.                         }
  24.                     }
  25.                 },
  26.                 {
  27.                     "name": "title_query",
  28.                     "params": [
  29.                         "keywords"
  30.                     ],
  31.                     "template": {
  32.                         "match": {
  33.                             "title": ""
  34.                         }
  35.                     }
  36.                 }
  37.         ]}}}

Next steps

Learn about searching with LTR.