Create or update a controller

Introduced 2.12

Use this API to create or update a controller for a model. A model may be shared by multiple users. A controller sets rate limits for the number of Predict API calls users can make on the model. A controller consists of a set of rate limiters for different users.

You can only create a controller for a model once you have registered the model and received a model ID.

The POST method creates a new controller. The PUT method updates an existing controller.

To learn how to set rate limits at the model level for all users, see Update Model API. The rate limit is set to either the model-level limit or the user-level limit, whichever is more restrictive. For example, if the model-level limit is 2 requests per minute and the user-level limit is 4 requests per minute, the overall limit will be set to 2 requests per minute.

Path and HTTP methods

  1. POST /_plugins/_ml/controllers/<model_id>
  2. PUT /_plugins/_ml/controllers/<model_id>

copy

Path parameters

The following table lists the available path parameters.

ParameterData typeDescription
model_idStringThe model ID of the model for which you want to set rate limits. Required.

Request fields

The following table lists the available request fields.

FieldData typeRequired/OptionalDescription
user_rate_limiterObjectRequiredLimits the number of times users can call the Predict API on the model. For more information, see Rate limiting inference calls.

The user_rate_limiter object contains an object for each user, specified by username. The user object contains the following fields.

FieldData typeDescription
limitIntegerThe maximum number of times the user can call the Predict API on the model per unit of time. By default, there is no limit on the number of Predict API calls. Once you set a limit, you cannot reset it to no limit. As an alternative, you can specify a high limit value and a small time unit, for example, 1 request per nanosecond.
unitStringThe unit of time for the rate limiter. Valid values are DAYS, HOURS, MICROSECONDS, MILLISECONDS, MINUTES, NANOSECONDS, and SECONDS.

Example request: Create a controller

  1. POST _plugins/_ml/controllers/mtw-ZI0B_1JGmyB068C0
  2. {
  3.   "user_rate_limiter": {
  4.     "user1": {
  5.       "limit": 4,
  6.       "unit": "MINUTES"
  7.     },
  8.     "user2": {
  9.       "limit": 4,
  10.       "unit": "MINUTES"
  11.     }
  12.   }
  13. }

copy

Example response

  1. {
  2.   "model_id": "mtw-ZI0B_1JGmyB068C0",
  3.   "status": "CREATED"
  4. }

Example request: Update the rate limit for one user

To update the limit for user1, send a PUT request and specify the updated information:

  1. PUT _plugins/_ml/controllers/mtw-ZI0B_1JGmyB068C0
  2. {
  3.   "user_rate_limiter": {
  4.     "user1": {
  5.       "limit": 6,
  6.       "unit": "MINUTES"
  7.     }
  8.   }
  9. }

copy

This will update only the user1 object, leaving all other user limits intact:

  1. {
  2.   "model_id": "mtw-ZI0B_1JGmyB068C0",
  3.   "user_rate_limiter": {
  4.     "user1": {
  5.       "limit": "6",
  6.       "unit": "MINUTES"
  7.     },
  8.     "user2": {
  9.       "limit": "4",
  10.       "unit": "MINUTES"
  11.     }
  12.   }
  13. }

Example response

  1. {
  2.   "_index": ".plugins-ml-controller",
  3.   "_id": "mtw-ZI0B_1JGmyB068C0",
  4.   "_version": 2,
  5.   "result": "updated",
  6.   "forced_refresh": true,
  7.   "_shards": {
  8.     "total": 2,
  9.     "successful": 2,
  10.     "failed": 0
  11.   },
  12.   "_seq_no": 1,
  13.   "_primary_term": 1
  14. }

Example request: Delete the rate limit for one user

To delete the limit for user2, send a POST request containing all other users’ limits:

  1. POST _plugins/_ml/controllers/mtw-ZI0B_1JGmyB068C0
  2. {
  3.   "user_rate_limiter": {
  4.     "user1": {
  5.       "limit": 6,
  6.       "unit": "MINUTES"
  7.     }
  8.   }
  9. }

copy

This will overwrite the controller with the new information:

  1. {
  2.   "model_id": "mtw-ZI0B_1JGmyB068C0",
  3.   "user_rate_limiter": {
  4.     "user1": {
  5.       "limit": "6",
  6.       "unit": "MINUTES"
  7.     }
  8.   }
  9. }

Example response

  1. {
  2.   "_index": ".plugins-ml-controller",
  3.   "_id": "mtw-ZI0B_1JGmyB068C0",
  4.   "_version": 2,
  5.   "result": "updated",
  6.   "forced_refresh": true,
  7.   "_shards": {
  8.     "total": 2,
  9.     "successful": 2,
  10.     "failed": 0
  11.   },
  12.   "_seq_no": 1,
  13.   "_primary_term": 1
  14. }

Required permissions

If you use the Security plugin, make sure you have the appropriate permissions: cluster:admin/opensearch/ml/controllers/create and cluster:admin/opensearch/ml/controllers/update.