Deploy a model

The deploy model operation reads the model’s chunks from the model index and then creates an instance of the model to cache in memory. This operation requires the model_id.

Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set plugins.ml_commons.model_auto_deploy.enable to false:

  1. PUT _cluster/settings
  2. {
  3.   "persistent": {
  4.     "plugins.ml_commons.model_auto_deploy.enable": "false"
  5.   }
  6. }

copy

For information about user access for this API, see Model access control considerations.

Path and HTTP methods

  1. POST /_plugins/_ml/models/<model_id>/_deploy

Example request: Deploying to all available ML nodes

In this example request, OpenSearch deploys the model to any available OpenSearch ML node:

  1. POST /_plugins/_ml/models/WWQI44MBbzI2oUKAvNUt/_deploy

copy

Example request: Deploying to a specific node

If you want to reserve the memory of other ML nodes within your cluster, you can deploy your model to a specific node(s) by specifying the node_ids in the request body:

  1. POST /_plugins/_ml/models/WWQI44MBbzI2oUKAvNUt/_deploy
  2. {
  3.     "node_ids": ["4PLK7KJWReyX0oWKnBA8nA"]
  4. }

copy

Example response

  1. {
  2.   "task_id" : "hA8P44MBhyWuIwnfvTKP",
  3.   "status" : "DEPLOYING"
  4. }

Check the status of model deployment

To see the status of your model deployment and retrieve the model ID created for the new model version, pass the task_id as a path parameter to the Tasks API:

  1. GET /_plugins/_ml/tasks/hA8P44MBhyWuIwnfvTKP

copy

The response contains the model ID of the model version:

  1. {
  2.   "model_id": "Qr1YbogBYOqeeqR7sI9L",
  3.   "task_type": "DEPLOY_MODEL",
  4.   "function_name": "TEXT_EMBEDDING",
  5.   "state": "COMPLETED",
  6.   "worker_node": [
  7.     "N77RInqjTSq_UaLh1k0BUg"
  8.   ],
  9.   "create_time": 1685478486057,
  10.   "last_update_time": 1685478491090,
  11.   "is_async": true
  12. }

If a cluster or node is restarted, then you need to redeploy the model. To learn how to set up automatic redeployment, see Enable auto redeploy.