Agents and tools

Introduced 2.12

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated GitHub issue.

You can automate machine learning (ML) tasks using agents and tools. An agent orchestrates and runs ML models and tools. A tool performs a set of specific tasks. Some examples of tools are the VectorDBTool, which supports vector search, and the CATIndexTool, which executes the cat indices operation. For a list of supported tools, see Tools.

Agents

An agent is a coordinator that uses a large language model (LLM) to solve a problem. After the LLM reasons and decides what action to take, the agent coordinates the action execution. OpenSearch supports the following agent types:

  • Flow agent: Runs tools sequentially, in the order specified in its configuration. The workflow of a flow agent is fixed. Useful for retrieval-augmented generation (RAG).
  • Conversational flow agent: Runs tools sequentially, in the order specified in its configuration. The workflow of a conversational flow agent is fixed. Stores conversation history so that users can ask follow-up questions. Useful for creating a chatbot.
  • Conversational agent: Reasons in order to provide a response based on the available knowledge, including the LLM knowledge base and a set of tools provided to the LLM. Stores conversation history so that users can ask follow-up questions. The workflow of a conversational agent is variable, based on follow-up questions. For specific questions, uses the Chain-of-Thought (CoT) process to select the best tool from the configured tools for providing a response to the question. Useful for creating a chatbot that employs RAG.

Flow agents

A flow agent is configured with a set of tools that it runs in order. For example, the following agent runs the VectorDBTool and then the MLModelTool. The agent coordinates the tools so that one tool’s output can become another tool’s input. In this example, the VectorDBTool queries the k-NN index and the agent passes its output ${parameters.VectorDBTool.output} to the MLModelTool as context, along with the ${parameters.question} (see the prompt parameter):

  1. POST /_plugins/_ml/agents/_register
  2. {
  3.   "name": "Test_Agent_For_RAG",
  4.   "type": "flow",
  5.   "description": "this is a test agent",
  6.   "tools": [
  7.     {
  8.       "type": "VectorDBTool",
  9.       "parameters": {
  10.         "model_id": "YOUR_TEXT_EMBEDDING_MODEL_ID",
  11.         "index": "my_test_data",
  12.         "embedding_field": "embedding",
  13.         "source_field": ["text"],
  14.         "input": "${parameters.question}"
  15.       }
  16.     },
  17.     {
  18.       "type": "MLModelTool",
  19.       "description": "A general tool to answer any question",
  20.       "parameters": {
  21.         "model_id": "YOUR_LLM_MODEL_ID",
  22.         "prompt": "\n\nHuman:You are a professional data analyst. You will always answer a question based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say you don't know. \n\n Context:\n${parameters.VectorDBTool.output}\n\nHuman:${parameters.question}\n\nAssistant:"
  23.       }
  24.     }
  25.   ]
  26. }

Conversational flow agents

Similarly to a flow agent, a conversational flow agent is configured with a set of tools that it runs in order. The difference between them is that a conversational flow agent stores the conversation in an index, in the following example, the conversation_index. The following agent runs the VectorDBTool and then the MLModelTool:

  1. POST /_plugins/_ml/agents/_register
  2. {
  3.   "name": "population data analysis agent",
  4.   "type": "conversational_flow",
  5.   "description": "This is a demo agent for population data analysis",
  6.   "app_type": "rag",
  7.   "memory": {
  8.     "type": "conversation_index"
  9.   },
  10.   "tools": [
  11.     {
  12.       "type": "VectorDBTool",
  13.       "name": "population_knowledge_base",
  14.       "parameters": {
  15.         "model_id": "YOUR_TEXT_EMBEDDING_MODEL_ID",
  16.         "index": "test_population_data",
  17.         "embedding_field": "population_description_embedding",
  18.         "source_field": [
  19.           "population_description"
  20.         ],
  21.         "input": "${parameters.question}"
  22.       }
  23.     },
  24.     {
  25.       "type": "MLModelTool",
  26.       "name": "bedrock_claude_model",
  27.       "description": "A general tool to answer any question",
  28.       "parameters": {
  29.         "model_id": "YOUR_LLM_MODEL_ID",
  30.         "prompt": """
  32. Human:You are a professional data analyst. You will always answer question based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say don't know. 
  34. Context:
  35. ${parameters.population_knowledge_base.output:-}
  37. ${parameters.chat_history:-}
  39. Human:${parameters.question}
  41. Assistant:"""
  42.       }
  43.     }
  44.   ]
  45. }

Conversational agents

Similarly to a conversational flow agent, a conversational agent stores the conversation in an index, in the following example, the conversation_index. A conversational agent can be configured with an LLM and a set of supplementary tools that perform specific jobs. For example, you can set up an LLM and a CATIndexTool when configuring an agent. When you send a question to the model, the agent also includes the CATIndexTool as context. The LLM then decides whether it needs to use the CATIndexTool to answer questions like “How many indexes are in my cluster?” The context allows an LLM to answer specific questions that are outside of its knowledge base. For example, the following agent is configured with an LLM and a CATIndexTool that retrieves information about your OpenSearch indexes:

  1. POST /_plugins/_ml/agents/_register
  2. {
  3.   "name": "Test_Agent_For_ReAct_ClaudeV2",
  4.   "type": "conversational",
  5.   "description": "this is a test agent",
  6.   "llm": {
  7.     "model_id": "YOUR_LLM_MODEL_ID",
  8.     "parameters": {
  9.       "max_iteration": 5,
  10.       "stop_when_no_tool_found": true,
  11.       "response_filter": "$.completion"
  12.     }
  13.   },
  14.   "memory": {
  15.     "type": "conversation_index"
  16.   },
  17.   "tools": [
  18.     {
  19.       "type": "VectorDBTool",
  20.       "name": "VectorDBTool",
  21.       "description": "A tool to search opensearch index with natural language quesiotn. If you don't know answer for some question, you should always try to search data with this tool. Action Input: <natrual language question>",
  22.       "parameters": {
  23.         "model_id": "YOUR_TEXT_EMBEDDING_MODEL_ID",
  24.         "index": "my_test_data",
  25.         "embedding_field": "embedding",
  26.         "source_field": [ "text" ],
  27.         "input": "${parameters.question}"
  28.       }
  29.     },
  30.     {
  31.       "type": "CatIndexTool",
  32.       "name": "RetrieveIndexMetaTool",
  33.       "description": "Use this tool to get OpenSearch index information: (health, status, index, uuid, primary count, replica count, docs.count, docs.deleted, store.size, primary.store.size)."
  34.     }
  35.   ],
  36.   "app_type": "my app"
  37. }

It is important to provide thorough descriptions of the tools so that the LLM can decide in which situations to use those tools.

Enabling the feature

To enable agents and tools, configure the following setting:

  1. plugins.ml_commons.agent_framework_enabled: true

copy

For conversational agents, you also need to enable RAG for use in conversational search. To enable RAG, configure the following setting:

  1. plugins.ml_commons.rag_pipeline_feature_enabled: true

copy

For more information about ways to enable experimental features, see Experimental feature flags.

Next steps