Query Insights plugin health

The Query Insights plugin provides an API and metrics for monitoring its health and performance, enabling proactive identification of issues that may affect query processing or system resources.

Health Stats API

Introduced 2.18

The Health Stats API provides health metrics for each node running the Query Insights plugin. These metrics allow for an in-depth view of resource usage and the health of the query processing components.

Path and HTTP methods

  1. GET _insights/health_stats

Example request

  1. GET _insights/health_stats

copy

Example response

The response includes a set of health-related fields for each node:

  1. PUT _cluster/settings
  2. {
  3.   "AqegbPL0Tv2XWvZV4PTS8Q": {
  4.     "ThreadPoolInfo": {
  5.       "query_insights_executor": {
  6.         "type": "scaling",
  7.         "core": 1,
  8.         "max": 5,
  9.         "keep_alive": "5m",
  10.         "queue_size": 2
  11.       }
  12.     },
  13.     "QueryRecordsQueueSize": 2,
  14.     "TopQueriesHealthStats": {
  15.       "latency": {
  16.         "TopQueriesHeapSize": 5,
  17.         "QueryGroupCount_Total": 0,
  18.         "QueryGroupCount_MaxHeap": 0
  19.       },
  20.       "memory": {
  21.         "TopQueriesHeapSize": 5,
  22.         "QueryGroupCount_Total": 0,
  23.         "QueryGroupCount_MaxHeap": 0
  24.       },
  25.       "cpu": {
  26.         "TopQueriesHeapSize": 5,
  27.         "QueryGroupCount_Total": 0,
  28.         "QueryGroupCount_MaxHeap": 0
  29.       }
  30.     }
  31.   }
  32. }

Response fields

The following table lists all response body fields.

FieldData typeDescription
ThreadPoolInfoObjectInformation about the Query Insights thread pool, including type, core count, max threads, and queue size. See The ThreadPoolInfo object.
QueryRecordsQueueSizeIntegerThe size of the queue that buffers incoming search queries before processing. A high value may suggest increased load or slower processing.
TopQueriesHealthStatsObjectPerformance metrics for each top query service that provide information about memory allocation (heap size) and query grouping. See The TopQueriesHealthStats object.

The ThreadPoolInfo object

The ThreadPoolInfo object contains the following detailed configuration and performance data for the thread pool dedicated to the Query Insights plugin.

FieldData typeDescription
typeStringThe thread pool type (for example, scaling).
coreIntegerThe minimum number of threads in the thread pool.
maxIntegerThe maximum number of threads in the thread pool.
keep_aliveTime unitThe amount of time that idle threads are retained.
queue_sizeIntegerThe maximum number of tasks in the queue.

The TopQueriesHealthStats object

The TopQueriesHealthStats object provides breakdowns for latency, memory, and CPU usage and contains the following information.

FieldData typeDescription
TopQueriesHeapSizeIntegerThe heap memory allocation for the query group.
QueryGroupCount_TotalIntegerThe total number of processed query groups.
QueryGroupCount_MaxHeapIntegerThe size of the max heap that stores all query groups in memory.

OpenTelemetry error metrics counters

The Query Insights plugin integrates with OpenTelemetry to provide real-time error metrics counters. These counters help to identify specific operational failures in the plugin and improve reliability. Each metric provides targeted insights into potential error sources in the plugin workflow, allowing for more focused debugging and maintenance.

To collect these metrics, you must configure and collect query metrics. For more information, see Query metrics.

The following table lists all available metrics.

FieldDescription
LOCAL_INDEX_READER_PARSING_EXCEPTIONSThe number of errors that occur when parsing data using the LocalIndexReader.
LOCAL_INDEX_EXPORTER_BULK_FAILURESThe number of failures that occur when ingesting Query Insights plugin data into local indexes.
LOCAL_INDEX_EXPORTER_EXCEPTIONSThe number of exceptions that occur in the Query Insights plugin LocalIndexExporter.
INVALID_EXPORTER_TYPE_FAILURESThe number of invalid exporter type failures.
INVALID_INDEX_PATTERN_EXCEPTIONSThe number of invalid index pattern exceptions.
DATA_INGEST_EXCEPTIONSThe number of exceptions that occur when ingesting data into the Query Insights plugin.
QUERY_CATEGORIZE_EXCEPTIONSThe number of exceptions that occur when categorizing the queries.
EXPORTER_FAIL_TO_CLOSE_EXCEPTIONThe number of failures that occur when closing the exporter.