Rejected requests

Rejected requests

When Elasticsearch rejects a request, it stops the operation and returns an error with a 429 response code. Rejected requests are commonly caused by:

• A depleted thread pool. A depleted search or write thread pool returns a TOO_MANY_REQUESTS error message.
• A circuit breaker error.
• High indexing pressure that exceeds the indexing_pressure.memory.limit.

Check rejected tasks

To check the number of rejected tasks for each thread pool, use the cat thread pool API. A high ratio of rejected to completed tasks, particularly in the search and write thread pools, means Elasticsearch regularly rejects requests.

resp = client.cat.thread_pool(
    v=True,
    h="id,name,queue,active,rejected,completed",
)
print(resp)

const response = await client.cat.threadPool({
  v: "true",
  h: "id,name,queue,active,rejected,completed",
});
console.log(response);

GET /_cat/thread_pool?v=true&h=id,name,queue,active,rejected,completed

write thread pool rejections frequently appear in the erring API and correlating log as EsRejectedExecutionException with either QueueResizingEsThreadPoolExecutor or queue capacity.

These errors are often related to backlogged tasks.

See this video for a walkthrough of troubleshooting threadpool rejections.

Check circuit breakers

To check the number of tripped circuit breakers, use the node stats API.

resp = client.nodes.stats(
    metric="breaker",
)
print(resp)

const response = await client.nodes.stats({
  metric: "breaker",
});
console.log(response);

GET /_nodes/stats/breaker

These statistics are cumulative from node startup. For more information, see circuit breaker errors.

See this video for a walkthrough of diagnosing circuit breaker errors.

Check indexing pressure

To check the number of indexing pressure rejections, use the node stats API.

resp = client.nodes.stats(
    human=True,
    filter_path="nodes.*.indexing_pressure",
)
print(resp)

const response = await client.nodes.stats({
  human: "true",
  filter_path: "nodes.*.indexing_pressure",
});
console.log(response);

GET _nodes/stats?human&filter_path=nodes.*.indexing_pressure

These stats are cumulative from node startup.

Indexing pressure rejections appear as an EsRejectedExecutionException, and indicate that they were rejected due to combined_coordinating_and_primary, coordinating, primary, or replica.

These errors are often related to backlogged tasks, bulk index sizing, or the ingest target’s refresh_interval setting.

See this video for a walkthrough of diagnosing indexing pressure rejections.

Prevent rejected requests

Fix high CPU and memory usage

If Elasticsearch regularly rejects requests and other tasks, your cluster likely has high CPU usage or high JVM memory pressure. For tips, see High CPU usage and High JVM memory pressure.