Alerting on cluster and index events

The alerting features enable you to watch for changes or anomalies in your data and perform the necessary actions in response. For example, you might want to:

• Monitor social media as another way to detect failures in user-facing automated systems like ATMs or ticketing systems. When the number of tweets and posts in an area exceeds a threshold of significance, notify a service technician.
• Monitor your infrastructure, tracking disk usage over time. Open a helpdesk ticket when any servers are likely to run out of free space in the next few days.
• Track network activity to detect malicious activity, and proactively change firewall configuration to reject the malicious user.
• Monitor Elasticsearch, and send immediate notification to the system administrator if nodes leave the cluster or query throughput exceeds an expected range.
• Track application response times and if page-load time exceeds SLAs for more than 5 minutes, open a helpdesk ticket. If SLAs are exceeded for an hour, page the administrator on duty.

All of these use-cases share a few key properties:

• The relevant data or changes in data can be identified with a periodic Elasticsearch query.
• The results of the query can be checked against a condition.
• One or more actions are taken if the condition is true — an email is sent, a 3rd party system is notified, or the query results are stored.

How watches work

The alerting features provide an API for creating, managing and testing watches. A watch describes a single alert and can contain multiple notification actions.

A watch is constructed from four simple building blocks:

Schedule

A schedule for running a query and checking the condition.

Query

The query to run as input to the condition. Watches support the full Elasticsearch query language, including aggregations.

Condition

A condition that determines whether or not to execute the actions. You can use simple conditions (always true), or use scripting for more sophisticated scenarios.

Actions

One or more actions, such as sending email, pushing data to 3rd party systems through a webhook, or indexing the results of the query.

A full history of all watches is maintained in an Elasticsearch index. This history keeps track of each time a watch is triggered and records the results from the query, whether the condition was met, and what actions were taken.