CAUTION: This page documents an old version of Prometheus. Check out the latest stable version.

Unit Testing for Rules

You can use promtool to test your rules.

  1. # For a single test file.
  2. ./promtool test rules test.yml
  4. # If you have multiple test files, say test1.yml,test2.yml,test2.yml
  5. ./promtool test rules test1.yml test2.yml test3.yml

Test file format

  1. # This is a list of rule files to consider for testing. Globs are supported.
  2. rule_files:
  3.   [ - <file_name> ]
  5. [ evaluation_interval: <duration> | default = 1m ]
  7. # The order in which group names are listed below will be the order of evaluation of
  8. # rule groups (at a given evaluation time). The order is guaranteed only for the groups mentioned below.
  9. # All the groups need not be mentioned below.
  10. group_eval_order:
  11.   [ - <group_name> ]
  13. # All the tests are listed here.
  14. tests:
  15.   [ - <test_group> ]

<test_group>

  1. # Series data
  2. interval: <duration>
  3. input_series:
  4.   [ - <series> ]
  6. # Name of the test group
  7. [ name: <string> ]
  9. # Unit tests for the above data.
  11. # Unit tests for alerting rules. We consider the alerting rules from the input file.
  12. alert_rule_test:
  13.   [ - <alert_test_case> ]
  15. # Unit tests for PromQL expressions.
  16. promql_expr_test:
  17.   [ - <promql_test_case> ]
  19. # External labels accessible to the alert template.
  20. external_labels:
  21.   [ <labelname>: <string> ... ]
  23. # External URL accessible to the alert template.
  24. # Usually set using --web.external-url.
  25.   [ external_url: <string> ]

<series>

  1. # This follows the usual series notation '<metric name>{<label name>=<label value>, ...}'
  2. # Examples:
  3. #      series_name{label1="value1", label2="value2"}
  4. #      go_goroutines{job="prometheus", instance="localhost:9090"}
  5. series: <string>
  7. # This uses expanding notation.
  8. # Expanding notation:
  9. #     'a+bxc' becomes 'a a+b a+(2*b) a+(3*b) … a+(c*b)'
  10. #     'a-bxc' becomes 'a a-b a-(2*b) a-(3*b) … a-(c*b)'
  11. # There are special values to indicate missing and stale samples:
  12. #    '_' represents a missing sample from scrape
  13. #    'stale' indicates a stale sample
  14. # Examples:
  15. #     1. '-2+4x3' becomes '-2 2 6 10'
  16. #     2. ' 1-2x4' becomes '1 -1 -3 -5 -7'
  17. #     3. ' 1 _x3 stale' becomes '1 _ _ _ stale'
  18. values: <string>

<alert_test_case>

Prometheus allows you to have same alertname for different alerting rules. Hence in this unit testing, you have to list the union of all the firing alerts for the alertname under a single <alert_test_case>.

  1. # The time elapsed from time=0s when the alerts have to be checked.
  2. eval_time: <duration>
  4. # Name of the alert to be tested.
  5. alertname: <string>
  7. # List of expected alerts which are firing under the given alertname at
  8. # given evaluation time. If you want to test if an alerting rule should
  9. # not be firing, then you can mention the above fields and leave 'exp_alerts' empty.
  10. exp_alerts:
  11.   [ - <alert> ]

<alert>

  1. # These are the expanded labels and annotations of the expected alert.
  2. # Note: labels also include the labels of the sample associated with the
  3. # alert (same as what you see in `/alerts`, without series `__name__` and `alertname`)
  4. exp_labels:
  5.   [ <labelname>: <string> ]
  6. exp_annotations:
  7.   [ <labelname>: <string> ]

<promql_test_case>

  1. # Expression to evaluate
  2. expr: <string>
  4. # The time elapsed from time=0s when the expression has to be evaluated.
  5. eval_time: <duration>
  7. # Expected samples at the given evaluation time.
  8. exp_samples:
  9.   [ - <sample> ]

<sample>

  1. # Labels of the sample in usual series notation '<metric name>{<label name>=<label value>, ...}'
  2. # Examples:
  3. #      series_name{label1="value1", label2="value2"}
  4. #      go_goroutines{job="prometheus", instance="localhost:9090"}
  5. labels: <string>
  7. # The expected value of the PromQL expression.
  8. value: <number>

Example

This is an example input file for unit testing which passes the test. test.yml is the test file which follows the syntax above and alerts.yml contains the alerting rules.

With alerts.yml in the same directory, run ./promtool test rules test.yml.

test.yml

  1. # This is the main input for unit testing.
  2. # Only this file is passed as command line argument.
  4. rule_files:
  5.     - alerts.yml
  7. evaluation_interval: 1m
  9. tests:
  10.     # Test 1.
  11.     - interval: 1m
  12.       # Series data.
  13.       input_series:
  14.           - series: 'up{job="prometheus", instance="localhost:9090"}'
  15.             values: '0 0 0 0 0 0 0 0 0 0 0 0 0 0 0'
  16.           - series: 'up{job="node_exporter", instance="localhost:9100"}'
  17.             values: '1+0x6 0 0 0 0 0 0 0 0' # 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
  18.           - series: 'go_goroutines{job="prometheus", instance="localhost:9090"}'
  19.             values: '10+10x2 30+20x5' # 10 20 30 30 50 70 90 110 130
  20.           - series: 'go_goroutines{job="node_exporter", instance="localhost:9100"}'
  21.             values: '10+10x7 10+30x4' # 10 20 30 40 50 60 70 80 10 40 70 100 130
  23.       # Unit test for alerting rules.
  24.       alert_rule_test:
  25.           # Unit test 1.
  26.           - eval_time: 10m
  27.             alertname: InstanceDown
  28.             exp_alerts:
  29.                 # Alert 1.
  30.                 - exp_labels:
  31.                       severity: page
  32.                       instance: localhost:9090
  33.                       job: prometheus
  34.                   exp_annotations:
  35.                       summary: "Instance localhost:9090 down"
  36.                       description: "localhost:9090 of job prometheus has been down for more than 5 minutes."
  37.       # Unit tests for promql expressions.
  38.       promql_expr_test:
  39.           # Unit test 1.
  40.           - expr: go_goroutines > 5
  41.             eval_time: 4m
  42.             exp_samples:
  43.                 # Sample 1.
  44.                 - labels: 'go_goroutines{job="prometheus",instance="localhost:9090"}'
  45.                   value: 50
  46.                 # Sample 2.
  47.                 - labels: 'go_goroutines{job="node_exporter",instance="localhost:9100"}'
  48.                   value: 50

alerts.yml

  1. # This is the rules file.
  3. groups:
  4. - name: example
  5.   rules:
  7.   - alert: InstanceDown
  8.     expr: up == 0
  9.     for: 5m
  10.     labels:
  11.         severity: page
  12.     annotations:
  13.         summary: "Instance {{ $labels.instance }} down"
  14.         description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes."
  16.   - alert: AnotherInstanceDown
  17.     expr: up == 0
  18.     for: 10m
  19.     labels:
  20.         severity: page
  21.     annotations:
  22.         summary: "Instance {{ $labels.instance }} down"
  23.         description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes."

This documentation is open-source. Please help improve it by filing issues or pull requests.