Template examples

Prometheus supports templating in the annotations and labels of alerts,as well as in served console pages. Templates have the ability to runqueries against the local database, iterate over data, use conditionals,format data, etc. The Prometheus templating language is based on the Gotemplating system.

Simple alert field templates

  1. alert: InstanceDown
  2. expr: up == 0
  3. for: 5m
  4. labels:
  5.   severity: page
  6. annotations:
  7.   summary: "Instance {{$labels.instance}} down"
  8.   description: "{{$labels.instance}} of job {{$labels.job}} has been down for more than 5 minutes."

Alert field templates will be executed during every rule iteration for eachalert that fires, so keep any queries and templates lightweight. If you have aneed for more complicated templates for alerts, it is recommended to link to aconsole instead.

Simple iteration

This displays a list of instances, and whether they are up:

  1. {{ range query "up" }}
  2.   {{ .Labels.instance }} {{ .Value }}
  3. {{ end }}

The special . variable contains the value of the current sample for each loop iteration.

Display one value

  1. {{ with query "some_metric{instance='someinstance'}" }}
  2.   {{ . | first | value | humanize }}
  3. {{ end }}

Go and Go's templating language are both strongly typed, so one must check thatsamples were returned to avoid an execution error. For example this couldhappen if a scrape or rule evaluation has not run yet, or a host was down.

The included prom_query_drilldown template handles this, allows forformatting of results, and linking to the expression browser.

Using console URL parameters

  1. {{ with printf "node_memory_MemTotal{job='node',instance='%s'}" .Params.instance | query }}
  2.   {{ . | first | value | humanize1024 }}B
  3. {{ end }}

If accessed as console.html?instance=hostname, .Params.instance will evaluate to hostname.

Advanced iteration

  1. <table>
  2. {{ range printf "node_network_receive_bytes{job='node',instance='%s',device!='lo'}" .Params.instance | query | sortByLabel "device"}}
  3.   <tr><th colspan=2>{{ .Labels.device }}</th></tr>
  4.   <tr>
  5.     <td>Received</td>
  6.     <td>{{ with printf "rate(node_network_receive_bytes{job='node',instance='%s',device='%s'}[5m])" .Labels.instance .Labels.device | query }}{{ . | first | value | humanize }}B/s{{end}}</td>
  7.   </tr>
  8.   <tr>
  9.     <td>Transmitted</td>
  10.     <td>{{ with printf "rate(node_network_transmit_bytes{job='node',instance='%s',device='%s'}[5m])" .Labels.instance .Labels.device | query }}{{ . | first | value | humanize }}B/s{{end}}</td>
  11.   </tr>{{ end }}
  12. </table>

Here we iterate over all network devices and display the network traffic for each.

As the range action does not specify a variable, .Params.instance is notavailable inside the loop as . is now the loop variable.

Defining reusable templates

Prometheus supports defining templates that can be reused. This is particularlypowerful when combined withconsole library support, allowingsharing of templates across consoles.

  1. {{/* Define the template */}}
  2. {{define "myTemplate"}}
  3.   do something
  4. {{end}}
  6. {{/* Use the template */}}
  7. {{template "myTemplate"}}

Templates are limited to one argument. The args function can be used to wrap multiple arguments.

  1. {{define "myMultiArgTemplate"}}
  2.   First argument: {{.arg0}}
  3.   Second argument: {{.arg1}}
  4. {{end}}
  5. {{template "myMultiArgTemplate" (args 1 2)}}