Prometheus metric parsing formats

Prometheus-formatted metrics are parsed and written to InfluxDB in one of two formats, depending on the scraping tool used:

Scraping tools and parsing format

Scraping toolInfluxDB Metric version
Telegraf Prometheus pluginDetermined by metric_version configuration option
InfluxDB scraper1
Flux prometheus.scrape()2

Metric version 1

  • _time: timestamp
  • _measurement: Prometheus metric name (_bucket, _sum, and _count are trimmed from histogram and summary metric names)
  • _field: depends on the Prometheus metric type
    • Counter: counter
    • Gauge: gauge
    • Histogram: histogram bucket upper limits, count, sum
    • Summary: summary quantiles, count, sum
  • _value: Prometheus metric value
  • tags: A tag for each Prometheus label (except for histogram bucket upper limits (le) or summary quantiles (quantile)). The label name is the tag key and the label value is the tag value.

Example Prometheus query results

The following are example Prometheus metrics scraped from the InfluxDB OSS 2.x /metrics endpoint:

  1. # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
  2. # TYPE go_memstats_alloc_bytes_total counter
  3. go_memstats_alloc_bytes_total 1.42276424e+09
  4. # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
  5. # TYPE go_memstats_buck_hash_sys_bytes gauge
  6. go_memstats_buck_hash_sys_bytes 5.259247e+06
  7. # HELP task_executor_run_latency_seconds Records the latency between the time the run was due to run and the time the task started execution, by task type
  8. # TYPE task_executor_run_latency_seconds histogram
  9. task_executor_run_latency_seconds_bucket{task_type="system",le="0.25"} 4413
  10. task_executor_run_latency_seconds_bucket{task_type="system",le="0.5"} 11901
  11. task_executor_run_latency_seconds_bucket{task_type="system",le="1"} 12565
  12. task_executor_run_latency_seconds_bucket{task_type="system",le="2.5"} 12823
  13. task_executor_run_latency_seconds_bucket{task_type="system",le="5"} 12844
  14. task_executor_run_latency_seconds_bucket{task_type="system",le="10"} 12864
  15. task_executor_run_latency_seconds_bucket{task_type="system",le="+Inf"} 74429
  16. task_executor_run_latency_seconds_sum{task_type="system"} 4.256783538679698e+11
  17. task_executor_run_latency_seconds_count{task_type="system"} 74429
  18. # HELP task_executor_run_duration The duration in seconds between a run starting and finishing.
  19. # TYPE task_executor_run_duration summary
  20. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.5"} 5.178160855
  21. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.9"} 5.178160855
  22. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.99"} 5.178160855
  23. task_executor_run_duration_sum{taskID="00xx0Xx0xx00XX0x0",task_type="threshold"} 2121.9758301650004
  24. task_executor_run_duration_count{taskID="00xx0Xx0xx00XX0x0",task_type="threshold"} 570

Resulting line protocol

  1. go_memstats_alloc_bytes_total counter=1.42276424e+09
  2. go_memstats_buck_hash_sys_bytes gauge=5.259247e+06
  3. task_executor_run_latency_seconds,task_type=system 0.25=4413,0.5=11901,1=12565,2.5=12823,5=12844,10=12864,+Inf=74429,sum=4.256783538679698e+11,count=74429
  4. task_executor_run_duration,taskID=00xx0Xx0xx00XX0x0,task_type=threshold 0.5=5.178160855,0.9=5.178160855,0.99=5.178160855,sum=2121.9758301650004,count=570

View version 1 tables when queried from InfluxDB

_time_measurement_field_value
2021-01-01T00:00:00Zgo_memstats_alloc_bytes_totalcounter1422764240.0
_time_measurement_field_value
2021-01-01T00:00:00Zgo_memstats_buck_hash_sys_bytesgauge5259247.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem0.254413.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem0.511901.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem112565.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem2.512823.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem512844.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystem+Inf74429.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystemsum425678353867.9698
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_latency_secondssystemcount74429.0
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_durationthreshold0.55.178160855
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_durationthreshold0.95.178160855
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_durationthreshold0.995.178160855
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_durationthresholdsum2121.9758301650004
_time_measurementtask_type_field_value
2021-01-01T00:00:00Ztask_executor_run_durationthresholdcount570.0

Metrics version 2

Example Prometheus query results

The following are example Prometheus metrics scraped from the InfluxDB OSS 2.x /metrics endpoint:

  1. # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
  2. # TYPE go_memstats_alloc_bytes_total counter
  3. go_memstats_alloc_bytes_total 1.42276424e+09
  4. # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
  5. # TYPE go_memstats_buck_hash_sys_bytes gauge
  6. go_memstats_buck_hash_sys_bytes 5.259247e+06
  7. # HELP task_executor_run_latency_seconds Records the latency between the time the run was due to run and the time the task started execution, by task type
  8. # TYPE task_executor_run_latency_seconds histogram
  9. task_executor_run_latency_seconds_bucket{task_type="system",le="0.25"} 4413
  10. task_executor_run_latency_seconds_bucket{task_type="system",le="0.5"} 11901
  11. task_executor_run_latency_seconds_bucket{task_type="system",le="1"} 12565
  12. task_executor_run_latency_seconds_bucket{task_type="system",le="2.5"} 12823
  13. task_executor_run_latency_seconds_bucket{task_type="system",le="5"} 12844
  14. task_executor_run_latency_seconds_bucket{task_type="system",le="10"} 12864
  15. task_executor_run_latency_seconds_bucket{task_type="system",le="+Inf"} 74429
  16. task_executor_run_latency_seconds_sum{task_type="system"} 4.256783538679698e+11
  17. task_executor_run_latency_seconds_count{task_type="system"} 74429
  18. # HELP task_executor_run_duration The duration in seconds between a run starting and finishing.
  19. # TYPE task_executor_run_duration summary
  20. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.5"} 5.178160855
  21. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.9"} 5.178160855
  22. task_executor_run_duration{taskID="00xx0Xx0xx00XX0x0",task_type="threshold",quantile="0.99"} 5.178160855
  23. task_executor_run_duration_sum{taskID="00xx0Xx0xx00XX0x0",task_type="threshold"} 2121.9758301650004
  24. task_executor_run_duration_count{taskID="00xx0Xx0xx00XX0x0",task_type="threshold"} 570

Resulting line protocol

  1. prometheus,url=http://localhost:8086/metrics go_memstats_alloc_bytes_total=1.42276424e+09
  2. prometheus,url=http://localhost:8086/metrics go_memstats_buck_hash_sys_bytes=5.259247e+06
  3. prometheus,url=http://localhost:8086/metrics,task_type=system,le=0.25 task_executor_run_latency_seconds=4413
  4. prometheus,url=http://localhost:8086/metrics,task_type=system,le=0.5 task_executor_run_latency_seconds=11901
  5. prometheus,url=http://localhost:8086/metrics,task_type=system,le=1 task_executor_run_latency_seconds=12565
  6. prometheus,url=http://localhost:8086/metrics,task_type=system,le=2.5 task_executor_run_latency_seconds=12823
  7. prometheus,url=http://localhost:8086/metrics,task_type=system,le=5 task_executor_run_latency_seconds=12844
  8. prometheus,url=http://localhost:8086/metrics,task_type=system,le=10 task_executor_run_latency_seconds=12864
  9. prometheus,url=http://localhost:8086/metrics,task_type=system,le=+Inf task_executor_run_latency_seconds=74429
  10. prometheus,url=http://localhost:8086/metrics,task_type=system task_executor_run_latency_seconds_sum=4.256783538679698e+11
  11. prometheus,url=http://localhost:8086/metrics,task_type=system task_executor_run_latency_seconds_count=74429
  12. prometheus,url=http://localhost:8086/metrics,taskID=00xx0Xx0xx00XX0x0,task_type=threshold quantile=0.5 task_executor_run_duration=5.178160855
  13. prometheus,url=http://localhost:8086/metrics,taskID=00xx0Xx0xx00XX0x0,task_type=threshold quantile=0.9 task_executor_run_duration=5.178160855
  14. prometheus,url=http://localhost:8086/metrics,taskID=00xx0Xx0xx00XX0x0,task_type=threshold quantile=0.99 task_executor_run_duration=5.178160855
  15. prometheus,url=http://localhost:8086/metrics,taskID=00xx0Xx0xx00XX0x0,task_type=threshold task_executor_run_duration_sum=2121.9758301650004
  16. prometheus,url=http://localhost:8086/metrics,taskID=00xx0Xx0xx00XX0x0,task_type=threshold task_executor_run_duration_count=570

View version 2 tables when queried from InfluxDB

_time_measurementurl_field_value
2021-01-01T00:00:00Zprometheushttp://localhost:8086/metricsgo_memstats_alloc_bytes_total1422764240.0
_time_measurementurl_field_value
2021-01-01T00:00:00Zprometheushttp://localhost:8086/metricsgo_memstats_buck_hash_sys_bytes5259247.0
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics0.25task_executor_run_latency_seconds4413
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics0.5task_executor_run_latency_seconds11901
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics1task_executor_run_latency_seconds12565
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics2.5task_executor_run_latency_seconds12823
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics5task_executor_run_latency_seconds12844
_time_measurementtask_typeurlle_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metrics+Inftask_executor_run_latency_seconds74429
_time_measurementtask_typeurl_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metricstask_executor_run_latency_seconds_sum425678353867.9698
_time_measurementtask_typeurl_field_value
2021-01-01T00:00:00Zprometheussystemhttp://localhost:8086/metricstask_executor_run_latency_seconds_count74429
_time_measurementtask_typetaskIDurlquantile_field_value
2021-01-01T00:00:00Zprometheusthreshold00xx0Xx0xx00XX0x0http://localhost:8086/metrics0.5task_executor_run_duration5.178160855
_time_measurementtask_typetaskIDurlquantile_field_value
2021-01-01T00:00:00Zprometheusthreshold00xx0Xx0xx00XX0x0http://localhost:8086/metrics0.9task_executor_run_duration5.178160855
_time_measurementtask_typetaskIDurlquantile_field_value
2021-01-01T00:00:00Zprometheusthreshold00xx0Xx0xx00XX0x0http://localhost:8086/metrics0.99task_executor_run_duration5.178160855
_time_measurementtask_typetaskIDurl_field_value
2021-01-01T00:00:00Zprometheusthreshold00xx0Xx0xx00XX0x0http://localhost:8086/metricstask_executor_run_duration_sum2121.9758301650004
_time_measurementtask_typetaskIDurl_field_value
2021-01-01T00:00:00Zprometheusthreshold00xx0Xx0xx00XX0x0http://localhost:8086/metricstask_executor_run_duration_count570

prometheus