数据拉取配置

scrape_configs 主要用于配置拉取数据节点，每一个拉取配置主要包含以下参数：

  • job_name：任务名称
  • honor_labels： 用于解决拉取数据标签有冲突，当设置为 true, 以拉取数据为准，否则以服务配置为准
  • params：数据拉取访问时带的请求参数
  • scrape_interval： 拉取时间间隔
  • scrape_timeout: 拉取超时时间
  • metrics_path： 拉取节点的 metric 路径
  • scheme： 拉取数据访问协议
  • sample_limit： 存储的数据标签个数限制，如果超过限制，该数据将被忽略，不入存储；默认值为0，表示没有限制
  • relabel_configs： 拉取数据重置标签配置
  • metric_relabel_configs：metric 重置标签配置

其代码结构体定义为：

  1. // ScrapeConfig configures a scraping unit for Prometheus.
  2. type ScrapeConfig struct {
  3.     // The job name to which the job label is set by default.
  4.     JobName string `yaml:"job_name"`
  5.     // Indicator whether the scraped metrics should remain unmodified.
  6.     HonorLabels bool `yaml:"honor_labels,omitempty"`
  7.     // A set of query parameters with which the target is scraped.
  8.     Params url.Values `yaml:"params,omitempty"`
  9.     // How frequently to scrape the targets of this scrape config.
  10.     ScrapeInterval model.Duration `yaml:"scrape_interval,omitempty"`
  11.     // The timeout for scraping targets of this config.
  12.     ScrapeTimeout model.Duration `yaml:"scrape_timeout,omitempty"`
  13.     // The HTTP resource path on which to fetch metrics from targets.
  14.     MetricsPath string `yaml:"metrics_path,omitempty"`
  15.     // The URL scheme with which to fetch metrics from targets.
  16.     Scheme string `yaml:"scheme,omitempty"`
  17.     // More than this many samples post metric-relabelling will cause the scrape to fail.
  18.     SampleLimit uint `yaml:"sample_limit,omitempty"`
  20.     // We cannot do proper Go type embedding below as the parser will then parse
  21.     // values arbitrarily into the overflow maps of further-down types.
  23.     ServiceDiscoveryConfig ServiceDiscoveryConfig `yaml:",inline"`
  24.     HTTPClientConfig       HTTPClientConfig       `yaml:",inline"`
  26.     // List of target relabel configurations.
  27.     RelabelConfigs []*RelabelConfig `yaml:"relabel_configs,omitempty"`
  28.     // List of metric relabel configurations.
  29.     MetricRelabelConfigs []*RelabelConfig `yaml:"metric_relabel_configs,omitempty"`
  31.     // Catches all undefined fields and must be empty after parsing.
  32.     XXX map[string]interface{} `yaml:",inline"`
  33. }

以上配置定义中还包含 ServiceDiscoveryConfig，它的代码定义为：

  1. // ServiceDiscoveryConfig configures lists of different service discovery mechanisms.
  2. type ServiceDiscoveryConfig struct {
  3.     // List of labeled target groups for this job.
  4.     StaticConfigs []*TargetGroup `yaml:"static_configs,omitempty"`
  5.     // List of DNS service discovery configurations.
  6.     DNSSDConfigs []*DNSSDConfig `yaml:"dns_sd_configs,omitempty"`
  7.     // List of file service discovery configurations.
  8.     FileSDConfigs []*FileSDConfig `yaml:"file_sd_configs,omitempty"`
  9.     // List of Consul service discovery configurations.
  10.     ConsulSDConfigs []*ConsulSDConfig `yaml:"consul_sd_configs,omitempty"`
  11.     // List of Serverset service discovery configurations.
  12.     ServersetSDConfigs []*ServersetSDConfig `yaml:"serverset_sd_configs,omitempty"`
  13.     // NerveSDConfigs is a list of Nerve service discovery configurations.
  14.     NerveSDConfigs []*NerveSDConfig `yaml:"nerve_sd_configs,omitempty"`
  15.     // MarathonSDConfigs is a list of Marathon service discovery configurations.
  16.     MarathonSDConfigs []*MarathonSDConfig `yaml:"marathon_sd_configs,omitempty"`
  17.     // List of Kubernetes service discovery configurations.
  18.     KubernetesSDConfigs []*KubernetesSDConfig `yaml:"kubernetes_sd_configs,omitempty"`
  19.     // List of GCE service discovery configurations.
  20.     GCESDConfigs []*GCESDConfig `yaml:"gce_sd_configs,omitempty"`
  21.     // List of EC2 service discovery configurations.
  22.     EC2SDConfigs []*EC2SDConfig `yaml:"ec2_sd_configs,omitempty"`
  23.     // List of OpenStack service discovery configurations.
  24.     OpenstackSDConfigs []*OpenstackSDConfig `yaml:"openstack_sd_configs,omitempty"`
  25.     // List of Azure service discovery configurations.
  26.     AzureSDConfigs []*AzureSDConfig `yaml:"azure_sd_configs,omitempty"`
  27.     // List of Triton service discovery configurations.
  28.     TritonSDConfigs []*TritonSDConfig `yaml:"triton_sd_configs,omitempty"`
  30.     // Catches all undefined fields and must be empty after parsing.
  31.     XXX map[string]interface{} `yaml:",inline"`
  32. }

ServiceDiscoveryConfig 主要用于 target 发现，大体分为两类，静态配置和动态发现。

所以，一份完整的 scrape_configs 配置大致为：

  1. # The job name assigned to scraped metrics by default.
  2. job_name: <job_name>
  4. # How frequently to scrape targets from this job.
  5. [ scrape_interval: <duration> | default = <global_config.scrape_interval> ]
  7. # Per-scrape timeout when scraping this job.
  8. [ scrape_timeout: <duration> | default = <global_config.scrape_timeout> ]
  10. # The HTTP resource path on which to fetch metrics from targets.
  11. [ metrics_path: <path> | default = /metrics ]
  13. # honor_labels controls how Prometheus handles conflicts between labels that are
  14. # already present in scraped data and labels that Prometheus would attach
  15. # server-side ("job" and "instance" labels, manually configured target
  16. # labels, and labels generated by service discovery implementations).
  17. #
  18. # If honor_labels is set to "true", label conflicts are resolved by keeping label
  19. # values from the scraped data and ignoring the conflicting server-side labels.
  20. #
  21. # If honor_labels is set to "false", label conflicts are resolved by renaming
  22. # conflicting labels in the scraped data to "exported_<original-label>" (for
  23. # example "exported_instance", "exported_job") and then attaching server-side
  24. # labels. This is useful for use cases such as federation, where all labels
  25. # specified in the target should be preserved.
  26. #
  27. # Note that any globally configured "external_labels" are unaffected by this
  28. # setting. In communication with external systems, they are always applied only
  29. # when a time series does not have a given label yet and are ignored otherwise.
  30. [ honor_labels: <boolean> | default = false ]
  32. # Configures the protocol scheme used for requests.
  33. [ scheme: <scheme> | default = http ]
  35. # Optional HTTP URL parameters.
  36. params:
  37.   [ <string>: [<string>, ...] ]
  39. # Sets the `Authorization` header on every scrape request with the
  40. # configured username and password.
  41. basic_auth:
  42.   [ username: <string> ]
  43.   [ password: <string> ]
  45. # Sets the `Authorization` header on every scrape request with
  46. # the configured bearer token. It is mutually exclusive with `bearer_token_file`.
  47. [ bearer_token: <string> ]
  49. # Sets the `Authorization` header on every scrape request with the bearer token
  50. # read from the configured file. It is mutually exclusive with `bearer_token`.
  51. [ bearer_token_file: /path/to/bearer/token/file ]
  53. # Configures the scrape request's TLS settings.
  54. tls_config:
  55.   [ <tls_config> ]
  57. # Optional proxy URL.
  58. [ proxy_url: <string> ]
  60. # List of Azure service discovery configurations.
  61. azure_sd_configs:
  62.   [ - <azure_sd_config> ... ]
  64. # List of Consul service discovery configurations.
  65. consul_sd_configs:
  66.   [ - <consul_sd_config> ... ]
  68. # List of DNS service discovery configurations.
  69. dns_sd_configs:
  70.   [ - <dns_sd_config> ... ]
  72. # List of EC2 service discovery configurations.
  73. ec2_sd_configs:
  74.   [ - <ec2_sd_config> ... ]
  76. # List of OpenStack service discovery configurations.
  77. openstack_sd_configs:
  78.   [ - <openstack_sd_config> ... ]
  80. # List of file service discovery configurations.
  81. file_sd_configs:
  82.   [ - <file_sd_config> ... ]
  84. # List of GCE service discovery configurations.
  85. gce_sd_configs:
  86.   [ - <gce_sd_config> ... ]
  88. # List of Kubernetes service discovery configurations.
  89. kubernetes_sd_configs:
  90.   [ - <kubernetes_sd_config> ... ]
  92. # List of Marathon service discovery configurations.
  93. marathon_sd_configs:
  94.   [ - <marathon_sd_config> ... ]
  96. # List of AirBnB's Nerve service discovery configurations.
  97. nerve_sd_configs:
  98.   [ - <nerve_sd_config> ... ]
  100. # List of Zookeeper Serverset service discovery configurations.
  101. serverset_sd_configs:
  102.   [ - <serverset_sd_config> ... ]
  104. # List of Triton service discovery configurations.
  105. triton_sd_configs:
  106.   [ - <triton_sd_config> ... ]
  108. # List of labeled statically configured targets for this job.
  109. static_configs:
  110.   [ - <static_config> ... ]
  112. # List of target relabel configurations.
  113. relabel_configs:
  114.   [ - <relabel_config> ... ]
  116. # List of metric relabel configurations.
  117. metric_relabel_configs:
  118.   [ - <relabel_config> ... ]
  120. # Per-scrape limit on number of scraped samples that will be accepted.
  121. # If more than this number of samples are present after metric relabelling
  122. # the entire scrape will be treated as failed. 0 means no limit.
  123. [ sample_limit: <int> | default = 0 ]