Configuring Promtail

Promtail is configured in a YAML file (usually referred to as config.yaml)which contains information on the Promtail server, where positions are stored,and how to scrape logs from files.

Configuration File Reference

To specify which configuration file to load, pass the -config.file flag at thecommand line. The file is written in YAML format,defined by the schema below. Brackets indicate that a parameter is optional. Fornon-list parameters the value is set to the specified default.

For more detailed information on configuring how to discover and scrape logs fromtargets, see Scraping. For more information on transforming logsfrom scraped targets, see Pipelines.

Generic placeholders are defined as follows:

  • <boolean>: a boolean that can take the values true or false
  • <int>: any integer matching the regular expression [1-9]+[0-9]*
  • <duration>: a duration matching the regular expression [0-9]+(ms|[smhdwy])
  • <labelname>: a string matching the regular expression [a-zA-Z_][a-zA-Z0-9_]*
  • <labelvalue>: a string of Unicode characters
  • <filename>: a valid path relative to current working directory or an absolute path.
  • <host>: a valid string consisting of a hostname or IP followed by an optional port number
  • <string>: a regular string
  • <secret>: a regular string that is a secret, such as a password

Supported contents and default values of config.yaml:

  1. # Configures the server for Promtail.
  2. [server: <server_config>]
  3. # Describes how Promtail connects to multiple instances
  4. # of Loki, sending logs to each.
  5. clients:
  6. - [<client_config>]
  7. # Describes how to save read file offsets to disk
  8. [positions: <position_config>]
  9. scrape_configs:
  10. - [<scrape_config>]
  11. # Configures how tailed targets will be watched.
  12. [target_config: <target_config>]

server_config

The server_config block configures Promtail’s behavior as an HTTP server:

  1. # HTTP server listen host
  2. [http_listen_host: <string>]
  3. # HTTP server listen port
  4. [http_listen_port: <int> | default = 80]
  5. # gRPC server listen host
  6. [grpc_listen_host: <string>]
  7. # gRPC server listen port
  8. [grpc_listen_port: <int> | default = 9095]
  9. # Register instrumentation handlers (/metrics, etc.)
  10. [register_instrumentation: <boolean> | default = true]
  11. # Timeout for graceful shutdowns
  12. [graceful_shutdown_timeout: <duration> | default = 30s]
  13. # Read timeout for HTTP server
  14. [http_server_read_timeout: <duration> | default = 30s]
  15. # Write timeout for HTTP server
  16. [http_server_write_timeout: <duration> | default = 30s]
  17. # Idle timeout for HTTP server
  18. [http_server_idle_timeout: <duration> | default = 120s]
  19. # Max gRPC message size that can be received
  20. [grpc_server_max_recv_msg_size: <int> | default = 4194304]
  21. # Max gRPC message size that can be sent
  22. [grpc_server_max_send_msg_size: <int> | default = 4194304]
  23. # Limit on the number of concurrent streams for gRPC calls (0 = unlimited)
  24. [grpc_server_max_concurrent_streams: <int> | default = 100]
  25. # Log only messages with the given severity or above. Supported values [debug,
  26. # info, warn, error]
  27. [log_level: <string> | default = "info"]
  28. # Base path to server all API routes from (e.g., /v1/).
  29. [http_path_prefix: <string>]

client_config

The client_config block configures how Promtail connects to an instance ofLoki:

  1. # The URL where Loki is listening, denoted in Loki as http_listen_host and
  2. # http_listen_port. If Loki is running in microservices mode, this is the HTTP
  3. # URL for the Distributor.
  4. url: <string>
  5. # Maximum amount of time to wait before sending a batch, even if that
  6. # batch isn't full.
  7. [batchwait: <duration> | default = 1s]
  8. # Maximum batch size (in bytes) of logs to accumulate before sending
  9. # the batch to Loki.
  10. [batchsize: <int> | default = 102400]
  11. # If using basic auth, configures the username and password
  12. # sent.
  13. basic_auth:
  14. # The username to use for basic auth
  15. [username: <string>]
  16. # The password to use for basic auth
  17. [password: <string>]
  18. # The file containing the password for basic auth
  19. [password_file: <filename>]
  20. # Bearer token to send to the server.
  21. [bearer_token: <secret>]
  22. # File containing bearer token to send to the server.
  23. [bearer_token_file: <filename>]
  24. # HTTP proxy server to use to connect to the server.
  25. [proxy_url: <string>]
  26. # If connecting to a TLS server, configures how the TLS
  27. # authentication handshake will operate.
  28. tls_config:
  29. # The CA file to use to verify the server
  30. [ca_file: <string>]
  31. # The cert file to send to the server for client auth
  32. [cert_file: <filename>]
  33. # The key file to send to the server for client auth
  34. [key_file: <filename>]
  35. # Validates that the server name in the server's certificate
  36. # is this value.
  37. [server_name: <string>]
  38. # If true, ignores the server certificate being signed by an
  39. # unknown CA.
  40. [insecure_skip_verify: <boolean> | default = false]
  41. # Configures how to retry requests to Loki when a request
  42. # fails.
  43. backoff_config:
  44. # Initial backoff time between retries
  45. [minbackoff: <duration> | default = 100ms]
  46. # Maximum backoff time between retries
  47. [maxbackoff: <duration> | default = 10s]
  48. # Maximum number of retries to do
  49. [maxretries: <int> | default = 10]
  50. # Static labels to add to all logs being sent to Loki.
  51. # Use map like {"foo": "bar"} to add a label foo with
  52. # value bar.
  53. external_labels:
  54. [ <labelname>: <labelvalue> ... ]
  55. # Maximum time to wait for a server to respond to a request
  56. [timeout: <duration> | default = 10s]

position_config

The position_config block configures where Promtail will save a fileindicating how far it has read into a file. It is needed for when Promtailis restarted to allow it to continue from where it left off.

  1. # Location of positions file
  2. [filename: <string> | default = "/var/log/positions.yaml"]
  3. # How often to update the positions file
  4. [sync_period: <duration> | default = 10s]

scrape_config

The scrape_config block configures how Promtail can scrape logs from a seriesof targets using a specified discovery method:

  1. # Name to identify this scrape config in the Promtail UI.
  2. job_name: <string>
  3. # Describes how to parse log lines. Suported values [cri docker raw]
  4. [entry_parser: <string> | default = "docker"]
  5. # Describes how to transform logs from targets.
  6. [pipeline_stages: <pipeline_stages>]
  7. # Describes how to scrape logs from the journal.
  8. [journal: <journal_config>]
  9. # Describes how to relabel targets to determine if they should
  10. # be processed.
  11. relabel_configs:
  12. - [<relabel_config>]
  13. # Static targets to scrape.
  14. static_configs:
  15. - [<static_config>]
  16. # Files containing targets to scrape.
  17. file_sd_configs:
  18. - [<file_sd_configs>]
  19. # Describes how to discover Kubernetes services running on the
  20. # same host.
  21. kubernetes_sd_configs:
  22. - [<kubernetes_sd_config>]

pipeline_stages

The pipeline stages (pipeline_stages) is used to transformlog entries and their labels after discovery. It is simply an array of variousstages, defined below.

The purpose of most stages is to extract fields and values into a temporaryset of key-value pairs that is passed around from stage to stage.

  1. - [
  2. <regex_stage>
  3. <json_stage> |
  4. <template_stage> |
  5. <match_stage> |
  6. <timestamp_stage> |
  7. <output_stage> |
  8. <labels_stage> |
  9. <metrics_stage>
  10. ]

Example:

  1. pipeline_stages:
  2. - regex:
  3. expr: "./*"
  4. - json:
  5. timestamp:
  6. source: time
  7. format: RFC3339
  8. labels:
  9. stream:
  10. source: json_key_name.json_sub_key_name
  11. output:

regex_stage

The Regex stage takes a regular expression and extracts captured named groups tobe used in further stages.

  1. regex:
  2. # The RE2 regular expression. Each capture group must be named.
  3. expression: <string>
  4. # Name from extracted data to parse. If empty, uses the log message.
  5. [source: <string>]

json_stage

The JSON stage parses a log line as JSON and takesJMESPath expressions to extract data from the JSON to beused in further stages.

  1. json:
  2. # Set of key/value pairs of JMESPath expressions. The key will be
  3. # the key in the extracted data while the expression will the value,
  4. # evaluated as a JMESPath from the source data.
  5. expressions:
  6. [ <string>: <string> ... ]
  7. # Name from extracted data to parse. If empty, uses the log message.
  8. [source: <string>]

template_stage

The template stage uses Go’stext/template language to manipulatevalues.

  1. template:
  2. # Name from extracted data to parse. If key in extract data doesn't exist, an
  3. # entry for it will be created.
  4. source: <string>
  5. # Go template string to use. In additional to normal template
  6. # functions, ToLower, ToUpper, Replace, Trim, TrimLeft, TrimRight,
  7. # TrimPrefix, TrimSuffix, and TrimSpace are available as functions.
  8. template: <string>

Example:

  1. template:
  2. source: level
  3. template: '{{ if eq .Value "WARN" }}{{ Replace .Value "WARN" "OK" -1 }}{{ else }}{{ .Value }}{{ end }}'

match_stage

The match stage conditionally executes a set of stages when a log entry matchesa configurable LogQL stream selector.

  1. match:
  2. # LogQL stream selector.
  3. selector: <string>
  4. # Names the pipeline. When defined, creates an additional label in
  5. # the pipeline_duration_seconds histogram, where the value is
  6. # concatenated with job_name using an underscore.
  7. [pipieline_name: <string>]
  8. # Nested set of pipeline stages only if the selector
  9. # matches the labels of the log entries:
  10. stages:
  11. - [
  12. <regex_stage>
  13. <json_stage> |
  14. <template_stage> |
  15. <match_stage> |
  16. <timestamp_stage> |
  17. <output_stage> |
  18. <labels_stage> |
  19. <metrics_stage>
  20. ]

timestamp_stage

The timestamp stage parses data from the extracted map and overrides the finaltime value of the log that is stored by Loki. If this stage isn’t present,Promtail will associate the timestamp of the log entry with the time thatlog entry was read.

  1. timestamp:
  2. # Name from extracted data to use for the timestamp.
  3. source: <string>
  4. # Determines how to parse the time string. Can use
  5. # pre-defined formats by name: [ANSIC UnixDate RubyDate RFC822
  6. # RFC822Z RFC850 RFC1123 RFC1123Z RFC3339 RFC3339Nano Unix
  7. # UnixMs UnixNs].
  8. format: <string>
  9. # IANA Timezone Database string.
  10. [location: <string>]
output_stage

The output stage takes data from the extracted map and sets the contents of thelog entry that will be stored by Loki.

  1. output:
  2. # Name from extracted data to use for the log entry.
  3. source: <string>

labels_stage

The labels stage takes data from the extracted map and sets additional labelson the log entry that will be sent to Loki.

  1. labels:
  2. # Key is REQUIRED and the name for the label that will be created.
  3. # Value is optional and will be the name from extracted data whose value
  4. # will be used for the value of the label. If empty, the value will be
  5. # inferred to be the same as the key.
  6. [ <string>: [<string>] ... ]

metrics_stage

The metrics stage allows for defining metrics from the extracted data.

Created metrics are not pushed to Loki and are instead exposed via Promtail’s/metrics endpoint. Prometheus should be configured to scrape Promtail to beable to retrieve the metrics configured by this stage.

  1. # A map where the key is the name of the metric and the value is a specific
  2. # metric type.
  3. metrics:
  4. [<string>: [ <metric_counter> | <metric_gauge> | <metric_histogram> ] ...]
metric_counter

Defines a counter metric whose value only goes up.

  1. # The metric type. Must be Counter.
  2. type: Counter
  3. # Describes the metric.
  4. [description: <string>]
  5. # Key from the extracted data map to use for the mtric,
  6. # defaulting to the metric's name if not present.
  7. [source: <string>]
  8. config:
  9. # Filters down source data and only changes the metric
  10. # if the targeted value exactly matches the provided string.
  11. # If not present, all data will match.
  12. [value: <string>]
  13. # Must be either "inc" or "add" (case insensitive). If
  14. # inc is chosen, the metric value will increase by 1 for each
  15. # log line receieved that passed the filter. If add is chosen,
  16. # the extracted value most be convertible to a positive float
  17. # and its value will be added to the metric.
  18. action: <string>
metric_gauge

Defines a gauge metric whose value can go up or down.

  1. # The metric type. Must be Gauge.
  2. type: Gauge
  3. # Describes the metric.
  4. [description: <string>]
  5. # Key from the extracted data map to use for the mtric,
  6. # defaulting to the metric's name if not present.
  7. [source: <string>]
  8. config:
  9. # Filters down source data and only changes the metric
  10. # if the targeted value exactly matches the provided string.
  11. # If not present, all data will match.
  12. [value: <string>]
  13. # Must be either "set", "inc", "dec"," add", or "sub". If
  14. # add, set, or sub is chosen, the extracted value must be
  15. # convertible to a positive float. inc and dec will increment
  16. # or decrement the metric's value by 1 respectively.
  17. action: <string>
metric_histogram

Defines a histogram metric whose values are bucketed.

  1. # The metric type. Must be Histogram.
  2. type: Histogram
  3. # Describes the metric.
  4. [description: <string>]
  5. # Key from the extracted data map to use for the mtric,
  6. # defaulting to the metric's name if not present.
  7. [source: <string>]
  8. config:
  9. # Filters down source data and only changes the metric
  10. # if the targeted value exactly matches the provided string.
  11. # If not present, all data will match.
  12. [value: <string>]
  13. # Must be either "inc" or "add" (case insensitive). If
  14. # inc is chosen, the metric value will increase by 1 for each
  15. # log line receieved that passed the filter. If add is chosen,
  16. # the extracted value most be convertible to a positive float
  17. # and its value will be added to the metric.
  18. action: <string>
  19. # Holds all the numbers in which to bucket the metric.
  20. buckets:
  21. - <int>

journal_config

The journal_config block configures reading from the systemd journal fromPromtail. Requires a build of Promtail that has journal support enabled. Ifusing the AMD64 Docker image, this is enabled by default.

  1. # The oldest relative time from process start that will be read
  2. # and sent to Loki.
  3. [max_age: <duration> | default = 7h]
  4. # Label map to add to every log coming out of the journal
  5. labels:
  6. [ <labelname>: <labelvalue> ... ]
  7. # Path to a directory to read entries from. Defaults to system
  8. # path when empty.
  9. [path: <string>]

relabel_config

Relabeling is a powerful tool to dynamically rewrite the label set of a targetbefore it gets scraped. Multiple relabeling steps can be configured per scrapeconfiguration. They are applied to the label set of each target in order oftheir appearance in the configuration file.

After relabeling, the instance label is set to the value of __address__ bydefault if it was not set during relabeling. The __scheme__ and__metrics_path__ labels are set to the scheme and metrics path of the targetrespectively. The __param_<name> label is set to the value of the first passedURL parameter called <name>.

Additional labels prefixed with __meta_ may be available during the relabelingphase. They are set by the service discovery mechanism that provided the targetand vary between mechanisms.

Labels starting with __ will be removed from the label set after targetrelabeling is completed.

If a relabeling step needs to store a label value only temporarily (as theinput to a subsequent relabeling step), use the __tmp label name prefix. Thisprefix is guaranteed to never be used by Prometheus itself.

  1. # The source labels select values from existing labels. Their content is concatenated
  2. # using the configured separator and matched against the configured regular expression
  3. # for the replace, keep, and drop actions.
  4. [ source_labels: '[' <labelname> [, ...] ']' ]
  5. # Separator placed between concatenated source label values.
  6. [ separator: <string> | default = ; ]
  7. # Label to which the resulting value is written in a replace action.
  8. # It is mandatory for replace actions. Regex capture groups are available.
  9. [ target_label: <labelname> ]
  10. # Regular expression against which the extracted value is matched.
  11. [ regex: <regex> | default = (.*) ]
  12. # Modulus to take of the hash of the source label values.
  13. [ modulus: <uint64> ]
  14. # Replacement value against which a regex replace is performed if the
  15. # regular expression matches. Regex capture groups are available.
  16. [ replacement: <string> | default = $1 ]
  17. # Action to perform based on regex matching.
  18. [ action: <relabel_action> | default = replace ]

<regex> is any validRE2 regular expression. It isrequired for the replace, keep, drop, labelmap,labeldrop andlabelkeep actions. The regex is anchored on both ends. To un-anchor the regex,use .*<regex>.*.

<relabel_action> determines the relabeling action to take:

  • replace: Match regex against the concatenated source_labels. Then, settarget_label to replacement, with match group references(${1}, ${2}, …) in replacement substituted by their value. If regexdoes not match, no replacement takes place.
  • keep: Drop targets for which regex does not match the concatenated source_labels.
  • drop: Drop targets for which regex matches the concatenated source_labels.
  • hashmod: Set target_label to the modulus of a hash of the concatenated source_labels.
  • labelmap: Match regex against all label names. Then copy the values of the matching labels to label names given by replacement with match group references(${1}, ${2}, …) in replacement substituted by their value.
  • labeldrop: Match regex against all label names. Any label that matches will beremoved from the set of labels.
  • labelkeep: Match regex against all label names. Any label that does not match will beremoved from the set of labels.

Care must be taken with labeldrop and labelkeep to ensure that logs arestill uniquely labeled once the labels are removed.

static_config

A static_config allows specifying a list of targets and a common label setfor them. It is the canonical way to specify static targets in a scrapeconfiguration.

  1. # Configures the discovery to look on the current machine. Must be either
  2. # localhost or the hostname of the current computer.
  3. targets:
  4. - localhost
  5. # Defines a file to scrape and an optional set of additional labels to apply to
  6. # all streams defined by the files from __path__.
  7. labels:
  8. # The path to load logs from. Can use glob patterns (e.g., /var/log/*.log).
  9. __path__: <string>
  10. # Additional labels to assign to the logs
  11. [ <labelname>: <labelvalue> ... ]

file_sd_config

File-based service discovery provides a more generic way to configure statictargets and serves as an interface to plug in custom service discoverymechanisms.

It reads a set of files containing a list of zero or more<static_config>s. Changes to all defined files are detected via disk watchesand applied immediately. Files may be provided in YAML or JSON format. Onlychanges resulting in well-formed target groups are applied.

The JSON file must contain a list of static configs, using this format:

  1. [
  2. {
  3. "targets": [ "localhost" ],
  4. "labels": {
  5. "__path__": "<string>", ...
  6. "<labelname>": "<labelvalue>", ...
  7. }
  8. },
  9. ...
  10. ]

As a fallback, the file contents are also re-read periodically at the specifiedrefresh interval.

Each target has a meta label __meta_filepath during therelabeling phase. Its value is set to thefilepath from which the target was extracted.

  1. # Patterns for files from which target groups are extracted.
  2. files:
  3. [ - <filename_pattern> ... ]
  4. # Refresh interval to re-read the files.
  5. [ refresh_interval: <duration> | default = 5m ]

Where <filename_pattern> may be a path ending in .json, .yml or .yaml.The last path segment may contain a single * that matches any charactersequence, e.g. my/path/tg_*.json.

kubernetes_sd_config

Kubernetes SD configurations allow retrieving scrape targets fromKubernetes’ REST API and always staying synchronizedwith the cluster state.

One of the following role types can be configured to discover targets:

node

The node role discovers one target per cluster node with the addressdefaulting to the Kubelet’s HTTP port.

The target address defaults to the first existing address of the Kubernetesnode object in the address type order of NodeInternalIP, NodeExternalIP,NodeLegacyHostIP, and NodeHostName.

Available meta labels:

  • __meta_kubernetes_node_name: The name of the node object.
  • __meta_kubernetes_node_label_<labelname>: Each label from the node object.
  • __meta_kubernetes_node_labelpresent_<labelname>: true for each label from the node object.
  • __meta_kubernetes_node_annotation_<annotationname>: Each annotation from the node object.
  • __meta_kubernetes_node_annotationpresent_<annotationname>: true for each annotation from the node object.
  • __meta_kubernetes_node_address_<address_type>: The first address for each node address type, if it exists.

In addition, the instance label for the node will be set to the node nameas retrieved from the API server.

service

The service role discovers a target for each service port of each service.This is generally useful for blackbox monitoring of a service.The address will be set to the Kubernetes DNS name of the service and respectiveservice port.

Available meta labels:

  • __meta_kubernetes_namespace: The namespace of the service object.
  • __meta_kubernetes_service_annotation_<annotationname>: Each annotation from the service object.
  • __meta_kubernetes_service_annotationpresent_<annotationname>: “true” for each annotation of the service object.
  • __meta_kubernetes_service_cluster_ip: The cluster IP address of the service. (Does not apply to services of type ExternalName)
  • __meta_kubernetes_service_external_name: The DNS name of the service. (Applies to services of type ExternalName)
  • __meta_kubernetes_service_label_<labelname>: Each label from the service object.
  • __meta_kubernetes_service_labelpresent_<labelname>: true for each label of the service object.
  • __meta_kubernetes_service_name: The name of the service object.
  • __meta_kubernetes_service_port_name: Name of the service port for the target.
  • __meta_kubernetes_service_port_protocol: Protocol of the service port for the target.

pod

The pod role discovers all pods and exposes their containers as targets. Foreach declared port of a container, a single target is generated. If a containerhas no specified ports, a port-free target per container is created for manuallyadding a port via relabeling.

Available meta labels:

  • __meta_kubernetes_namespace: The namespace of the pod object.
  • __meta_kubernetes_pod_name: The name of the pod object.
  • __meta_kubernetes_pod_ip: The pod IP of the pod object.
  • __meta_kubernetes_pod_label_<labelname>: Each label from the pod object.
  • __meta_kubernetes_pod_labelpresent_<labelname>: truefor each label from the pod object.
  • __meta_kubernetes_pod_annotation_<annotationname>: Each annotation from the pod object.
  • __meta_kubernetes_pod_annotationpresent_<annotationname>: true for each annotation from the pod object.
  • __meta_kubernetes_pod_container_init: true if the container is an InitContainer
  • __meta_kubernetes_pod_container_name: Name of the container the target address points to.
  • __meta_kubernetes_pod_container_port_name: Name of the container port.
  • __meta_kubernetes_pod_container_port_number: Number of the container port.
  • __meta_kubernetes_pod_container_port_protocol: Protocol of the container port.
  • __meta_kubernetes_pod_ready: Set to true or false for the pod’s ready state.
  • __meta_kubernetes_pod_phase: Set to Pending, Running, Succeeded, Failed or Unknownin the lifecycle.
  • __meta_kubernetes_pod_node_name: The name of the node the pod is scheduled onto.
  • __meta_kubernetes_pod_host_ip: The current host IP of the pod object.
  • __meta_kubernetes_pod_uid: The UID of the pod object.
  • __meta_kubernetes_pod_controller_kind: Object kind of the pod controller.
  • __meta_kubernetes_pod_controller_name: Name of the pod controller.

endpoints

The endpoints role discovers targets from listed endpoints of a service. Foreach endpoint address one target is discovered per port. If the endpoint isbacked by a pod, all additional container ports of the pod, not bound to anendpoint port, are discovered as targets as well.

Available meta labels:

  • __meta_kubernetes_namespace: The namespace of the endpoints object.
  • __meta_kubernetes_endpoints_name: The names of the endpoints object.
  • For all targets discovered directly from the endpoints list (those not additionally inferredfrom underlying pods), the following labels are attached:
    • __meta_kubernetes_endpoint_hostname: Hostname of the endpoint.
    • __meta_kubernetes_endpoint_node_name: Name of the node hosting the endpoint.
    • __meta_kubernetes_endpoint_ready: Set to true or false for the endpoint’s ready state.
    • __meta_kubernetes_endpoint_port_name: Name of the endpoint port.
    • __meta_kubernetes_endpoint_port_protocol: Protocol of the endpoint port.
    • __meta_kubernetes_endpoint_address_target_kind: Kind of the endpoint address target.
    • __meta_kubernetes_endpoint_address_target_name: Name of the endpoint address target.
  • If the endpoints belong to a service, all labels of the role: service discovery are attached.
  • For all targets backed by a pod, all labels of the role: pod discovery are attached.

ingress

The ingress role discovers a target for each path of each ingress.This is generally useful for blackbox monitoring of an ingress.The address will be set to the host specified in the ingress spec.

Available meta labels:

  • __meta_kubernetes_namespace: The namespace of the ingress object.
  • __meta_kubernetes_ingress_name: The name of the ingress object.
  • __meta_kubernetes_ingress_label_<labelname>: Each label from the ingress object.
  • __meta_kubernetes_ingress_labelpresent_<labelname>: true for each label from the ingress object.
  • __meta_kubernetes_ingress_annotation_<annotationname>: Each annotation from the ingress object.
  • __meta_kubernetes_ingress_annotationpresent_<annotationname>: true for each annotation from the ingress object.
  • __meta_kubernetes_ingress_scheme: Protocol scheme of ingress, https if TLSconfig is set. Defaults to http.
  • __meta_kubernetes_ingress_path: Path from ingress spec. Defaults to /.

See below for the configuration options for Kubernetes discovery:

  1. # The information to access the Kubernetes API.
  2. # The API server addresses. If left empty, Prometheus is assumed to run inside
  3. # of the cluster and will discover API servers automatically and use the pod's
  4. # CA certificate and bearer token file at /var/run/secrets/kubernetes.io/serviceaccount/.
  5. [ api_server: <host> ]
  6. # The Kubernetes role of entities that should be discovered.
  7. role: <role>
  8. # Optional authentication information used to authenticate to the API server.
  9. # Note that `basic_auth`, `bearer_token` and `bearer_token_file` options are
  10. # mutually exclusive.
  11. # password and password_file are mutually exclusive.
  12. # Optional HTTP basic authentication information.
  13. basic_auth:
  14. [ username: <string> ]
  15. [ password: <secret> ]
  16. [ password_file: <string> ]
  17. # Optional bearer token authentication information.
  18. [ bearer_token: <secret> ]
  19. # Optional bearer token file authentication information.
  20. [ bearer_token_file: <filename> ]
  21. # Optional proxy URL.
  22. [ proxy_url: <string> ]
  23. # TLS configuration.
  24. tls_config:
  25. [ <tls_config> ]
  26. # Optional namespace discovery. If omitted, all namespaces are used.
  27. namespaces:
  28. names:
  29. [ - <string> ]

Where <role> must be endpoints, service, pod, node, oringress.

Seethis example Prometheus configuration filefor a detailed example of configuring Prometheus for Kubernetes.

You may wish to check out the 3rd partyPrometheus Operator,which automates the Prometheus setup on top of Kubernetes.

target_config

The target_config block controls the behavior of reading files from discoveredtargets.

  1. # Period to resync directories being watched and files being tailed to discover
  2. # new ones or stop watching removed ones.
  3. sync_period: "10s"

Example Docker Config

  1. server:
  2. http_listen_port: 9080
  3. grpc_listen_port: 0
  4. positions:
  5. filename: /tmp/positions.yaml
  6. client:
  7. url: http://ip_or_hostname_where_Loki_run:3100/loki/api/v1/push
  8. scrape_configs:
  9. - job_name: system
  10. pipeline_stages:
  11. - docker:
  12. static_configs:
  13. - targets:
  14. - localhost
  15. labels:
  16. job: varlogs
  17. host: yourhost
  18. __path__: /var/log/*.log
  19. - job_name: someone_service
  20. pipeline_stages:
  21. - docker:
  22. static_configs:
  23. - targets:
  24. - localhost
  25. labels:
  26. job: someone_service
  27. host: yourhost
  28. __path__: /srv/log/someone_service/*.log

Example Journal Config

  1. server:
  2. http_listen_port: 9080
  3. grpc_listen_port: 0
  4. positions:
  5. filename: /tmp/positions.yaml
  6. clients:
  7. - url: http://ip_or_hostname_where_loki_runns:3100/loki/api/v1/push
  8. scrape_configs:
  9. - job_name: journal
  10. journal:
  11. max_age: 12h
  12. path: /var/log/journal
  13. labels:
  14. job: systemd-journal
  15. relabel_configs:
  16. - source_labels: ['__journal__systemd_unit']
  17. target_label: 'unit'