regex stage

The regex stage is a parsing stage that parses a log line using a regularexpression. Named capture groups in the regex support adding data into theextracted map.

Schema

  1. regex:
  2.   # The RE2 regular expression. Each capture group must be named.
  3.   expression: <string>
  5.   # Name from extracted data to parse. If empty, uses the log message.
  6.   [source: <string>]

expression needs to be a Go RE2 regexstring. Every capture group (re)will be set into the extracted map, every capture group must be named:(?P<name>re). The name of the capture group will be used as the key in theextracted map.

Example

Without source

Given the pipeline:

  1. - regex:
  2.     expression: "^(?s)(?P<time>\\S+?) (?P<stream>stdout|stderr) (?P<flags>\\S+?) (?P<content>.*)$"

And the log line:

  1. 2019-01-01T01:00:00.000000001Z stderr P i'm a log message!

The following key-value pairs would be added to the extracted map:

  • time: 2019-01-01T01:00:00.000000001Z,
  • stream: stderr,
  • flags: P,
  • content: i'm a log message

With source

Given the pipeline:

  1. - json:
  2.     expressions:
  3.       time:
  4. - regex:
  5.     expression: "^(?P<year>\\d+)"
  6.     source:     "time"

And the log line:

  1. {"time":"2019-01-01T01:00:00.000000001Z"}

The first stage would add the following key-value pairs into the extractedmap:

  • time: 2019-01-01T01:00:00.000000001Z

While the regex stage would then parse the value for time in the extracted mapand append the following key-value pairs back into the extracted map:

  • year: 2019