Pattern capture token filter

The pattern_capture token filter is a powerful filter that uses regular expressions to capture and extract parts of text according to specific patterns. This filter can be useful when you want to extract particular parts of tokens, such as email domains, hashtags, or numbers, and reuse them for further analysis or indexing.

Parameters

The pattern_capture token filter can be configured with the following parameters.

ParameterRequired/OptionalData typeDescription
patternsRequiredArray of stringsAn array of regular expressions used to capture parts of text.
preserve_originalRequiredBooleanWhether to keep the original token in the output. Default is true.

Example

The following example request creates a new index named email_index and configures an analyzer with a pattern_capture filter to extract the local part and domain name from an email address:

  1. PUT /email_index
  2. {
  3.   "settings": {
  4.     "analysis": {
  5.       "filter": {
  6.         "email_pattern_capture": {
  7.           "type": "pattern_capture",
  8.           "preserve_original": true,
  9.           "patterns": [
  10.             "^([^@]+)",
  11.             "@(.+)$"
  12.           ]
  13.         }
  14.       },
  15.       "analyzer": {
  16.         "email_analyzer": {
  17.           "tokenizer": "uax_url_email",
  18.           "filter": [
  19.             "email_pattern_capture",
  20.             "lowercase"
  21.           ]
  22.         }
  23.       }
  24.     }
  25.   }
  26. }

copy

Generated tokens

Use the following request to examine the tokens generated using the analyzer:

  1. POST /email_index/_analyze
  2. {
  3.   "text": "john.doe@example.com",
  4.   "analyzer": "email_analyzer"
  5. }

copy

The response contains the generated tokens:

  1. {
  2.   "tokens": [
  3.     {
  4.       "token": "john.doe@example.com",
  5.       "start_offset": 0,
  6.       "end_offset": 20,
  7.       "type": "<EMAIL>",
  8.       "position": 0
  9.     },
  10.     {
  11.       "token": "john.doe",
  12.       "start_offset": 0,
  13.       "end_offset": 20,
  14.       "type": "<EMAIL>",
  15.       "position": 0
  16.     },
  17.     {
  18.       "token": "example.com",
  19.       "start_offset": 0,
  20.       "end_offset": 20,
  21.       "type": "<EMAIL>",
  22.       "position": 0
  23.     }
  24.   ]
  25. }