Keep words token filter

The keep_words token filter is designed to keep only certain words during the analysis process. This filter is useful if you have a large body of text but are only interested in certain keywords or terms.

Parameters

The keep_words token filter can be configured with the following parameters.

ParameterRequired/OptionalData typeDescription
keep_wordsRequired if keep_words_path is not configuredList of stringsThe list of words to keep.
keep_words_pathRequired if keep_words is not configuredStringThe path to the file containing the list of words to keep.
keep_words_caseOptionalBooleanWhether to lowercase all words during comparison. Default is false.

Example

The following example request creates a new index named my_index and configures an analyzer with a keep_words filter:

  1. PUT my_index
  2. {
  3.   "settings": {
  4.     "analysis": {
  5.       "analyzer": {
  6.         "custom_keep_word": {
  7.           "tokenizer": "standard",
  8.           "filter": [ "keep_words_filter" ]
  9.         }
  10.       },
  11.       "filter": {
  12.         "keep_words_filter": {
  13.           "type": "keep",
  14.           "keep_words": ["example", "world", "opensearch"],
  15.           "keep_words_case": true
  16.         }
  17.       }
  18.     }
  19.   }
  20. }

copy

Generated tokens

Use the following request to examine the tokens generated using the analyzer:

  1. GET /my_index/_analyze
  2. {
  3.   "analyzer": "custom_keep_word",
  4.   "text": "Hello, world! This is an OpenSearch example."
  5. }

copy

The response contains the generated tokens:

  1. {
  2.   "tokens": [
  3.     {
  4.       "token": "world",
  5.       "start_offset": 7,
  6.       "end_offset": 12,
  7.       "type": "<ALPHANUM>",
  8.       "position": 1
  9.     },
  10.     {
  11.       "token": "OpenSearch",
  12.       "start_offset": 25,
  13.       "end_offset": 35,
  14.       "type": "<ALPHANUM>",
  15.       "position": 5
  16.     },
  17.     {
  18.       "token": "example",
  19.       "start_offset": 36,
  20.       "end_offset": 43,
  21.       "type": "<ALPHANUM>",
  22.       "position": 6
  23.     }
  24.   ]
  25. }