Synonym graph token filter

The synonym_graph token filter is a more advanced version of the synonym token filter. It supports multiword synonyms and processes synonyms across multiple tokens, making it ideal for phrases or scenarios in which relationships between tokens are important.

Parameters

The synonym_graph token filter can be configured with the following parameters.

ParameterRequired/OptionalData typeDescription
synonymsEither synonyms or synonyms_path must be specifiedStringA list of synonym rules defined directly in the configuration.
synonyms_pathEither synonyms or synonyms_path must be specifiedStringThe file path to a file containing synonym rules (either an absolute path or a path relative to the config directory).
lenientOptionalBooleanWhether to ignore exceptions when loading the rule configurations. Default is false.
formatOptionalStringSpecifies the format used to determine how OpenSearch defines and interprets synonyms. Valid values are:
- solr
- wordnet.
Default is solr.
expandOptionalBooleanWhether to expand equivalent synonym rules. Default is false.

For example:
If synonyms are defined as “quick, fast” and expand is set to true, then the synonym rules are configured as follows:
- quick => quick
- quick => fast
- fast => quick
- fast => fast

If expand is set to false, the synonym rules are configured as follows:
- quick => quick
- fast => quick

Example: Solr format

The following example request creates a new index named my-index and configures an analyzer with a synonym_graph filter. The filter is configured with the default solr rule format:

  1. PUT /my-index
  2. {
  3.   "settings": {
  4.     "analysis": {
  5.       "filter": {
  6.         "my_synonym_graph_filter": {
  7.           "type": "synonym_graph",
  8.           "synonyms": [
  9.             "sports car, race car",
  10.             "fast car, speedy vehicle",
  11.             "luxury car, premium vehicle",
  12.             "electric car, EV"
  13.           ]
  14.         }
  15.       },
  16.       "analyzer": {
  17.         "my_synonym_graph_analyzer": {
  18.           "type": "custom",
  19.           "tokenizer": "standard",
  20.           "filter": [
  21.             "lowercase",
  22.             "my_synonym_graph_filter"
  23.           ]
  24.         }
  25.       }
  26.     }
  27.   }
  28. }

copy

Generated tokens

Use the following request to examine the tokens generated using the analyzer:

  1. GET /my-car-index/_analyze
  2. {
  3.   "analyzer": "my_synonym_graph_analyzer",
  4.   "text": "I just bought a sports car and it is a fast car."
  5. }

copy

The response contains the generated tokens:

  1. {
  2.   "tokens": [
  3.     {"token": "i","start_offset": 0,"end_offset": 1,"type": "<ALPHANUM>","position": 0},
  4.     {"token": "just","start_offset": 2,"end_offset": 6,"type": "<ALPHANUM>","position": 1},
  5.     {"token": "bought","start_offset": 7,"end_offset": 13,"type": "<ALPHANUM>","position": 2},
  6.     {"token": "a","start_offset": 14,"end_offset": 15,"type": "<ALPHANUM>","position": 3},
  7.     {"token": "race","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 4},
  8.     {"token": "sports","start_offset": 16,"end_offset": 22,"type": "<ALPHANUM>","position": 4,"positionLength": 2},
  9.     {"token": "car","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 5,"positionLength": 2},
  10.     {"token": "car","start_offset": 23,"end_offset": 26,"type": "<ALPHANUM>","position": 6},
  11.     {"token": "and","start_offset": 27,"end_offset": 30,"type": "<ALPHANUM>","position": 7},
  12.     {"token": "it","start_offset": 31,"end_offset": 33,"type": "<ALPHANUM>","position": 8},
  13.     {"token": "is","start_offset": 34,"end_offset": 36,"type": "<ALPHANUM>","position": 9},
  14.     {"token": "a","start_offset": 37,"end_offset": 38,"type": "<ALPHANUM>","position": 10},
  15.     {"token": "speedy","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 11},
  16.     {"token": "fast","start_offset": 39,"end_offset": 43,"type": "<ALPHANUM>","position": 11,"positionLength": 2},
  17.     {"token": "vehicle","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 12,"positionLength": 2},
  18.     {"token": "car","start_offset": 44,"end_offset": 47,"type": "<ALPHANUM>","position": 13}
  19.   ]
  20. }

Example: WordNet format

The following example request creates a new index named my-wordnet-index and configures an analyzer with a synonym_graph filter. The filter is configured with the wordnet rule format:

  1. PUT /my-wordnet-index
  2. {
  3.   "settings": {
  4.     "analysis": {
  5.       "filter": {
  6.         "my_synonym_graph_filter": {
  7.           "type": "synonym_graph",
  8.           "format": "wordnet",
  9.           "synonyms": [
  10.             "s(100000001, 1, 'sports car', n, 1, 0).",
  11.             "s(100000001, 2, 'race car', n, 1, 0).",
  12.             "s(100000001, 3, 'fast car', n, 1, 0).",
  13.             "s(100000001, 4, 'speedy vehicle', n, 1, 0)."
  14.           ]
  15.         }
  16.       },
  17.       "analyzer": {
  18.         "my_synonym_graph_analyzer": {
  19.           "type": "custom",
  20.           "tokenizer": "standard",
  21.           "filter": [
  22.             "lowercase",
  23.             "my_synonym_graph_filter"
  24.           ]
  25.         }
  26.       }
  27.     }
  28.   }
  29. }

copy

Generated tokens

Use the following request to examine the tokens generated using the analyzer:

  1. GET /my-wordnet-index/_analyze
  2. {
  3.   "analyzer": "my_synonym_graph_analyzer",
  4.   "text": "I just bought a sports car and it is a fast car."
  5. }

copy

The response contains the generated tokens:

  1. {
  2.   "tokens": [
  3.     {"token": "i","start_offset": 0,"end_offset": 1,"type": "<ALPHANUM>","position": 0},
  4.     {"token": "just","start_offset": 2,"end_offset": 6,"type": "<ALPHANUM>","position": 1},
  5.     {"token": "bought","start_offset": 7,"end_offset": 13,"type": "<ALPHANUM>","position": 2},
  6.     {"token": "a","start_offset": 14,"end_offset": 15,"type": "<ALPHANUM>","position": 3},
  7.     {"token": "race","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 4},
  8.     {"token": "fast","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 4,"positionLength": 2},
  9.     {"token": "speedy","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 4,"positionLength": 3},
  10.     {"token": "sports","start_offset": 16,"end_offset": 22,"type": "<ALPHANUM>","position": 4,"positionLength": 4},
  11.     {"token": "car","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 5,"positionLength": 4},
  12.     {"token": "car","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 6,"positionLength": 3},
  13.     {"token": "vehicle","start_offset": 16,"end_offset": 26,"type": "SYNONYM","position": 7,"positionLength": 2},
  14.     {"token": "car","start_offset": 23,"end_offset": 26,"type": "<ALPHANUM>","position": 8},
  15.     {"token": "and","start_offset": 27,"end_offset": 30,"type": "<ALPHANUM>","position": 9},
  16.     {"token": "it","start_offset": 31,"end_offset": 33,"type": "<ALPHANUM>","position": 10},
  17.     {"token": "is","start_offset": 34,"end_offset": 36,"type": "<ALPHANUM>","position": 11},
  18.     {"token": "a","start_offset": 37,"end_offset": 38,"type": "<ALPHANUM>","position": 12},
  19.     {"token": "sports","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 13},
  20.     {"token": "race","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 13,"positionLength": 2},
  21.     {"token": "speedy","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 13,"positionLength": 3},
  22.     {"token": "fast","start_offset": 39,"end_offset": 43,"type": "<ALPHANUM>","position": 13,"positionLength": 4},
  23.     {"token": "car","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 14,"positionLength": 4},
  24.     {"token": "car","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 15,"positionLength": 3},
  25.     {"token": "vehicle","start_offset": 39,"end_offset": 47,"type": "SYNONYM","position": 16,"positionLength": 2},
  26.     {"token": "car","start_offset": 44,"end_offset": 47,"type": "<ALPHANUM>","position": 17}
  27.   ]
  28. }