Phrase and Proximity Search with ArangoSearch

Search for phrases and nearby words in full-text

With phrase search, you can query for tokens in a certain order. This allows you to match partial or full sentences. You can also specify how many arbitrary tokens may occur between defined tokens for word proximity searches.

Dataset: IMDB movie dataset

View definition:

  1. {
  2.   "links": {
  3.     "imdb_vertices": {
  4.       "fields": {
  5.         "description": {
  6.           "analyzers": [
  7.             "text_en"
  8.           ]
  9.         }
  10.       }
  11.     }
  12.   }
  13. }

AQL Queries:

Search for movies that have the (normalized and stemmed) tokens biggest and blockbust in their description, in this order:

  1. FOR doc IN imdb
  2.   SEARCH ANALYZER(PHRASE(doc.description, "BIGGEST Blockbuster"), "text_en")
  3.   RETURN {
  4.     title: doc.title,
  5.     description: doc.description
  6.   }
titledescription
Jurassic Park Series… Steven Spielberg gives us on of the biggest blockbusters of the 1990’s
Scary Movie… some of Hollywood’s biggest blockbusters, …

The text_en Analyzer set via the context is applied to the search term BIGGEST Blockbuster, effectively resulting in the query:

  1. FOR doc IN imdb
  2.   SEARCH PHRASE(doc.description, ["biggest", "blockbust"], "text_en")
  3.   RETURN {
  4.     title: doc.title,
  5.     description: doc.description
  6.   }

The search phrase can be handed in via a bind parameter, but it can also be constructed dynamically using a subquery for instance:

  1. LET p = (
  2.   FOR word IN ["tale", "of", "a", "woman"]
  3.     SORT RAND()
  4.     LIMIT 2
  5.     RETURN word
  6. )
  7. FOR doc IN imdb
  8.   SEARCH ANALYZER(PHRASE(doc.description, p), "text_en")
  9.   RETURN {
  10.     title: doc.title,
  11.     description: doc.description
  12.   }

You will get different results if you re-run this query multiple times.

The PHRASE() functions lets you specify tokens and the number of wildcard tokens in an alternating order. You can use this to search for two words with one arbitrary word in between the two words, for instance.

AQL Queries:

Match movies that contain the phrase epic <something> film in their description, where <something> can be exactly one arbitrary token:

  1. FOR doc IN imdb
  2.   SEARCH ANALYZER(PHRASE(doc.description, "epic", 1, "film"), "text_en")
  3.   RETURN {
  4.     title: doc.title,
  5.     description: doc.description
  6.   }
titledescription
O thiasosThe Travelling Players is an epic Greek film from director Theo Angelopoulos …
On Your Mark… The video feels like a very compressed version of an epic Miyazaki film. …
Double Decade… It is with great pride that we present this epic snowboarding film. …
The ApocalypseIn this epic disaster film of faith, a mother and father search for their only child …

Analyzer pre-processing is applied automatically to epic and film based on the Analyzer context or an optional last argument in the call to PHRASE().

The search phrase can also be dynamic. The following query looks up a particular movie with the title Family Business, tokenizes the title and then performs a proximity search for movies with the phrase family <something> business or family <something> <something> business in their description:

  1. LET title = DOCUMENT("imdb_vertices/39967").title // Family Business
  2. FOR doc IN imdb
  3.   SEARCH ANALYZER(
  4.     PHRASE(doc.description, INTERLEAVE(TOKENS(title, "text_en"), [1])) OR
  5.     PHRASE(doc.description, INTERLEAVE(TOKENS(title, "text_en"), [2])), "text_en")
  6.   RETURN {
  7.     title: doc.title,
  8.     description: doc.description
  9.   }
titledescription
Spy Kids 2: The Island of Lost Dreams… now joined the family spy business as …
Do Not DisturbCombining a family vacation with business, …

Combining Phrase Search with other Techniques

Phrase search is not limited to finding full and exact tokens in a particular order. It also lets you search for prefixes, strings with wildcards, etc. in the specified order. See the object tokens description of the PHRASE() function for a full list of options.

AQL Queries:

Match movies where the title has a token that starts with Härr (normalized to harr), followed by six arbitrary tokens and then a token that contains eni:

  1. FOR doc IN imdb
  2.   SEARCH ANALYZER(PHRASE(doc.title, {STARTS_WITH: TOKENS("Härr", "text_en")[0]}, 6, {WILDCARD: "%eni%"}), "text_en")
  3.   RETURN doc.title
Result
Harry Potter and the Order of the Phoenix

The search terms used in object tokens need to be pre-processed manually as shown above with STARTS_WITH.