ANN

Approximate Nearest Neighbor (ANN) index configuration for storing vector embeddings.

backend

  1. backend: faiss|hnsw|annoy|numpy|torch|pgvector|sqlite|custom

Sets the ANN backend. Defaults to faiss. Additional backends are available via the ann extras package. Set custom backends via setting this parameter to the fully resolvable class string.

Backend-specific settings are set with a corresponding configuration object having the same name as the backend (i.e. annoy, faiss, or hnsw). These are optional and set to defaults if omitted.

faiss

  1. faiss:
  2.     components: comma separated list of components - defaults to "IDMap,Flat" for small
  3.                 indices and "IVFx,Flat" for larger indexes where
  4.                 x = min(4 * sqrt(embeddings count), embeddings count / 39)
  5.                 automatically calculates number of IVF cells when omitted (supports "IVF,Flat")
  6.     nprobe: search probe setting (int) - defaults to x/16 (as defined above)
  7.             for larger indexes
  8.     nflip: same as nprobe - only used with binary hash indexes
  9.     quantize: store vectors with x-bit precision vs 32-bit (boolean|int)
  10.               true sets 8-bit precision, false disables, int sets specified
  11.               precision
  12.     mmap: load as on-disk index (boolean) - trade query response time for a
  13.           smaller RAM footprint, defaults to false
  14.     sample: percent of data to use for model training (0.0 - 1.0)
  15.             reduces indexing time for larger (>1M+ row) indexes, defaults to 1.0

Faiss supports both floating point and binary indexes. Floating point indexes are the default. Binary indexes are used when indexing scalar-quantized datasets.

See the following Faiss documentation links for more information.

hnsw

  1. hnsw:
  2.     efconstruction:  ef_construction param for init_index (int) - defaults to 200
  3.     m: M param for init_index (int) - defaults to 16
  4.     randomseed: random-seed param for init_index (int) - defaults to 100
  5.     efsearch: ef search param (int) - defaults to None and not set

See Hnswlib documentation for more information on these parameters.

annoy

  1. annoy:
  2.     ntrees: number of trees (int) - defaults to 10
  3.     searchk: search_k search setting (int) - defaults to -1

See Annoy documentation for more information on these parameters. Note that annoy indexes can not be modified after creation, upserts/deletes and other modifications are not supported.

numpy

The NumPy backend is a k-nearest neighbors backend. It’s designed for simplicity and works well with smaller datasets.

The torch backend supports the same options. The only difference is that the vectors can be search using GPUs.

pgvector

  1. pgvector:
  2.     url: database url connection string, alternatively can be set via
  3.          ANN_URL environment variable
  4.     table: database table to store vectors - defaults to `vectors`
  5.     efconstruction:  ef_construction param (int) - defaults to 200
  6.     m: M param for init_index (int) - defaults to 16

The pgvector backend stores embeddings in a Postgres database. See the pgvector documentation for more information on these parameters. See the SQLAlchemy documentation for more information on how to construct url connection strings.

sqlite

  1. sqlite:
  2.     quantize: store vectors with x-bit precision vs 32-bit (boolean|int)
  3.               true sets 8-bit precision, false disables, int sets specified
  4.               precision
  5.     table: database table to store vectors - defaults to `vectors`

The SQLite backend stores embeddings in a SQLite database using sqlite-vec. This backend supports 1-bit and 8-bit quantization at the storage level.

See this note on how to run this ANN on MacOS.