Langchain Embeddings

Embedding Functions

Chroma and Langchain both offer embedding functions which are wrappers on top of popular embedding models.

Unfortunately Chroma and LC’s embedding functions are not compatible with each other. Below we offer two adapters to convert Chroma’s embedding functions to LC’s and vice versa.

Links:

Chroma Built-in Langchain Adapter

As of version 0.5.x Chroma offers a built-in two-way adapter to convert Langchain’s embedding function to an adapted embeddings that can be used by both LC and Chroma. Implementation can be found here.

  1. # pip install chromadb langchain langchain-huggingface langchain-chroma
  2. import chromadb
  3. from chromadb.utils.embedding_functions import create_langchain_embedding
  4. from langchain_huggingface import HuggingFaceEmbeddings
  6. langchain_embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
  8. ef = create_langchain_embedding(langchain_embeddings)
  9. client = chromadb.PersistentClient(path="/test_folder_1")
  10. collection = client.get_or_create_collection(name="name_1", embedding_function=ef)

Custom Adapter

Here is the adapter to convert Chroma’s embedding functions to LC’s:

  1. from langchain_core.embeddings import Embeddings
  2. from chromadb.api.types import EmbeddingFunction
  5. class ChromaEmbeddingsAdapter(Embeddings):
  6.     def __init__(self, ef: EmbeddingFunction):
  7.         self.ef = ef
  9.     def embed_documents(self, texts):
  10.         return self.ef(texts)
  12.     def embed_query(self, query):
  13.         return self.ef([query])[0]

Here is the adapter to convert LC’s embedding function s to Chroma’s:

  1. from langchain_core.embeddings import Embeddings
  2. from chromadb.api.types import EmbeddingFunction, Documents
  5. class LangChainEmbeddingAdapter(EmbeddingFunction[Documents]):
  6.     def __init__(self, ef: Embeddings):
  7.         self.ef = ef
  9.     def __call__(self, input: Documents) -> Embeddings:
  10.         # LC EFs also have embed_query but Chroma doesn't support that so we just use embed_documents
  11.         # TODO: better type checking
  12.         return self.ef.embed_documents(input)

Example Usage

Using Chroma Embedding Functions with Langchain:

  1. # pip install chromadb langchain langchain-huggingface langchain-chroma
  2. from langchain.vectorstores.chroma import Chroma
  3. from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction
  5. texts = ["foo", "bar", "baz"]
  7. docs_vectorstore = Chroma.from_texts(
  8.     texts=texts,
  9.     collection_name="docs_store",
  10.     embedding=ChromaEmbeddingsAdapter(SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")),
  11. )

Using Langchain Embedding Functions with Chroma:

  1. # pip install chromadb langchain langchain-huggingface langchain-chroma
  2. from langchain_huggingface import HuggingFaceEmbeddings
  3. import chromadb
  5. client = chromadb.Client()
  7. collection = client.get_or_create_collection("test", embedding_function=LangChainEmbeddingAdapter(
  8.     HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")))
  9. collection.add(ids=["1", "2", "3"], documents=["foo", "bar", "baz"])

July 15, 2024