Ollama

Ollama offers out-of-the-box embedding API which allows you to generate embeddings for your documents. Chroma provides a convenient wrapper around Ollama’s embedding API.

Ollama Embedding Models

While you can use any of the ollama models including LLMs to generate embeddings. We generally recommend using specialized models like nomic-embed-text for text embeddings. The latter models are specifically trained for embeddings and are more efficient for this purpose (e.g. the dimensions of the output embeddings are much smaller than those from LLMs e.g. 1024 - nomic-embed-text vs 4096 - llama3)

Models:

Basic Usage

First let’s run a local docker container with Ollama. We’ll pull nomic-embed-text model:

docker run -d --rm -v ./ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run nomic-embed-text # press Ctrl+D to exit after model downloads successfully
# test it
curl http://localhost:11434/api/embeddings -d '{"model": "nomic-embed-text","prompt": "Here is an article about llamas..."}'

Ollama Docs

For more information on Ollama, visit the Ollama GitHub repository.

Using the CLI

If you have or prefer to use the Ollama CLI, you can use the following command to get a model:

ollama pull nomic-embed-text

Now let’s configure our OllamaEmbeddingFunction Embedding (python) function with the default Ollama endpoint:

Python

import chromadb
from chromadb.utils.embedding_functions import OllamaEmbeddingFunction
client = chromadb.PersistentClient(path="ollama")
# create EF with custom endpoint
ef = OllamaEmbeddingFunction(
    model_name="nomic-embed-text",
    url="http://localhost:11434/api/embeddings",
)
print(ef(["Here is an article about llamas..."]))

JavaScript

For JS users, you can use the OllamaEmbeddingFunction class to create embeddings:

const {OllamaEmbeddingFunction} = require('chromadb');
const embedder = new OllamaEmbeddingFunction({
    url: "http://localhost:11434/api/embeddings",
    model: "nomic-embed-text"
})
// use directly
const embeddings = embedder.generate(["Here is an article about llamas..."])

Golang

For Golang you can use the chroma-go client’s OllamaEmbeddingFunction embedding function to generate embeddings for your documents:

package main
import (
    "context"
    "fmt"
    ollama "github.com/amikos-tech/chroma-go/ollama"
)
func main() {
    documents := []string{
        "Document 1 content here",
        "Document 2 content here",
    }
    // the `/api/embeddings` endpoint is automatically appended to the base URL
    ef, err := ollama.NewOllamaEmbeddingFunction(ollama.WithBaseURL("http://127.0.0.1:11434"), ollama.WithModel("nomic-embed-text"))
    if err != nil {
        fmt.Printf("Error creating Ollama embedding function: %s \n", err)
    }
    resp, err := ef.EmbedDocuments(context.Background(), documents)
    if err != nil {
        fmt.Printf("Error embedding documents: %s \n", err)
    }
    fmt.Printf("Embedding response: %v \n", resp)
}

Golang Client

You can install the Golang client by running the following command:

go get github.com/amikos-tech/chroma-go

For more information visit https://go-client.chromadb.dev/

May 9, 2024