Running Chroma

Local Server

Article Link

This article is also available on Medium Running ChromaDB — Part 1: Local Server.

Chroma CLI

The simplest way to run Chroma locally is via the Chroma cli which is part of the core Chroma package.

Prerequisites:

  1. pip install chromadb
  2. chroma run --host localhost --port 8000 --path ./my_chroma_data

--host The host to which to listen to, by default it is [localhost](http://localhost:8000/docs) , but if you want to expose it to your entire network then you can specify `0.0.0.0``

--port The port on which to listen to, by default this is 8000.

--path The path where to persist your Chroma data locally.

Target Path Install

It is possible to install Chroma in a specific directory by running pip install chromadb -t /path/to/dir. To run Chroma CLI from the installation dir expor the Python Path export PYTHONPATH=$PYTHONPATH:/path/to/dir.

Docker

Running Chroma server locally can be achieved via a simple docker command as shown below.

Prerequisites:

  1. docker run -d --rm --name chromadb -v ./chroma:/chroma/chroma -e IS_PERSISTENT=TRUE -e ANONYMIZED_TELEMETRY=TRUE chromadb/chroma:latest

Options:

  • -v specifies a local dir which is where Chroma will store its data so when the container is destroyed the data remains. Note: If you are using -e PERSIST_DIRECTORY then you need to point the volume to that directory.
  • -e IS_PERSISTENT=TRUE let’s Chroma know to persist data
  • -e PERSIST_DIRECTORY=/path/in/container specifies the path in the container where the data will be stored, by default it is /chroma/chroma
  • -e ANONYMIZED_TELEMETRY=TRUE allows you to turn on (TRUE) or off (FALSE) anonymous product telemetry which helps the Chroma team in making informed decisions about Chroma OSS and commercial direction.
  • chromadb/chroma:latest indicates the latest Chroma version but can be replaced with any valid tag if a prior version is needed (e.g. chroma:0.4.24)

Docker Compose (Cloned Repo)

If you are feeling adventurous you can also use the Chroma main branch to run a local Chroma server with the latest changes:

Prerequisites:

  1. git clone https://github.com/chroma-core/chroma && cd chroma
  2. docker compose up -d --build

If you want to run a specific version of Chroma you can checkout the version tag you need:

  1. git checkout release/0.4.24

Docker Compose (Without Cloning the Repo)

If you do not wish or are able to clone the repo locally, Chroma server can also be run with docker compose by creating (or using a gist) a docker-compose.yaml

Prerequisites:

  1. version: '3.9'
  3. networks:
  4.   net:
  5.     driver: bridge
  6. services:
  7.   chromadb:
  8.     image: chromadb/chroma:latest
  9.     volumes:
  10.       - ./chromadb:/chroma/chroma
  11.     environment:
  12.       - IS_PERSISTENT=TRUE
  13.       - PERSIST_DIRECTORY=/chroma/chroma # this is the default path, change it as needed
  14.       - ANONYMIZED_TELEMETRY=${ANONYMIZED_TELEMETRY:-TRUE}
  15.     ports:
  16.       - 8000:8000
  17.     networks:
  18.       - net

The above will create a container with the latest Chroma (chromadb/chroma:latest), will expose it to port 8000 on the local machine and will persist data in ./chromadb relative path from where the docker-compose.yaml has been ran.

We have also created a small gist with the above file for convenience:

  1. curl -s https://gist.githubusercontent.com/tazarov/4fd933274bbacb3b9f286b15c01e904b/raw/87268142d64d8ee0f7f98c27a62a5d089923a1df/docker-compose.yaml | docker-compose -f - up

Minikube With Helm Chart

Note: This deployment can just as well be done with KinD depending on your preference.

A more advanced approach to running Chroma locally (but also on a remote cluster) is to deploy it using a Helm chart.

Disclaimer: The chart used here is not a 1st party chart, but is contributed by a core contributor to Chroma.

Prerequisites:

Once you have all of the above running Chroma in a local minikube cluster quite simple

Create a minikube cluster:

  1. minikube start --addons=ingress -p chroma
  2. minikube profile chroma

Get and install the chart:

  1. helm repo add chroma https://amikos-tech.github.io/chromadb-chart/
  2. helm repo update
  3. helm install chroma chroma/chromadb --set chromadb.apiVersion="0.4.24"

By default the chart will enable authentication in Chroma. To get the token run the following:

  1. kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode
  2. # or use this to directly export variable
  3. export CHROMA_TOKEN=$(kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode)

The first step to connect and start using Chroma is to forward your port:

  1. minikube service chroma-chromadb --url

The above should print something like this:

  1. http://127.0.0.1:61892
  2. ❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

Note: Depending on your OS the message might be slightly different.

Test it out (pip install chromadb):

  1. import chromadb
  2. from chromadb.config import Settings
  4. client = chromadb.HttpClient(host="http://127.0.0.1:61892",
  5.                              settings=Settings(
  6.                                  chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
  7.                                  chroma_client_auth_credentials="<your_chroma_token>"))
  8. client.heartbeat()  # this should work with or without authentication - it is a public endpoint
  10. client.get_version()  # this should work with or without authentication - it is a public endpoint
  12. client.list_collections()  # this is a protected endpoint and requires authentication

For more information about the helm chart consult - https://github.com/amikos-tech/chromadb-chart

May 23, 2024