Claude Code with ChromaDB: Local Vector Search Done Right
Why ChromaDB without CLAUDE.md ships data that vanishes on restart
ChromaDB is the default open-source vector database for teams that want a local-first developer experience. There is no cloud signup, no API key, no monthly bill. You install one package, write three lines, and have a working semantic search index in a SQLite file on disk. The simplicity is the feature. It is also the trap Claude Code falls into when it writes ChromaDB code without explicit rules. The same three lines can give you a vector store that survives forever, or one that disappears the moment the Python process exits.
The most common Claude defaults that produce broken ChromaDB code: instantiating chromadb.Client() instead of chromadb.PersistentClient(path=...) (data lives in memory only), declaring a collection without an embedding function (Chroma silently uses the default all-MiniLM-L6-v2 model), using random UUIDs as document ids without tracking source ids in metadata, calling collection.add() one document at a time, and writing where={"key": "value"} syntax that fails because Chroma's where filter uses operator dictionaries. On top of those, Claude makes a consistent ergonomic mistake: it imports chromadb and calls top-level functions like chromadb.create_collection that do not exist, because the API surface lives on the client instance.
This guide covers the CLAUDE.md configuration that locks Claude Code into ChromaDB's correct model: a PersistentClient with an explicit path, embedding functions declared on every collection, document ids tied to source records, batched adds under the SQLite ceiling, metadata filters with the right operator syntax, and the client-server transition when you outgrow the local file. For the embedding pipeline that feeds ChromaDB, Claude Code with the OpenAI SDK covers the rate limiting that keeps embedding generation cheap. If you are deciding between vector stores, Claude Code with Pinecone covers the managed alternative.
The ChromaDB CLAUDE.md template
The CLAUDE.md at your project root is read at the start of every Claude Code session. For a ChromaDB integration it needs to declare: the SDK version, the client mode, the persistence path, the embedding model and its dimension, the collection naming convention, the add and query patterns, and the hard rules that block the mistakes Claude makes most often.
# ChromaDB rules
## Stack
- chromadb ^0.5.x (Python 3.10+) or chromadb-default-embed ^2.x (Node.js)
- Embedding model: text-embedding-3-small (1536 dimensions) via OpenAIEmbeddingFunction
- Persistence: PersistentClient at ./chroma-data (gitignored)
- OPENAI_API_KEY in .env (never hardcode)
## Project structure
- src/lib/chroma.py , Client singleton + collection accessors
- src/lib/embeddings.py , Embedding function factory
- src/lib/ingest.py , Add function with chunked batching
- src/lib/retrieve.py , Query function with metadata filters
- scripts/reset-collection.py, One-shot rebuild script (gated)
## Client mode (DECLARED, not implicit)
- ALWAYS use PersistentClient(path="./chroma-data") for development
- ALWAYS use HttpClient(host=..., port=...) for shared / production deployments
- NEVER use chromadb.Client() (EphemeralClient), data is lost on process exit
- NEVER instantiate the client inline in route handlers, import from src/lib/chroma.py
## Collection naming (VERSIONED)
- Pattern: {domain}_{model}_v{n}
- Examples: docs_oai3small_v1, notes_oai3small_v1, kb_voyage3_v2
- The model suffix prevents accidental mixing of incompatible embeddings
- The version suffix lets you migrate without dropping the old data
## Embedding function (MANDATORY on every collection)
- ALWAYS pass embedding_function= to create_collection() and get_collection()
- NEVER rely on Chroma's default embedding function in production code
- The function declared at create time MUST match the function at query time
- If the function mismatches, queries silently return wrong results
## Add pattern (MANDATORY batching)
- Batch size: 200 documents per collection.add() call
- SQLite write ceiling sits around 5 MB per transaction with metadata
- ALWAYS pass ids, documents, and metadatas as parallel arrays
- ALWAYS use stable, deterministic ids (hash of source id + chunk index)
- NEVER call collection.add() one document at a time in a loop
## Query pattern (MANDATORY arguments)
- ALWAYS pass n_results explicitly (typical range 5-20)
- ALWAYS pass include=["documents", "metadatas", "distances"] when results matter
- ALWAYS scope by metadata where=... when querying multi-tenant data
- NEVER omit the where clause when querying user-scoped or org-scoped collections
## Hard rules
- NEVER use chromadb.Client() in code that needs to survive a restart
- NEVER create a collection without embedding_function
- NEVER mix embedding models in the same collection
- NEVER write filter syntax like where={"userId": "u_123"} without the $eq operator
- NEVER store the same content with different ids across runs (use deterministic ids)
- ALWAYS check the persistence path exists and is writable at startup
Four rules here prevent the majority of broken ChromaDB integrations Claude generates without them.
The client mode rule prevents the most painful first-week experience. ChromaDB has three modes: Client() (in-memory only), PersistentClient(path=...) (local SQLite + parquet store), and HttpClient(host=...) (talks to a server). The first is the docs default. Claude generates chromadb.Client(), the adds succeed, the queries return results, the engineer ships a demo, restarts the process, and discovers the index is gone. Declaring PersistentClient with an explicit path in CLAUDE.md makes the right choice the only choice.
The embedding function rule prevents an insidious correctness failure. If you call create_collection("docs") with no embedding function, Chroma silently uses all-MiniLM-L6-v2 at 384 dimensions. If you later call get_collection("docs") from a script that uses OpenAI's text-embedding-3-small (1536 dimensions), the embeddings do not match and retrieval is broken in a way that is hard to spot because the API still returns ranked results, they are just the wrong results. The rule forces the function to be declared at both create and get time.
The collection naming rule prevents irreversible migrations. Chroma collections cannot change their embedding model once data is in them. When you switch from text-embedding-3-small to text-embedding-3-large you need a new collection. The versioned name pattern (docs_oai3small_v1, docs_oai3large_v2) makes the migration path obvious and lets the new collection live alongside the old one during the transition.
The batched add rule prevents SQLite write contention. Each collection.add() call is a transaction. One document per call means thousands of transactions for a modest corpus, each touching the same file. At any scale this stalls the import and locks the file for readers. Batches of 200 cut transaction overhead by 200x and stay well under SQLite's practical write ceiling.
Install and client setup
Install ChromaDB for Python:
pip install chromadb openai
Add your OpenAI API key to your environment:
# .env
OPENAI_API_KEY=sk-proj-your-key-here
Create the singleton client and collection accessors:
# src/lib/chroma.py
import os
from pathlib import Path
import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
CHROMA_PATH = Path(__file__).parent.parent.parent / "chroma-data"
CHROMA_PATH.mkdir(parents=True, exist_ok=True)
if not os.environ.get("OPENAI_API_KEY"):
raise RuntimeError("OPENAI_API_KEY is not set")
# Singleton client
_client = chromadb.PersistentClient(path=str(CHROMA_PATH))
# Singleton embedding function
_embedding_function = OpenAIEmbeddingFunction(
api_key=os.environ["OPENAI_API_KEY"],
model_name="text-embedding-3-small",
)
def get_client() -> chromadb.api.ClientAPI:
return _client
def get_collection(name: str):
"""Get an existing collection or raise. Use create_collection for new ones."""
return _client.get_collection(
name=name,
embedding_function=_embedding_function,
)
def create_collection(name: str):
"""Create a new collection. Idempotent: returns existing if it exists."""
return _client.get_or_create_collection(
name=name,
embedding_function=_embedding_function,
metadata={"hnsw:space": "cosine"},
)
Three things in this file Claude will not generate without CLAUDE.md rules: the startup check for OPENAI_API_KEY, the mkdir(parents=True, exist_ok=True) on the persistence path, and the metadata={"hnsw:space": "cosine"} argument that pins the distance metric. The default is L2 (squared euclidean), which produces non-intuitive distance scores. Cosine is the right default for OpenAI embeddings.
For a Node.js project, the equivalent setup uses the JS client:
// src/lib/chroma.ts
import { ChromaClient } from "chromadb";
import { OpenAIEmbeddingFunction } from "chromadb";
if (!process.env.OPENAI_API_KEY) {
throw new Error("OPENAI_API_KEY is not set");
}
export const client = new ChromaClient({
path: "http://localhost:8000",
});
export const embeddingFunction = new OpenAIEmbeddingFunction({
openai_api_key: process.env.OPENAI_API_KEY,
openai_model: "text-embedding-3-small",
});
export async function getCollection(name: string) {
return client.getCollection({ name, embeddingFunction });
}
export async function createCollection(name: string) {
return client.getOrCreateCollection({
name,
embeddingFunction,
metadata: { "hnsw:space": "cosine" },
});
}
The JS client always talks to a running Chroma server, there is no embedded persistent mode in JavaScript. For local development you run chroma run --path ./chroma-data in a terminal. Declaring this in CLAUDE.md prevents Claude from generating Python-style PersistentClient calls in TypeScript files.
Creating collections with the right metadata
A collection is the unit of organisation in Chroma. You typically have one per content type and one per tenant if you are multi-tenant. Naming is important because the embedding model and the metric are locked into the collection at create time.
# scripts/create-collections.py
from src.lib.chroma import create_collection
def main():
docs = create_collection("docs_oai3small_v1")
print(f"docs collection: {docs.count()} documents")
notes = create_collection("notes_oai3small_v1")
print(f"notes collection: {notes.count()} documents")
if __name__ == "__main__":
main()
Run it once:
python scripts/create-collections.py
The script is idempotent because get_or_create_collection returns the existing collection if it already exists, with the same embedding function and metric. Running it twice does not duplicate the data.
Add a hard rule to CLAUDE.md that names the embedding model and the dimension as a locked pair, the same way Pinecone's CLAUDE.md does:
## Embedding model and collection (LOCKED PAIR)
If embedding model is text-embedding-3-small : dimension 1536, collection suffix _oai3small
If embedding model is text-embedding-3-large : dimension 3072, collection suffix _oai3large
If embedding model is voyage-3 : dimension 1024, collection suffix _voyage3
If embedding model is all-MiniLM-L6-v2 : dimension 384, collection suffix _minilm
NEVER mix models in the same collection. To switch models, create a new collection
with the new suffix, run the ingest, then drop the old collection.
The ingest pattern
ChromaDB does not require you to generate embeddings yourself if you pass an embedding function at collection creation. The collection calls the function on each add() to compute the vectors from the raw text. This is the right default because it keeps the embedding model centralised in one place.
# src/lib/ingest.py
import hashlib
from dataclasses import dataclass
from src.lib.chroma import create_collection
ADD_BATCH_SIZE = 200
@dataclass
class Document:
source_id: str
chunk_index: int
text: str
source: str
user_id: str
created_at: str
def doc_id(source_id: str, chunk_index: int) -> str:
"""Deterministic id, same source + chunk always produces the same id."""
raw = f"{source_id}::{chunk_index}".encode("utf-8")
return hashlib.sha256(raw).hexdigest()[:32]
def ingest_documents(docs: list[Document], collection_name: str) -> None:
if not docs:
return
collection = create_collection(collection_name)
# Build parallel arrays
ids = [doc_id(d.source_id, d.chunk_index) for d in docs]
texts = [d.text for d in docs]
metadatas = [
{
"source_id": d.source_id,
"chunk_index": d.chunk_index,
"source": d.source,
"user_id": d.user_id,
"created_at": d.created_at,
}
for d in docs
]
# Batch in 200s
for i in range(0, len(docs), ADD_BATCH_SIZE):
batch_end = min(i + ADD_BATCH_SIZE, len(docs))
collection.add(
ids=ids[i:batch_end],
documents=texts[i:batch_end],
metadatas=metadatas[i:batch_end],
)
print(f"Ingested {len(docs)} documents into {collection_name}")
Three things in this function Claude will not generate without CLAUDE.md rules: the deterministic doc_id function, the parallel arrays pattern, and the batching loop with the correct ADD_BATCH_SIZE. The deterministic id rule matters most. If you generate UUIDs at ingest time, every re-run creates duplicate vectors with new ids and the collection grows without bound. With a deterministic hash, re-running the ingest is idempotent.
Add the ingest pattern to CLAUDE.md as a concrete example:
## Ingest pattern (CONCRETE)
1. Take a list of documents with source_id, chunk_index, text, and metadata
2. Compute deterministic ids: sha256(source_id + chunk_index)[:32]
3. Build parallel arrays for ids, documents, and metadatas
4. Get or create the collection (idempotent)
5. Call collection.add() in batches of 200
The deterministic id rule means re-running the ingest is safe and does not duplicate.
Querying with metadata filters
The query call takes one or more query texts, a n_results limit, and an optional where filter on metadata. Chroma's where syntax uses MongoDB-style operator dictionaries, the same shape as Pinecone. Claude consistently generates the wrong syntax because it imagines a simpler equality form that does not exist.
# src/lib/retrieve.py
from dataclasses import dataclass
from typing import Any
from src.lib.chroma import get_collection
@dataclass
class RetrieveOptions:
query: str
collection_name: str
user_id: str | None = None
source: str | None = None
n_results: int = 8
@dataclass
class RetrievalResult:
id: str
text: str
source: str
distance: float
created_at: str
def retrieve(opts: RetrieveOptions) -> list[RetrievalResult]:
collection = get_collection(opts.collection_name)
# Build where filter
where: dict[str, Any] | None = None
conditions: list[dict[str, Any]] = []
if opts.user_id:
conditions.append({"user_id": {"$eq": opts.user_id}})
if opts.source:
conditions.append({"source": {"$eq": opts.source}})
if len(conditions) == 1:
where = conditions[0]
elif len(conditions) > 1:
where = {"$and": conditions}
response = collection.query(
query_texts=[opts.query],
n_results=opts.n_results,
where=where,
include=["documents", "metadatas", "distances"],
)
ids = response["ids"][0]
docs = response["documents"][0]
metas = response["metadatas"][0]
dists = response["distances"][0]
return [
RetrievalResult(
id=ids[i],
text=docs[i] or "",
source=str(metas[i].get("source", "")),
distance=dists[i],
created_at=str(metas[i].get("created_at", "")),
)
for i in range(len(ids))
]
Calling the function from a FastAPI route:
# src/api/search.py
from fastapi import APIRouter, Depends, HTTPException
from src.lib.retrieve import retrieve, RetrieveOptions
from src.lib.auth import current_user
router = APIRouter()
@router.post("/search")
async def search(body: dict, user=Depends(current_user)):
if not user:
raise HTTPException(status_code=401, detail="Unauthorised")
query = body.get("query")
source = body.get("source")
if not query:
raise HTTPException(status_code=400, detail="Missing query")
results = retrieve(
RetrieveOptions(
query=query,
collection_name="docs_oai3small_v1",
user_id=user.id,
source=source,
n_results=8,
)
)
return {"results": results}
The user_id filter is derived from the authenticated session, not the request body. The same security pattern applies to ChromaDB collections that mix tenants. A user-supplied user_id lets the user query anyone's data. Declare the rule in CLAUDE.md:
## Query scoping (SECURITY)
- user_id and org_id metadata filters MUST come from the session, never the request body
- A user-supplied filter parameter lets the user query any tenant's data
- NEVER pass body.user_id directly to retrieve()
- Always derive: user_id = current_user.id
Metadata filter syntax
Chroma's where filter uses MongoDB-style operators. The common ones:
| Operator | Meaning | Example |
|---|---|---|
$eq |
Equals | {"user_id": {"$eq": "u_123"}} |
$ne |
Not equals | {"source": {"$ne": "archived"}} |
$in |
One of | {"tag": {"$in": ["docs", "guide"]}} |
$nin |
None of | {"tag": {"$nin": ["draft"]}} |
$gt |
Greater than | {"created_at": {"$gt": "2026-01-01"}} |
$gte |
Greater than or equal | {"score": {"$gte": 0.8}} |
$lt |
Less than | {"size": {"$lt": 1000}} |
$lte |
Less than or equal | {"size": {"$lte": 1000}} |
$and |
All conditions match | {"$and": [{"a": {"$eq": 1}}, {"b": {"$eq": 2}}]} |
$or |
Any condition matches | {"$or": [{"a": {"$eq": 1}}, {"b": {"$eq": 2}}]} |
ChromaDB also supports a where_document filter that runs on the raw document text, useful for keyword constraints layered on top of semantic search:
response = collection.query(
query_texts=["how do I configure rate limits"],
n_results=10,
where={"source": {"$eq": "api-docs"}},
where_document={"$contains": "rate limit"},
include=["documents", "metadatas", "distances"],
)
The where_document filter is substring match on the chunk text. It does not replace a real BM25 layer but is a useful pre-filter when a keyword must appear. Add the filter syntax to CLAUDE.md:
## Filter syntax (MongoDB-style operators)
- $eq, $ne, $in, $nin, $gt, $gte, $lt, $lte are the value operators
- $and, $or are the logical operators
- where filter applies to METADATA fields
- where_document filter applies to the raw text via $contains / $not_contains
- ALWAYS wrap values in operator dicts: {"k": {"$eq": v}}, not {"k": v}
- Strings compare exactly; numbers compare numerically
- where_document is substring match, not full-text search
The persistence path
ChromaDB's PersistentClient writes to a directory containing a SQLite database and parquet files for the HNSW index. Treat the directory like any other database file: do not commit it to git, back it up periodically, and gate any operation that would delete it.
Add the path to .gitignore:
# .gitignore
chroma-data/
*.sqlite
*.sqlite3
For production deployments where multiple processes need to share the index, switch from PersistentClient to the client-server mode:
# Start the server (separate terminal, supervisor, or systemd unit)
chroma run --path ./chroma-data --port 8000
Update your client setup to point at the server:
# src/lib/chroma.py (server mode)
import os
import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
CHROMA_HOST = os.environ.get("CHROMA_HOST", "localhost")
CHROMA_PORT = int(os.environ.get("CHROMA_PORT", "8000"))
_client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
_embedding_function = OpenAIEmbeddingFunction(
api_key=os.environ["OPENAI_API_KEY"],
model_name="text-embedding-3-small",
)
The API surface is identical between PersistentClient and HttpClient, so existing code continues to work. The difference is that the server is now the source of truth, and you need to plan for restarts, backups, and authentication. Declare the transition in CLAUDE.md:
## Production transition (PersistentClient -> HttpClient)
- Development: PersistentClient(path="./chroma-data")
- Production: HttpClient(host=os.environ["CHROMA_HOST"], port=int(os.environ["CHROMA_PORT"]))
- The Chroma server runs separately: chroma run --path /var/lib/chroma --port 8000
- Add basic auth via Caddy or nginx reverse proxy in front of the server
- Back up the path directory: tar + S3 nightly, or zfs snapshots
- NEVER expose the Chroma server directly to the public internet
Common Claude Code mistakes with ChromaDB
Six patterns Claude generates incorrectly without CLAUDE.md constraints.
1. EphemeralClient instead of PersistentClient. Claude generates client = chromadb.Client() and ships a demo where data vanishes on restart. Correct: client = chromadb.PersistentClient(path="./chroma-data") imported from a singleton module.
2. Collection created without an embedding function. Claude generates client.create_collection(name="docs") and Chroma silently uses all-MiniLM-L6-v2. Correct: every create and get call passes embedding_function= explicitly.
3. UUID document ids. Claude generates ids = [str(uuid.uuid4()) for _ in docs] and re-running creates duplicates. Correct: deterministic ids derived from source_id and chunk_index.
4. One document per add call. Claude generates for doc in docs: collection.add(...). Correct: batch into arrays of 200, add each batch in one call.
5. Wrong where filter syntax. Claude generates where={"user_id": "u_123"} which Chroma rejects. Correct: where={"user_id": {"$eq": "u_123"}}.
6. Top-level module calls. Claude generates chromadb.add(...) or chromadb.query(...) which do not exist. Correct: get a collection handle, call methods on the collection.
Add these six pairs as a common-mistakes section in CLAUDE.md. Explicit before/after comparisons let Claude match the pattern it would otherwise generate to the corrected form.
Permission hooks for ChromaDB scripts
A ChromaDB project accumulates scripts the same way a Pinecone project does: bulk reingest, collection rebuild, ad-hoc query, persistence path reset. Some are safe. Some destroy data at scale. Permission hooks gate the destructive ones.
In .claude/settings.local.json:
{
"permissions": {
"allow": [
"Bash(python scripts/inspect-collection.py*)",
"Bash(python scripts/sample-query.py*)",
"Bash(python scripts/list-collections.py*)"
],
"deny": [
"Bash(python scripts/reset-collection.py*)",
"Bash(python scripts/delete-collection.py*)",
"Bash(rm -rf chroma-data*)"
]
}
}
Inspecting, sampling, and listing are safe. Resetting a collection, deleting one, or removing the persistence path need explicit confirmation. The deny list forces Claude to surface those operations as prompts rather than running them as part of an automated workflow. For background on hooks, Claude Code with Supabase covers the same pattern applied to database migrations.
When ChromaDB stops being the right answer
ChromaDB excels at local development, single-process applications, and RAG corpora up to roughly ten million vectors. Beyond that you hit friction managed stores do not have. SQLite handles one writer at a time, so continuous ingest while serving queries produces lock contention. ChromaDB does not shard across machines, so a corpus larger than a single server's memory forces you to split collections manually. There is no built-in replication, so multi-region apps either tolerate the latency or run two independent indexes.
When you hit any of these, the migration target is typically Pinecone for managed simplicity, pgvector for SQL integration, or Qdrant for horizontal scaling. The data model is the same in each, so the application code shape barely shifts.
Building local-first retrieval that survives restarts
The ChromaDB CLAUDE.md in this guide produces integrations where the client persists to disk by default, every collection has an explicit embedding function, ids are deterministic, adds batch correctly, queries include metadata, filters use the right operators, and the persistence path is gitignored. The template removes each failure mode by making the correct pattern the only one Claude can generate.
For the RAG orchestration layer, Claude Code with the OpenAI SDK and Claude Code with the Anthropic SDK cover the model-side patterns. Claudify includes a ChromaDB-specific CLAUDE.md template with the PersistentClient default, embedding function rules, versioned collection naming, batched add pattern, filter operators, client-server transition guide, and all six common-mistake rules pre-configured.
Get Claudify. Ship retrieval that survives the next restart.
More like this
Ready to upgrade your Claude Code setup?
Get Claudify
