ChromaDB’s LRU cache doesn’t prevent OOM, and its v0.5.x data loss bug silently orphaned embeddings

From Theory Delta | Methodology | Published 2026-03-29

What the docs say

ChromaDB documentation presents itself as a production-ready vector database with automatic memory management via LRU cache. The official cookbook and getting-started guides do not prominently surface scaling limits or single-node constraints. ChromaDB is the most-starred open-source vector database and is frequently recommended as a default choice for AI application development.

Note: ChromaDB is under active development (current version v1.5.5, March 2026). Claims about v0.5.x behavior are historical. Re-verify on major releases, especially v1.x Rust rewrite changes.

What actually happens

The RAM ceiling is architectural, not configurable

ChromaDB is single-node and embedded-first. Database size is fundamentally limited by available system RAM — there is no horizontal scaling path. The capacity formula (derived from ChromaDB cookbook resource requirements): payload_bytes = vectors × dimensions × 4 bytes. At 1024 dimensions, 1 GB supports approximately 244,000 vectors. A 32 GB machine holds roughly 8 million vectors.

This is sufficient for prototyping and development. For production search at scale — a single enterprise knowledge base, a multi-tenant application — it is insufficient. Teams consistently migrate to Qdrant or Pinecone when they hit this ceiling. The open GitHub issue #1323 tracks this limitation; it has been open without a scaling roadmap.

Memory management failure (historical, fixed ~v0.5.6)

The LRU cache described in ChromaDB documentation did not prevent memory growth in Docker deployments. Issue #1908 documents a connection leak that caused monotonic memory growth until container OOM. Users required Docker restarts every 2–3 days before the fix landed in approximately v0.5.6 via PR #2014. The fix is confirmed closed; teams running v0.5.6+ are not affected. Teams on earlier versions should upgrade.

Data loss in multi-collection deployments (v0.5.7–v0.5.12, fixed in v0.5.13)

Issue #2922 documents a correctness bug in the log purge mechanism affecting v0.5.7 through v0.5.12. During log purge in multi-collection scenarios, embeddings were deleted while metadata survived. The result: collections with orphaned documents — metadata present, no vector representation. Queries returned metadata but could not perform similarity search on affected entries. The failure was silent; there was no error raised during the purge.

The fix landed in v0.5.13 via PR #2923. LangChain and LlamaIndex deprecated affected Chroma versions as a result. Teams on v0.5.7–v0.5.12 who have not migrated should treat their data as potentially incomplete and re-verify collection integrity.

Embedding dimension is locked on first insert

Once vectors are inserted into a collection, the embedding dimension is immutable. Attempting to insert embeddings with a different dimension raises an HTTP 500 error with no automatic migration path (issue #945). The specific internal exception name InvalidDimensionException does not exist in the codebase — the error surfaces as an HTTP 500. To change embedding models, teams must recreate entire collections. There is no public API for inspecting or migrating dimensions.

Batch size ceiling (SQLite constraint)

ChromaDB enforces a maximum of 41,666 embeddings per add() call. Attempting to add more returns: “Cannot submit more than 41,666 embeddings at once.” The root cause is SQLite’s query length constraint, inherited during the v0.3.x to v0.4.x migration (issue #1049, closed). The actual limit is available via the max_batch_size attribute of the producer object. Large ingestion pipelines must batch manually.

Metadata filtering degrades sharply at scale

ChromaDB has no indices on metadata fields — every filtered query performs a full table scan. At 40,000 documents, a single-filter metadata search was reported at approximately 5 minutes response time (issue #1394). The performance cliff scales multiplicatively with collection size and dimensionality, making filtered search impractical at 10K–100K vectors depending on embedding dimension and metadata selectivity.

Pydantic v1 compatibility removed (v1.5.3+)

ChromaDB v1.5.3 dropped the Pydantic v1 compatibility layer entirely. Any code using pydantic.v1 fallback import patterns — common in projects that maintained dual v1/v2 support — is incompatible with Chroma >= v1.5.3. Projects must either pin Chroma below v1.5.3 or complete migration to native Pydantic v2 APIs before upgrading. Note: this claim is sourced from the releases page and is vendor-documented; the Pydantic v2 requirement itself is the claimed behavior.

This finding would be disproved by: benchmarks showing ChromaDB v1.5.x (the Rust rewrite) operating stably above 100M vectors on a single node, or by horizontal scaling capabilities added to the v1.x architecture. It would also be disproved for the memory claim specifically by evidence that the LRU cache reliably evicts before OOM in v1.x Docker deployments.

What to do instead

For new projects: Evaluate ChromaDB for prototyping only. If your production knowledge base will exceed 5–10M vectors, or if you need metadata filtering at scale, evaluate Qdrant (horizontal scaling, payload indices) or Pinecone from the start. Migrating after the fact is painful — the embedding dimension lock means recreating collections.

If you are on v0.5.7–v0.5.12: Upgrade to v0.5.13+ immediately. Audit existing collections for data integrity: query all collection IDs and cross-reference metadata count vs vector count. Discrepancies indicate the data loss bug affected your deployment.

For ingestion: Batch your add() calls at or below 41,666 embeddings. Use collection.max_batch_size to confirm the limit in your environment.

For metadata filtering: At scale, avoid ChromaDB metadata filters for latency-sensitive queries. Consider pre-filtering document IDs in a separate metadata store (PostgreSQL, SQLite) and passing them as ids to ChromaDB get() or query() calls.

For memory management: In Docker, set explicit memory limits and monitor container memory. Do not assume LRU eviction will prevent OOM. If running v0.5.x, upgrade to v0.5.6+ to get the connection leak fix.

Environments tested

Tool	Version	Result
ChromaDB	v0.5.7–v0.5.12	source-reviewed: embeddings silently lost during multi-collection log purge while metadata survives (#2922, fixed v0.5.13)
ChromaDB	v0.4.x Docker	source-reviewed: connection leak → monotonic OOM; Docker restart every 2–3 days (#1908, fixed ~v0.5.6)
ChromaDB	v1.5.5 (current)	source-reviewed: single-node RAM-bound architecture, no horizontal scaling (#1323, open); Pydantic v1 dropped in v1.5.3+
ChromaDB	v0.4.x+	source-reviewed: 41,666 embedding batch limit (SQLite constraint) (#1049, closed)
ChromaDB	Unspecified	source-reviewed: metadata filtering 5-min at 40K docs (#1394)

Confidence and gaps

Confidence: medium — all claims are source-reviewed from GitHub issues, not tested by execution in Theory Delta’s environment. The data loss bug (#2922) and memory leak (#1908) are independently confirmed by third-party issue reporters, LangChain/LlamaIndex deprecation announcements, and fix PRs. The RAM ceiling claim is architectural (confirmed via cookbook resource docs and the open issue #1323). The Pydantic v1 claim is vendor-documented (releases page) and marked as such.

Strongest case against: ChromaDB’s v1.x Rust rewrite may have addressed the memory management issues at the architectural level, not just patched the connection leak. The data loss bug is fixed. The current version (v1.5.5) is a substantially different codebase from the v0.5.x series where most failures were observed. The RAM ceiling is real but the 0.245 shorthand formula is a simplification — actual capacity depends on index overhead, metadata storage, and the 2–4x overhead multiplier recommended in the cookbook.

Open questions: Does the Rust rewrite (v1.x) fix the LRU cache reliability issue, or only improve query speed? What is the first-query cold-start latency penalty for HNSW index loading in v1.x? Does v0.5.13+ fully close the data loss path in multi-collection deployments, or are edge cases remaining?

Seen different? Contribute your evidence — theory delta is what makes this knowledge base work.