Does txtfetch write directly to Chroma?

No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the add() call itself stay a caller-side step, exactly like the pipeline above shows.

Is there a real limit on how many chunks I can add at once?

Yes, though it isn't a fixed published number. Chroma exposes it as client.get_max_batch_size(), a SQLite parameter ceiling that varies by build. Call it at runtime, and split a large batch yourself if you're close to it.

Does txtfetch benchmark accuracy or speed against Chroma?

No. Chroma is an embedding database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

Extract text into Chroma

An open-source embedding database with a first-class field for the chunk text itself. Here's what changes between the text txtfetch returns and the first row you write to Chroma.

The problem

Chroma's add() call takes the chunk text directly, so there's no metadata workaround to design around. The trap is batch size instead. It's capped by a SQLite parameter ceiling that varies by build, roughly 5,000 to 44,000 records. Chroma won't split an oversized call for you.

what chroma needs

Constraints that shape chunking and the write step, cited and dated
Per-vector metadata limit	No published byte ceiling on a document or metadata value.
Stores the chunk text itself	Yes, as a dedicated documents field, alongside embeddings and metadatas.
ID format	Any unique string within the collection.
Max vector dimensions	No published ceiling. The first vector you insert fixes the collection's dimension, and every later insert must match it.
Batch upsert size	Governed by client.get_max_batch_size(), a SQLite parameter ceiling that varies by build (roughly 5,000 to 44,000 records). Chroma won't split an oversized call for you.
Index type	HNSW. Distance metric (l2, cosine, or ip) is set once at collection creation and can't change after.

No accuracy, speed, or recall figure is measured against Chroma on this page. See /benchmarks for txtfetch's own measured extraction numbers.

the pipeline

Extract, chunk, embed, and upsert. No framework, just the txtfetch Python SDK and Chroma's own client.

Python

import os

import chromadb
from openai import OpenAI
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
oai = OpenAI()
client = chromadb.PersistentClient(path="./chroma")
collection = client.get_or_create_collection(name="docs")


def chunk_text(text: str, size: int = 1000, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Chunk
chunks = chunk_text(result.extracted_text)

# 3. Embed
response = oai.embeddings.create(model="text-embedding-3-small", input=chunks)
embeddings = [item.embedding for item in response.data]

# 4. Upsert — batch size is capped by client.get_max_batch_size(), which
# varies by build. Split a large chunk list yourself if you're close to it.
ids = [f"whitepaper-{i}" for i in range(len(chunks))]
collection.add(
    ids=ids,
    embeddings=embeddings,
    documents=chunks,
    metadatas=[{"source": "whitepaper.pdf"} for _ in chunks],
)

Every call above was resolved against chromadb 1.5.9, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

Chunking for Chroma

Chroma's documents field carries the full chunk text with no published size ceiling. Pick a chunk size for retrieval quality, not to fit a limit. Watch the batch size instead. client.get_max_batch_size() varies by build, so a script that works on your laptop can still hit a wall in a different deployment. Preview your chunker's real boundaries and token counts in the chunk previewer first.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a 202 plus a job_id to poll, and ?async=true forces that path for any request. Pass webhook_url instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See async jobs & webhooks for the full lifecycle.

Submit (async)

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}

Poll

curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.

curl

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"

A retried request, from a client timeout or a dropped response, shouldn't extract and upsert the same document twice. Send the same Idempotency-Key on a retry and txtfetch returns the first result instead of running the job again. See idempotency for the full guarantee.

curl

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf

JavaScript

// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });

Python

# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")

txtfetch ships no connector, plugin, or client for Chroma. The pipeline above is the whole integration. Extract with txtfetch, then write to Chroma with Chroma's own client, the same as you would for any other source of text.

frequently asked questions

Does txtfetch write directly to Chroma?: No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the add() call itself stay a caller-side step, exactly like the pipeline above shows.
Is there a real limit on how many chunks I can add at once?: Yes, though it isn't a fixed published number. Chroma exposes it as client.get_max_batch_size(), a SQLite parameter ceiling that varies by build. Call it at runtime, and split a large batch yourself if you're close to it.
Does txtfetch benchmark accuracy or speed against Chroma?: No. Chroma is an embedding database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

sources

Chroma: Client setup (Persistent, Http, Cloud)Accessed 2026-09
Chroma: Getting started (add, documents, metadatas)Accessed 2026-09
Chroma: Collection API referenceAccessed 2026-09
Chroma: Client API reference (get_max_batch_size)Accessed 2026-09
Chroma: Configure collections (distance metric)Accessed 2026-09
Chroma Cookbook: batching strategies (batch ceiling varies by build)Accessed 2026-09directional

RAG & LLM ingestion, the wider use case this pipeline feeds.
RAG recipe: chunk, embed, index, the same pipeline through LangChain or LlamaIndex instead.
Chunk previewer, to test your own extracted text before you write the first row.
Async jobs & webhooks, the full lifecycle for a large batch.

other stores

Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Chroma upsert.

Read the RAG recipe →

Get an API key →

Extract text into Chroma

The problem

Chunking for Chroma

Related

Wire the extraction step in.