Does txtfetch write directly to Weaviate?

No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the upsert itself stay a caller-side step, exactly like the pipeline above shows.

Why does the client API matter this much for Weaviate specifically?

Weaviate's Python client moved to a new v4 shape, and even within v4 the method that fetches a collection was renamed. A snippet copied from an older guide can call a method that no longer exists. Check it against Weaviate's own current docs before you ship it.

Does txtfetch benchmark accuracy or speed against Weaviate?

No. Weaviate is a vector database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

Extract text into Weaviate

An open-source and managed vector database whose client API renamed its own core method this year. Here's what changes between the text txtfetch returns and the first row you write to Weaviate.

The problem

Weaviate's object properties hold the chunk text with no fuss. The trap is elsewhere. Object ids must be a valid UUID, and the Python client's own v4 API keeps moving, down to renaming the method that fetches a collection. A tutorial from even a year ago can silently call the wrong thing.

what weaviate needs

Constraints that shape chunking and the write step, cited and dated
Per-vector metadata limit	No published per-property byte ceiling. Bound by the overall request size, mostly relevant over gRPC.
Stores the chunk text itself	Yes. The chunk text is an ordinary schema property, for example content, on the same object as the vector.
ID format	A UUID. Weaviate can generate one deterministically from your own fields with generate_uuid5(...) if you don't supply one.
Max vector dimensions	65,535 dimensions, stored as a uint16 index. Memory runs out long before that ceiling matters.
Batch upsert size	No fixed cap. collection.data.insert_many(...) batches server-side, so you don't tune a batch size by hand.
Index type	HNSW by default, with flat, dynamic, and hfresh alternatives, plus PQ, BQ, SQ, and RQ quantization.

No accuracy, speed, or recall figure is measured against Weaviate on this page. See /benchmarks for txtfetch's own measured extraction numbers.

the pipeline

Extract, chunk, embed, and upsert. No framework, just the txtfetch Python SDK and Weaviate's own client.

Python

import os

import weaviate
from openai import OpenAI
from weaviate.classes.data import DataObject
from weaviate.classes.init import Auth
from weaviate.util import generate_uuid5
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
oai = OpenAI()
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
collection = client.collections.use("Docs")


def chunk_text(text: str, size: int = 1000, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Chunk
chunks = chunk_text(result.extracted_text)

# 3. Embed
response = oai.embeddings.create(model="text-embedding-3-small", input=chunks)

# 4. Upsert — object ids must be a UUID, generated here from the chunk's own
# content so a re-run overwrites the same object instead of duplicating it.
objects = [
    DataObject(
        properties={"source": "whitepaper.pdf", "content": chunk},
        vector=item.embedding,
        uuid=generate_uuid5(chunk),
    )
    for chunk, item in zip(chunks, response.data)
]

collection.data.insert_many(objects)
client.close()

Every call above was resolved against weaviate-client 4.18.3, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

Chunking for Weaviate

Weaviate's property values carry no published size ceiling, so chunk size is a retrieval-quality decision, not a Weaviate constraint. The real trap is the object id. Weaviate only accepts a UUID, so generate one deterministically from the chunk's own content with generate_uuid5, rather than inventing your own scheme. Preview your chunker's actual boundaries in the chunk previewer before your first batch.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a 202 plus a job_id to poll, and ?async=true forces that path for any request. Pass webhook_url instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See async jobs & webhooks for the full lifecycle.

Submit (async)

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}

Poll

curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.

curl

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"

A retried request, from a client timeout or a dropped response, shouldn't extract and upsert the same document twice. Send the same Idempotency-Key on a retry and txtfetch returns the first result instead of running the job again. See idempotency for the full guarantee.

curl

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf

JavaScript

// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });

Python

# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")

txtfetch ships no connector, plugin, or client for Weaviate. The pipeline above is the whole integration. Extract with txtfetch, then write to Weaviate with Weaviate's own client, the same as you would for any other source of text.

frequently asked questions

Does txtfetch write directly to Weaviate?: No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the upsert itself stay a caller-side step, exactly like the pipeline above shows.
Why does the client API matter this much for Weaviate specifically?: Weaviate's Python client moved to a new v4 shape, and even within v4 the method that fetches a collection was renamed. A snippet copied from an older guide can call a method that no longer exists. Check it against Weaviate's own current docs before you ship it.
Does txtfetch benchmark accuracy or speed against Weaviate?: No. Weaviate is a vector database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

sources

Weaviate: Python client library (v4, connect, collections.use)Accessed 2026-09
Weaviate Cloud: connect from PythonAccessed 2026-09
Weaviate: Import data (batch ingest, custom vectors, UUIDs)Accessed 2026-09
Weaviate: FAQ (vector dimension limit)Accessed 2026-09
Weaviate: Collection configuration reference (index types)Accessed 2026-09

RAG & LLM ingestion, the wider use case this pipeline feeds.
RAG recipe: chunk, embed, index, the same pipeline through LangChain or LlamaIndex instead.
Chunk previewer, to test your own extracted text before you write the first row.
Async jobs & webhooks, the full lifecycle for a large batch.

other stores

Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Weaviate upsert.

Read the RAG recipe →

Get an API key →

Extract text into Weaviate

The problem

Chunking for Weaviate

Related

Wire the extraction step in.