> Source: https://txtfetch.com/ingest/weaviate
> Plain-text twin — every page on txtfetch.com has one. https://txtfetch.com/text

---

# Extract text into Weaviate

An open-source and managed vector database whose client API renamed its own core method this year. Here's what changes between the text txtfetch returns and the first row you write to Weaviate.

## The problem

Weaviate's object properties hold the chunk text with no fuss. The trap is elsewhere. Object ids must be a valid UUID, and the Python client's own v4 API keeps moving, down to renaming the method that fetches a collection. A tutorial from even a year ago can silently call the wrong thing.

what weaviate needs

| Per-vector metadata limit | No published per-property byte ceiling. Bound by the overall request size, mostly relevant over gRPC. |
| --- | --- |
| Stores the chunk text itself | Yes. The chunk text is an ordinary schema property, for example content, on the same object as the vector. |
| ID format | A UUID. Weaviate can generate one deterministically from your own fields with generate\_uuid5(...) if you don't supply one. |
| Max vector dimensions | 65,535 dimensions, stored as a uint16 index. Memory runs out long before that ceiling matters. |
| Batch upsert size | No fixed cap. collection.data.insert\_many(...) batches server-side, so you don't tune a batch size by hand. |
| Index type | HNSW by default, with flat, dynamic, and hfresh alternatives, plus PQ, BQ, SQ, and RQ quantization. |

No accuracy, speed, or recall figure is measured against Weaviate on this page. See [/benchmarks](https://txtfetch.com/benchmarks) for txtfetch's own measured extraction numbers.

the pipeline

Extract, chunk, embed, and upsert. No framework, just the [txtfetch Python SDK](https://txtfetch.com/docs/quickstarts) and Weaviate's own client.

Python

```python
import os

import weaviate
from openai import OpenAI
from weaviate.classes.data import DataObject
from weaviate.classes.init import Auth
from weaviate.util import generate_uuid5
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
oai = OpenAI()
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
collection = client.collections.use("Docs")


def chunk_text(text: str, size: int = 1000, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Chunk
chunks = chunk_text(result.extracted_text)

# 3. Embed
response = oai.embeddings.create(model="text-embedding-3-small", input=chunks)

# 4. Upsert — object ids must be a UUID, generated here from the chunk's own
# content so a re-run overwrites the same object instead of duplicating it.
objects = [
    DataObject(
        properties={"source": "whitepaper.pdf", "content": chunk},
        vector=item.embedding,
        uuid=generate_uuid5(chunk),
    )
    for chunk, item in zip(chunks, response.data)
]

collection.data.insert_many(objects)
client.close()
```

Every call above was resolved against weaviate-client 4.18.3, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

## Chunking for Weaviate

Weaviate's property values carry no published size ceiling, so chunk size is a retrieval-quality decision, not a Weaviate constraint. The real trap is the object id. Weaviate only accepts a UUID, so generate one deterministically from the chunk's own content with generate\_uuid5, rather than inventing your own scheme. Preview your chunker's actual boundaries in the chunk previewer before your first batch.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a `202` plus a `job_id` to poll, and `?async=true` forces that path for any request. Pass `webhook_url` instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See [async jobs & webhooks](https://txtfetch.com/docs/async) for the full lifecycle.

Submit (async)

```submit
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}
```

Poll

```poll
curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.
```

curl

```curl
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"
```

A retried request, from a client timeout or a dropped response, shouldn't extract and upsert the same document twice. Send the same `Idempotency-Key` on a retry and txtfetch returns the first result instead of running the job again. See [idempotency](https://txtfetch.com/docs/idempotency) for the full guarantee.

curl

```curl
curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf
```

JavaScript

```javascript
// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });
```

Python

```python
# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")
```

txtfetch ships no connector, plugin, or client for Weaviate. The pipeline above is the whole integration. Extract with txtfetch, then write to Weaviate with Weaviate's own client, the same as you would for any other source of text.

frequently asked questions

**Does txtfetch write directly to Weaviate?**: No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the upsert itself stay a caller-side step, exactly like the pipeline above shows.

**Why does the client API matter this much for Weaviate specifically?**: Weaviate's Python client moved to a new v4 shape, and even within v4 the method that fetches a collection was renamed. A snippet copied from an older guide can call a method that no longer exists. Check it against Weaviate's own current docs before you ship it.

**Does txtfetch benchmark accuracy or speed against Weaviate?**: No. Weaviate is a vector database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

sources

-   [Weaviate: Python client library (v4, connect, collections.use)](https://docs.weaviate.io/weaviate/client-libraries/python) Accessed 2026-09
-   [Weaviate Cloud: connect from Python](https://docs.weaviate.io/cloud/manage-clusters/connect) Accessed 2026-09
-   [Weaviate: Import data (batch ingest, custom vectors, UUIDs)](https://docs.weaviate.io/weaviate/manage-data/import) Accessed 2026-09
-   [Weaviate: FAQ (vector dimension limit)](https://weaviate.io/developers/weaviate/more-resources/faq) Accessed 2026-09
-   [Weaviate: Collection configuration reference (index types)](https://docs.weaviate.io/weaviate/config-refs/collections) Accessed 2026-09

## Related

-   [RAG & LLM ingestion](https://txtfetch.com/solutions/rag-ingestion), the wider use case this pipeline feeds.
-   [RAG recipe: chunk, embed, index](https://txtfetch.com/docs/recipe), the same pipeline through LangChain or LlamaIndex instead.
-   [Chunk previewer](https://txtfetch.com/tools/chunk-preview), to test your own extracted text before you write the first row.
-   [Async jobs & webhooks](https://txtfetch.com/docs/async), the full lifecycle for a large batch.

other stores

-   [Extract text into pgvector →](https://txtfetch.com/ingest/pgvector)
-   [Extract text into Pinecone →](https://txtfetch.com/ingest/pinecone)
-   [Extract text into Qdrant →](https://txtfetch.com/ingest/qdrant)
-   [Extract text into Chroma →](https://txtfetch.com/ingest/chroma)
-   [All destinations →](https://txtfetch.com/ingest)

## Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Weaviate upsert.

[Read the RAG recipe →](https://txtfetch.com/docs/recipe)

[Get an API key →](https://app.txtfetch.com/signup)