> Source: https://txtfetch.com/ingest/pgvector
> Plain-text twin — every page on txtfetch.com has one. https://txtfetch.com/text

---

# Extract text into pgvector

A Postgres extension that adds a vector column type to a database you already run. Here's what changes between the text txtfetch returns and the first row you write to pgvector.

## The problem

pgvector is a Postgres column type, not a service. Postgres has no idea what a chunk is. You design the table, pick a vector width that matches your embedding model, and write the insert yourself. A bad chunk just becomes a bad row, with no store-side warning.

what pgvector needs

| Per-vector metadata limit | None from pgvector itself. You're bound by ordinary Postgres limits: up to 1 GB per field, with TOAST moving large text out of the row automatically. |
| --- | --- |
| Stores the chunk text itself | Yes. The chunk text is just another column in the same table as the vector. |
| ID format | None. Use whatever primary key type the table declares: bigserial, uuid, or text. |
| Max vector dimensions | 16,000 for a plain vector column. Building an HNSW or IVFFlat index on it caps out at 2,000 dimensions, or 4,000 for a halfvec column. |
| Batch upsert size | None from pgvector. You're bound by ordinary Postgres statement size. Use COPY for a bulk load. |
| Index type | Exact scan with no index, IVFFlat, or HNSW. HNSW is the common default today. |

No accuracy, speed, or recall figure is measured against pgvector on this page. See [/benchmarks](https://txtfetch.com/benchmarks) for txtfetch's own measured extraction numbers.

the pipeline

Extract, chunk, embed, and upsert. No framework, just the [txtfetch Python SDK](https://txtfetch.com/docs/quickstarts) and pgvector's own client.

Python

```python
import os

import psycopg
from openai import OpenAI
from pgvector import Vector
from pgvector.psycopg import register_vector
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
oai = OpenAI()

conn = psycopg.connect(os.environ["DATABASE_URL"], autocommit=True)
conn.execute("CREATE EXTENSION IF NOT EXISTS vector")
register_vector(conn)
conn.execute(
    """
    CREATE TABLE IF NOT EXISTS chunks (
        id bigserial PRIMARY KEY,
        source text,
        content text,
        embedding vector(1536)
    )
    """
)


def chunk_text(text: str, size: int = 1000, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Chunk
chunks = chunk_text(result.extracted_text)

# 3. Embed
response = oai.embeddings.create(model="text-embedding-3-small", input=chunks)
vectors = [Vector(item.embedding) for item in response.data]

# 4. Upsert
rows = [("whitepaper.pdf", text, vector) for text, vector in zip(chunks, vectors)]
with conn.cursor() as cur:
    cur.executemany(
        "INSERT INTO chunks (source, content, embedding) VALUES (%s, %s, %s)",
        rows,
    )

# Postgres requires an index name when you write IF NOT EXISTS.
conn.execute(
    "CREATE INDEX IF NOT EXISTS chunks_embedding_idx "
    "ON chunks USING hnsw (embedding vector_cosine_ops)"
)
```

Every call above was resolved against pgvector 0.4.2 with psycopg 3.2.13, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

## Chunking for pgvector

pgvector puts no ceiling on chunk length. The real constraint is your vector column width. It has to match your embedding model's output size, and an HNSW index on that column caps out at 2,000 dimensions, or 4,000 for halfvec. Pick a chunk size for retrieval quality, then check the actual boundaries in the chunk previewer before you write a single row.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a `202` plus a `job_id` to poll, and `?async=true` forces that path for any request. Pass `webhook_url` instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See [async jobs & webhooks](https://txtfetch.com/docs/async) for the full lifecycle.

Submit (async)

```submit
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}
```

Poll

```poll
curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.
```

curl

```curl
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"
```

A retried request, from a client timeout or a dropped response, shouldn't extract and upsert the same document twice. Send the same `Idempotency-Key` on a retry and txtfetch returns the first result instead of running the job again. See [idempotency](https://txtfetch.com/docs/idempotency) for the full guarantee.

curl

```curl
curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf
```

JavaScript

```javascript
// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });
```

Python

```python
# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")
```

txtfetch ships no connector, plugin, or client for pgvector. The pipeline above is the whole integration. Extract with txtfetch, then write to pgvector with pgvector's own client, the same as you would for any other source of text.

frequently asked questions

**Does txtfetch write directly to pgvector?**: No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the INSERT itself stay a caller-side step, exactly like the pipeline above shows.

**Why does the index dimension limit matter more than the column limit?**: A plain vector column accepts up to 16,000 dimensions, but building an HNSW or IVFFlat index on it caps out at 2,000. Most embedding models stay well under that, but check yours before you commit to a schema.

**Does txtfetch benchmark accuracy or speed against pgvector?**: No. pgvector is a database column type, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

sources

-   [pgvector: README (types, indexes, dimension limits)](https://github.com/pgvector/pgvector) Accessed 2026-09
-   [pgvector-python: README (install, register\_vector, insert pattern)](https://github.com/pgvector/pgvector-python) Accessed 2026-09
-   [PostgreSQL: Appendix K, limits (row size, TOAST)](https://www.postgresql.org/docs/current/limits.html) Accessed 2026-09

## Related

-   [RAG & LLM ingestion](https://txtfetch.com/solutions/rag-ingestion), the wider use case this pipeline feeds.
-   [RAG recipe: chunk, embed, index](https://txtfetch.com/docs/recipe), the same pipeline through LangChain or LlamaIndex instead.
-   [Chunk previewer](https://txtfetch.com/tools/chunk-preview), to test your own extracted text before you write the first row.
-   [Async jobs & webhooks](https://txtfetch.com/docs/async), the full lifecycle for a large batch.

other stores

-   [Extract text into Pinecone →](https://txtfetch.com/ingest/pinecone)
-   [Extract text into Qdrant →](https://txtfetch.com/ingest/qdrant)
-   [Extract text into Chroma →](https://txtfetch.com/ingest/chroma)
-   [Extract text into Weaviate →](https://txtfetch.com/ingest/weaviate)
-   [All destinations →](https://txtfetch.com/ingest)

## Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the pgvector upsert.

[Read the RAG recipe →](https://txtfetch.com/docs/recipe)

[Get an API key →](https://app.txtfetch.com/signup)