> Source: https://txtfetch.com/ingest/postgres-full-text
> Plain-text twin — every page on txtfetch.com has one. https://txtfetch.com/text

---

# Extract text into Postgres full-text search

Built into Postgres, and it silently clamps every word position past 16,383. Here's what changes between the text txtfetch returns and the first row you write to Postgres full-text search.

## The problem

A tsvector stores each lexeme with the positions where it appears. Postgres caps positions at 16,383. It silently sets every later position to 16,383, with no error. So in a long document stored as one row, the tail of the text collapses onto one position. Plain word matches still work. Phrase and proximity searches on that tail do not.

what postgres full-text search needs

| Where long text stops counting | Word positions stop at 16,383. Postgres silently sets every later position to 16,383. It also keeps at most 256 positions per lexeme and drops the rest. Only a tsvector over 1 MB raises an error. |
| --- | --- |
| Max request size | None from full-text search. Normal Postgres statement and row limits apply. Use COPY or executemany for a bulk load. |
| Built-in document extraction | None. to\_tsvector() parses text you already have. It does not extract text from files. |
| Keeps the full text | Yes. The source text is a normal column in the same row as its generated tsvector. |
| Document ID format | None from full-text search. Use the primary key type your table declares, such as bigserial, uuid, or text. |
| Bulk write API | No bulk endpoint. Use executemany(...) for a moderate batch, or COPY for a large one. |

No accuracy, speed, or recall figure is measured against Postgres full-text search on this page. See [/benchmarks](https://txtfetch.com/benchmarks) for txtfetch's own measured extraction numbers.

## Splitting for Postgres full-text search

Index one row per page or section, with a parent\_id column that points back to the source file. The limits apply per row. Keep each row well under 16,383 words, so no position gets clamped. To find real section breaks, use the page numbers in format=json output. Or test your text in the chunk previewer at /tools/chunk-preview.

the pipeline

Extract, split, and bulk-write. No framework, just the [txtfetch Python SDK](https://txtfetch.com/docs/quickstarts) and Postgres full-text search's own client.

Python

```python
import os

import psycopg
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])

conn = psycopg.connect(os.environ["DATABASE_URL"], autocommit=True)
conn.execute(
    """
    CREATE TABLE IF NOT EXISTS docs (
        id bigserial PRIMARY KEY,
        parent_id text,
        section int,
        content text,
        search tsvector GENERATED ALWAYS AS (to_tsvector('english', content)) STORED
    )
    """
)
conn.execute("CREATE INDEX IF NOT EXISTS docs_search_idx ON docs USING gin (search)")


def split_text(text: str, size: int = 2000, overlap: int = 200) -> list[str]:
    sections = []
    start = 0
    while start < len(text):
        end = start + size
        sections.append(text[start:end])
        start = end - overlap
    return sections


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Split — Postgres silently clamps every word position past 16,383, so
# one long row loses phrase and proximity search on its tail.
sections = split_text(result.extracted_text)

# 3. Bulk write — the generated column builds the tsvector automatically.
rows = [("whitepaper.pdf", i, section) for i, section in enumerate(sections)]
with conn.cursor() as cur:
    cur.executemany(
        "INSERT INTO docs (parent_id, section, content) VALUES (%s, %s, %s)",
        rows,
    )
```

Every call above was resolved against psycopg 3.3.6, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a `202` plus a `job_id` to poll, and `?async=true` forces that path for any request. Pass `webhook_url` instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See [async jobs & webhooks](https://txtfetch.com/docs/async) for the full lifecycle.

Submit (async)

```submit
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}
```

Poll

```poll
curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.
```

curl

```curl
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"
```

A retried request, from a client timeout or a dropped response, shouldn't extract and write the same document twice. Send the same `Idempotency-Key` on a retry and txtfetch returns the first result instead of running the job again. See [idempotency](https://txtfetch.com/docs/idempotency) for the full guarantee.

curl

```curl
curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf
```

JavaScript

```javascript
// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });
```

Python

```python
# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")
```

txtfetch ships no connector, plugin, or client for Postgres full-text search. The pipeline above is the whole integration. Extract with txtfetch. Then write the result with Postgres full-text search's own client.

frequently asked questions

**Does txtfetch write directly to Postgres?**: No. txtfetch returns the extracted text over one HTTP call. You split the text and run the INSERT yourself, with psycopg or any other Postgres client.

**Does Postgres warn me when a row goes over a limit?**: Only for the 1 MB tsvector cap, which raises an error. Positions past 16,383 are silently set to 16,383. Positions past 256 for one lexeme are silently dropped. A word over 2 KB gets a notice and stays out of the index. So split the document before you insert it.

**Does txtfetch benchmark accuracy or speed against Postgres full-text search?**: No. Postgres full-text search is a database feature, not a text-extraction service. There is no accuracy or speed comparison to make. See /benchmarks for txtfetch's own extraction numbers.

sources

-   [PostgreSQL: Full Text Search limitations (tsvector size, position caps, lexeme length)](https://www.postgresql.org/docs/current/textsearch-limitations.html) Accessed 2026-09
-   [PostgreSQL: Text search types (positions above 16,383 silently set to 16,383)](https://www.postgresql.org/docs/current/datatype-textsearch.html) Accessed 2026-09
-   [PostgreSQL source: to\_tsany.c (256 positions per lexeme, extras skipped)](https://github.com/postgres/postgres/blob/master/src/backend/tsearch/to_tsany.c) Accessed 2026-09
-   [PostgreSQL: Generated columns](https://www.postgresql.org/docs/current/ddl-generated-columns.html) Accessed 2026-09
-   [psycopg: connection and cursor API](https://www.psycopg.org/psycopg3/docs/api/connections.html) Accessed 2026-09

## Related

-   [Search indexing](https://txtfetch.com/solutions/search-indexing), the wider use case this pipeline feeds.
-   [Chunk previewer](https://txtfetch.com/tools/chunk-preview), to test your own extracted text before you write the first row.
-   [Async jobs & webhooks](https://txtfetch.com/docs/async), the full lifecycle for a large batch.
-   [All ingest destinations](https://txtfetch.com/ingest), vector stores and search engines together.

other engines

-   [Extract text into Elasticsearch →](https://txtfetch.com/ingest/elasticsearch)
-   [Extract text into OpenSearch →](https://txtfetch.com/ingest/opensearch)
-   [Extract text into Meilisearch →](https://txtfetch.com/ingest/meilisearch)
-   [Extract text into Typesense →](https://txtfetch.com/ingest/typesense)
-   [All destinations →](https://txtfetch.com/ingest)

## Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Postgres full-text search upsert.

[Read the RAG recipe →](https://txtfetch.com/docs/recipe)

[Get an API key →](https://app.txtfetch.com/signup)