search⌘KGet started

Extract text into Meilisearch

A typo-tolerant search engine that ignores every word past position 65,535 in a field, with no warning. Here's what changes between the text txtfetch returns and the first row you write to Meilisearch.

The problem

Meilisearch indexes at most 65,535 word positions per attribute. It ignores every word after that, with no error. So a long document in one field looks fine. The write succeeds and early matches work. Search just never finds anything near the end.

what meilisearch needs

Constraints that shape splitting and the write step, cited and dated
Where long text stops counting65,535 word positions per attribute. Meilisearch ignores every later word in that field, with no error and no warning.
Max request size100 MB per request by default. You can change it on a self-hosted instance.
Built-in document extractionNone. Meilisearch indexes the JSON you send. It does not extract text from files.
Keeps the full textYes. Meilisearch stores every field you send and can return it as-is.
Document ID formatA string or an integer. Use only letters, digits, hyphens, and underscores. The docs name no length limit.
Bulk write APIadd_documents(...). It returns a task that you can poll until the write finishes.

No accuracy, speed, or recall figure is measured against Meilisearch on this page. See /benchmarks for txtfetch's own measured extraction numbers.

Splitting for Meilisearch

Index one document per page or section, each with its own id and a parent_id field. The position limit applies per attribute, not per document. So keep each section's text field well under 65,535 words. To find real section breaks, use the page numbers in format=json output. Or test your text in the chunk previewer at /tools/chunk-preview.

the pipeline

Extract, split, and bulk-write. No framework, just the txtfetch Python SDK and Meilisearch's own client.

Python
import os

import meilisearch
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
client = meilisearch.Client(os.environ["MEILISEARCH_URL"], os.environ["MEILISEARCH_API_KEY"])
index = client.index("docs")


def split_text(text: str, size: int = 2000, overlap: int = 200) -> list[str]:
    sections = []
    start = 0
    while start < len(text):
        end = start + size
        sections.append(text[start:end])
        start = end - overlap
    return sections


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Split
sections = split_text(result.extracted_text)

# 3. Bulk write — one JSON object per section, tagged with the parent
# document it came from.
documents = [
    {"id": f"whitepaper-{i}", "parent_id": "whitepaper.pdf", "section": i, "text": section}
    for i, section in enumerate(sections)
]
index.add_documents(documents)

Every call above was resolved against meilisearch 0.43.0, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a 202 plus a job_id to poll, and ?async=true forces that path for any request. Pass webhook_url instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See async jobs & webhooks for the full lifecycle.

Submit (async)
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}
Poll
curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.
curl
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"

A retried request, from a client timeout or a dropped response, shouldn't extract and write the same document twice. Send the same Idempotency-Key on a retry and txtfetch returns the first result instead of running the job again. See idempotency for the full guarantee.

curl
curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf
JavaScript
// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });
Python
# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")

txtfetch ships no connector, plugin, or client for Meilisearch. The pipeline above is the whole integration. Extract with txtfetch. Then write the result with Meilisearch's own client.

frequently asked questions

Does txtfetch write directly to Meilisearch?
No. txtfetch returns the extracted text over one HTTP call. You split the text and call add_documents with Meilisearch's own client.
Why does the position limit matter more than the size limit?
The 100 MB request limit is large. The position limit is the one you hit first. One long field loses every word past position 65,535, and nothing tells you.
Does txtfetch benchmark accuracy or speed against Meilisearch?
No. Meilisearch is a search engine, not a text-extraction service. There is no accuracy or speed comparison to make. See /benchmarks for txtfetch's own extraction numbers.

sources

Related

other engines

Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Meilisearch upsert.

Read the RAG recipe →

Get an API key →