How to extract text from a PDF for RAG (without maintaining a parser)

Every RAG pipeline eventually has to answer the same boring question: how do you turn a PDF into text you can chunk and embed? The obvious answer — pdftotext, or a Python library like pdfminer or PyPDF2 — works right up until it doesn’t. Multi-column academic papers interleave columns into nonsense. Tables collapse into runs of numbers with no structure. Scanned contracts and invoices, which are just images wrapped in a PDF container, return nothing at all.

None of these are edge cases if you’re ingesting real-world documents at any volume. They’re the median case.

Why PDF text extraction is harder than it looks

A PDF doesn’t store “text” the way a Markdown file does. It stores a sequence of drawing instructions — put this glyph at this (x, y) coordinate, with this font. There is no inherent concept of “this is a paragraph” or “this is a table cell.” Extractors have to reconstruct reading order from glyph positions, which is where multi-column and table layouts fall apart: two columns of text at the same vertical position look, to a naive extractor, like one long garbled line.

And when the PDF is a scan — a photographed invoice, a faxed contract, a flattened image-only export — there’s no text layer to extract in the first place. You need OCR, which most “extract text from PDF” library calls silently don’t do.

For a RAG pipeline, garbled or missing text is worse than an error: it gets chunked, embedded, and retrieved with confidence, and nobody notices until a user asks a question the source document actually answers correctly on page one.

A one-request approach

txtfetch handles all three cases — normal PDFs, multi-column layouts, and scanned pages — behind a single HTTP call. It runs Apache Tika for structural text extraction and layout-aware reading order, and falls back to Tesseract OCR automatically when a page has no extractable text layer. You don’t tell it which path to take; it detects the document and picks one.

If you already have a PDF at a URL:

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

Or if you’re pulling files out of an upload flow, POST the bytes directly:

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@quarterly-report.pdf

Either way, the response is the same shape:

{
  "status": "success",
  "extracted_text": "Q3 revenue grew 34% year over year, driven by..."
}

That’s extracted_text ready to chunk and hand to your embedding model — no intermediate layout object to parse, no separate OCR branch to write and maintain.

Fitting it into a RAG pipeline

The integration point is wherever your ingestion pipeline currently calls a local PDF library. Replace that function call with a request to txtfetch and keep everything downstream — your chunker, your embedder, your vector store — exactly as it is:

curl -s -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@handbook.pdf | jq -r .extracted_text | your-chunker

Failures come back as explicit JSON ({"status": "error", "error": "..."}) instead of an empty string silently swallowed into an empty chunk, so bad documents fail loudly in your pipeline logs instead of quietly poisoning your index.

The result: no pdfminer version pinned in your requirements.txt, no separate Tesseract install to keep patched, no special-casing for scanned intake. One endpoint, and it’s already handled the PDF you’re about to throw at it.

Want to try it against your own documents? Get in touch and we’ll set you up with an API key.