Feed your RAG pipeline clean text, not parser output.

the-problem

RAG pipelines live or die on what enters the vector store. Most ingestion code spends more time on format detection than on chunking. A production knowledge base needs a PDF library, an Office parser, and an OCR fallback for scans. Each new upload format is one more parser to maintain, and one more way retrieval quality can degrade.

how-txtfetch-solves-it

txtfetch collapses that into one API call. PDF, Office file, scanned image, or a URL: the response always has the same shape. Pass that string straight to your chunker. Text-layer pages route through Apache Tika. Pages with no text layer route through Tesseract OCR automatically, in the same request. A batch of mixed digital and scanned documents needs no branching logic on your side. Add format=markdown to the same request when headings and tables need to survive the splitter.

• Every source format returns the same response shape, so no per-parser branch runs before chunking.
• OCR runs automatically on scanned pages inside an otherwise-digital PDF batch.
• Official LangChain and LlamaIndex loaders drop into an existing splitter and embedder pipeline.
• Pass a URL instead of downloading first, and txtfetch fetches the document server-side.
• Async job and webhook mode keeps large batches from blocking on slow OCR.
• Idempotency-Key support stops a retried ingestion job from re-embedding the same document.
• Add format=markdown to the same request to keep headings and tables intact for the splitter.

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@whitepaper.pdf

import os
import requests

with open("whitepaper.pdf", "rb") as f:
    r = requests.post(
        "https://api.txtfetch.com/v1/extract",
        headers={"Authorization": f"Bearer {os.environ['TXTFETCH_KEY']}"},
        files={"file": f},
    )

print(r.json()["extracted_text"])

import { readFile } from "node:fs/promises";

const file = new Blob([await readFile("whitepaper.pdf")]);
const form = new FormData();
form.append("file", file, "whitepaper.pdf");

const res = await fetch("https://api.txtfetch.com/v1/extract", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.TXTFETCH_KEY}` },
  body: form,
});

const { extracted_text } = await res.json();
console.log(extracted_text);

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"mime/multipart"
	"net/http"
	"os"
)

type extractResponse struct {
	Status        string `json:"status"`
	ExtractedText string `json:"extracted_text"`
}

func main() {
	f, err := os.Open("whitepaper.pdf")
	if err != nil {
		panic(err)
	}
	defer f.Close()

	var body bytes.Buffer
	writer := multipart.NewWriter(&body)
	part, err := writer.CreateFormFile("file", "whitepaper.pdf")
	if err != nil {
		panic(err)
	}
	if _, err := io.Copy(part, f); err != nil {
		panic(err)
	}
	writer.Close()

	req, err := http.NewRequest("POST", "https://api.txtfetch.com/v1/extract", &body)
	if err != nil {
		panic(err)
	}
	req.Header.Set("Authorization", "Bearer "+os.Getenv("TXTFETCH_KEY"))
	req.Header.Set("Content-Type", writer.FormDataContentType())

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var result extractResponse
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		panic(err)
	}
	fmt.Println(result.ExtractedText)
}

{
  "status": "success",
  "extracted_text": "..."
}

faq

Does txtfetch work with LangChain or LlamaIndex?

Yes. langchain-txtfetch (Python) and @txtfetch/langchain (JS) are official document loaders, and llama-index-readers-txtfetch is an official LlamaIndex reader. Each wraps the extract API and returns Document objects ready for your text splitter.

What does txtfetch return for a scanned PDF in a RAG pipeline?

The same { status, extracted_text } shape as a digital-native PDF. Pages with no text layer are OCR'd via Tesseract automatically, so your chunker doesn't need to know which pages were scanned.

Can I ingest a document directly from a URL instead of downloading it first?

Yes. Pass a url parameter and txtfetch fetches the document server-side, the same code path the LangChain and LlamaIndex loaders use for their urls= argument.

related-reading

• Extract text from a PDF for RAG →
• Chunking strategies for RAG →
• Extract tables for RAG →
• The LangChain & LlamaIndex document loader →
• RAG recipe: chunk → embed → index →
• SDK & framework quickstarts →
• Extract text into a vector store →
• Chunk previewer: test your own extracted text →
• Text vs Markdown vs element JSON, compared →
• See how txtfetch compares →
• Get an API key →