Feed your RAG pipeline clean text, not parser output.

the-problem

RAG pipelines live or die on what goes into the vector store, and most ingestion code spends more time on format detection than on chunking. A production knowledge base ends up needing a PDF library, an Office parser, an OCR fallback for scanned pages, and glue code to normalize their different outputs into one string before chunking even starts — and every new format users upload (an .eml thread, a .pptx deck) is another parser to add, another way for retrieval quality to silently degrade when a parser mis-handles a table or drops a scanned page's OCR.

how-txtfetch-solves-it

txtfetch collapses that into one POST: PDF, Office file, scanned image, or a URL, always comes back as the same { status, extracted_text } shape — a single normalized string ready for RecursiveCharacterTextSplitter or your chunker of choice. Text-layer pages go through Apache Tika, pages with no text layer route through Tesseract OCR automatically, in the same request, so a batch of mixed digital-native and scanned documents needs no branching logic on your side.

• One response shape for every source format — no per-parser branch before chunking
• Automatic OCR fallback for scanned pages inside an otherwise-digital PDF batch
• Official LangChain and LlamaIndex loaders (langchain-txtfetch, llama-index-readers-txtfetch) drop straight into an existing splitter/embedder pipeline
• Pass a URL instead of downloading first — ingest linked documents server-side
• Async job + webhook mode for large batches so ingestion jobs don't block on slow OCR

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@whitepaper.pdf

import os
import requests

with open("whitepaper.pdf", "rb") as f:
    r = requests.post(
        "https://api.txtfetch.com/v1/extract",
        headers={"Authorization": f"Bearer {os.environ['TXTFETCH_KEY']}"},
        files={"file": f},
    )

print(r.json()["extracted_text"])

import { readFile } from "node:fs/promises";

const file = new Blob([await readFile("whitepaper.pdf")]);
const form = new FormData();
form.append("file", file, "whitepaper.pdf");

const res = await fetch("https://api.txtfetch.com/v1/extract", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.TXTFETCH_KEY}` },
  body: form,
});

const { extracted_text } = await res.json();
console.log(extracted_text);

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"mime/multipart"
	"net/http"
	"os"
)

type extractResponse struct {
	Status        string `json:"status"`
	ExtractedText string `json:"extracted_text"`
}

func main() {
	f, err := os.Open("whitepaper.pdf")
	if err != nil {
		panic(err)
	}
	defer f.Close()

	var body bytes.Buffer
	writer := multipart.NewWriter(&body)
	part, err := writer.CreateFormFile("file", "whitepaper.pdf")
	if err != nil {
		panic(err)
	}
	if _, err := io.Copy(part, f); err != nil {
		panic(err)
	}
	writer.Close()

	req, err := http.NewRequest("POST", "https://api.txtfetch.com/v1/extract", &body)
	if err != nil {
		panic(err)
	}
	req.Header.Set("Authorization", "Bearer "+os.Getenv("TXTFETCH_KEY"))
	req.Header.Set("Content-Type", writer.FormDataContentType())

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var result extractResponse
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		panic(err)
	}
	fmt.Println(result.ExtractedText)
}

{
  "status": "success",
  "extracted_text": "..."
}

faq

Does txtfetch work with LangChain or LlamaIndex?

Yes — langchain-txtfetch (Python) and @txtfetch/langchain (JS) are official document loaders, and llama-index-readers-txtfetch is an official LlamaIndex reader. Each wraps the extract API and returns Document objects ready for your text splitter.

What does txtfetch return for a scanned PDF in a RAG pipeline?

The same { status, extracted_text } shape as a digital-native PDF — pages with no text layer are OCR'd via Tesseract automatically, so your chunker doesn't need to know which pages were scanned.

Can I ingest a document directly from a URL instead of downloading it first?

Yes — pass a url parameter and txtfetch fetches the document server-side, the same code path the LangChain/LlamaIndex loaders use for their urls= argument.

related-reading

• Extract text from a PDF for RAG →
• Chunking strategies for RAG →
• Extract tables for RAG →
• The LangChain & LlamaIndex document loader →
• RAG recipe: chunk → embed → index →
• SDK & framework quickstarts →
• See how txtfetch compares →
• Get an API key →