Ingesting large documents and big batches without falling over

Two kinds of scale break ingestion pipelines that work fine in a demo: one very large document, and a very large number of ordinary ones. They look different but stress the same two things — how long a single request takes, and how many requests you’re allowed to have in flight at once.

Large single documents

/v1/extract is a synchronous endpoint — there’s no async job/webhook mode, you send a request and get extracted_text back on that same connection. For a 500-page report or a multi-hundred-slide deck, that means the request itself takes longer, and your client needs a timeout that reflects it rather than the short timeout you’d set for a two-page memo:

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@annual-report-500pg.pdf \
  --max-time 300

If your HTTP client library defaults to a short timeout (many do — 10 or 30 seconds), raise it explicitly for extraction calls rather than discovering the hard way that your largest documents are the ones silently failing.

Large batches

For a backfill — ingesting an existing document archive rather than a steady trickle of new uploads — the constraint shifts from “how long does one request take” to “how many can I run at once without overwhelming either side.” A plain for loop making requests one at a time works but wastes most of your batch window waiting on network I/O. Some client-side concurrency, capped at a sane number, gets through a large batch without the plain sequential wait:

import asyncio
import httpx

SEM = asyncio.Semaphore(8)  # cap concurrent in-flight requests

async def extract(client, path, api_key):
    async with SEM:
        with open(path, "rb") as f:
            resp = await client.post(
                "https://api.txtfetch.com/v1/extract",
                headers={"Authorization": f"Bearer {api_key}"},
                files={"file": f},
                timeout=120,
            )
        body = resp.json()
        if body["status"] != "success":
            raise ValueError(f"{path}: {body.get('error')}")
        return path, body["extracted_text"]

async def run_batch(paths, api_key):
    async with httpx.AsyncClient() as client:
        results = await asyncio.gather(
            *(extract(client, p, api_key) for p in paths),
            return_exceptions=True,
        )
    return results

return_exceptions=True matters here: one bad file in a ten-thousand-file backfill shouldn’t take down the other 9,999. Collect the failures separately and retry or route them to a review queue instead of letting one exception cancel the whole batch.

Retry and backoff

Treat a failed request as retryable unless the response tells you otherwise. An explicit {"status": "error", "error": "..."} for a genuinely unreadable file (corrupt, unsupported, empty) isn’t worth retrying — retrying the same broken input just wastes another request. A transport-level failure (timeout, connection reset) usually is worth a retry, with backoff so a batch of failures doesn’t turn into a retry storm:

async def extract_with_retry(client, path, api_key, attempts=3):
    for attempt in range(attempts):
        try:
            return await extract(client, path, api_key)
        except (httpx.TimeoutException, httpx.TransportError):
            if attempt == attempts - 1:
                raise
            await asyncio.sleep(2 ** attempt)

There’s no idempotency key or job ID in play here — each request is independent, so a retry is just “send the same file again,” not a resume of a partially-completed job. That’s the tradeoff of a synchronous, no-job-state API: simpler to reason about, at the cost of the client owning concurrency and retry logic rather than polling a job status endpoint.

Where this fits into cost

Once ingestion completes reliably at whatever batch size you’re running, the next question is usually what it costs to run it repeatedly — see the per-document vs per-page pricing guide for how document count and page count each factor into that. If scanned documents are part of the batch, the OCR guide covers the one thing that changes: OCR requests can legitimately take longer than native-text extraction, which is worth accounting for in your timeout and concurrency tuning.

Planning a backfill and want to talk through concurrency limits? Get in touch and we’ll set you up with an API key.