Ingesting large documents and big batches without falling over

Two kinds of scale break ingestion pipelines that work fine in a demo. One is a single very large document. The other is a very large number of ordinary ones. They look different. But they stress the same two things: how long a single request takes, and how many you can run at once.

Large single documents

/v1/extract is a synchronous endpoint. There’s no async job or webhook mode. You send a request and get extracted_text back on the same connection. For a 500-page report or a multi-hundred-slide deck, the request itself takes longer. Your client needs a longer timeout too. Don’t use the short timeout you’d set for a two-page memo:

curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -F file=@annual-report-500pg.pdf \
  --max-time 300

Many HTTP client libraries default to a short timeout, sometimes 10 or 30 seconds. Raise it explicitly for extraction calls. Otherwise you’ll discover the hard way that your largest documents are the ones that fail silently.

Large batches

A backfill means ingesting an existing document archive, not a steady trickle of new uploads. There the constraint shifts. It’s no longer “how long does one request take” but “how many can I run at once without overwhelming either side.” A plain for loop making requests one at a time works. But it wastes most of your batch window waiting on network I/O. Some client-side concurrency, capped at a sane number, gets through a large batch faster than a sequential wait:

import asyncio
import httpx

SEM = asyncio.Semaphore(8)  # cap concurrent in-flight requests

async def extract(client, path, api_key):
    async with SEM:
        with open(path, "rb") as f:
            resp = await client.post(
                "https://api.txtfetch.com/v1/extract",
                headers={"Authorization": f"Bearer {api_key}"},
                files={"file": f},
                timeout=120,
            )
        body = resp.json()
        if body["status"] != "success":
            raise ValueError(f"{path}: {body.get('error')}")
        return path, body["extracted_text"]

async def run_batch(paths, api_key):
    async with httpx.AsyncClient() as client:
        results = await asyncio.gather(
            *(extract(client, p, api_key) for p in paths),
            return_exceptions=True,
        )
    return results

return_exceptions=True matters here: one bad file in a ten-thousand-file backfill shouldn’t take down the other 9,999. Collect the failures separately and retry or route them to a review queue instead of letting one exception cancel the whole batch.

Retry and backoff

Treat a failed request as retryable unless the response tells you otherwise. An explicit {"status": "error", "error": "..."} for a genuinely unreadable file (corrupt, unsupported, empty) isn’t worth retrying. Retrying the same broken input just wastes another request. A transport-level failure (timeout, connection reset) is usually worth a retry. Add backoff so a batch of failures doesn’t turn into a retry storm:

async def extract_with_retry(client, path, api_key, attempts=3):
    for attempt in range(attempts):
        try:
            return await extract(client, path, api_key)
        except (httpx.TimeoutException, httpx.TransportError):
            if attempt == attempts - 1:
                raise
            await asyncio.sleep(2 ** attempt)

There’s no idempotency key or job ID here. Each request is independent. A retry just means “send the same file again,” not resuming a partially completed job. That’s the tradeoff of a synchronous, no-job-state API. It’s simpler to reason about. But the client owns concurrency and retry logic instead of polling a job status endpoint.

Where this fits into cost

Once ingestion completes reliably at whatever batch size you’re running, the next question is usually cost. What does it cost to run this repeatedly? See the per-document vs per-page pricing guide for how document count and page count factor in.

If scanned documents are part of the batch, read the OCR guide too. OCR requests can take longer than native-text extraction. Account for that in your timeout and concurrency tuning.

Planning a backfill and want to talk through concurrency limits? Get in touch and we’ll set you up with an API key.