Extract text the moment a file lands in S3.

the-problem

A Lambda triggered by an S3 upload usually has to read the object into memory before it can do anything with it. Lambda's own memory and execution-time budget is the same limited pool the rest of the function's logic has to share. Streaming a large file through the function just to re-upload it as a multipart request wastes that budget. It spends resources on a document the function itself never needs to inspect. Lambda's execution limit also makes it a poor place to sit and wait on a slow OCR job.

how-to-wire-it-up

There is no txtfetch Lambda layer or S3 integration to attach. An S3 object-created event triggering a plain Lambda function, which calls the API directly, is the entire integration. Rather than streaming the uploaded object through the function, generate a short-lived presigned GET URL for it. Pass that as the url query parameter — txtfetch fetches the object directly from S3. The function's own memory footprint stays small, regardless of the file's size. For a large or slow document, pass async=true and webhook_url so the function returns immediately. A second Lambda, subscribed to the callback, finishes the job.

• Pass ?url= with a presigned S3 URL, so the Lambda function never streams the full object through its own memory.
• Automatic OCR covers scanned uploads and photographed documents, with no extra service to wire into the pipeline.
• Async mode returns a job_id immediately, so a slow OCR job never risks the function's execution-time limit.
• A second Lambda, subscribed to the webhook_url callback, keeps the upload-triggered function itself short and cheap to run.
• Idempotency-Key support keeps a re-delivered S3 event notification from re-billing the same extraction.

pass-a-link-not-a-file

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/report.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

import os
import requests

r = requests.post(
    "https://api.txtfetch.com/v1/extract",
    headers={"Authorization": f"Bearer {os.environ['TXTFETCH_KEY']}"},
    params={"url": "https://example.com/report.pdf"},
)

print(r.json()["extracted_text"])

const endpoint = new URL("https://api.txtfetch.com/v1/extract");
endpoint.searchParams.set("url", "https://example.com/report.pdf");

const res = await fetch(endpoint, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.TXTFETCH_KEY}` },
});

const { extracted_text } = await res.json();
console.log(extracted_text);

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"net/url"
	"os"
)

type extractResponse struct {
	Status        string `json:"status"`
	ExtractedText string `json:"extracted_text"`
}

func main() {
	endpoint, err := url.Parse("https://api.txtfetch.com/v1/extract")
	if err != nil {
		panic(err)
	}
	q := endpoint.Query()
	q.Set("url", "https://example.com/report.pdf")
	endpoint.RawQuery = q.Encode()

	req, err := http.NewRequest("POST", endpoint.String(), nil)
	if err != nil {
		panic(err)
	}
	req.Header.Set("Authorization", "Bearer "+os.Getenv("TXTFETCH_KEY"))

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var result extractResponse
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		panic(err)
	}
	fmt.Println(result.ExtractedText)
}

{
  "status": "success",
  "extracted_text": "..."
}

large-or-slow-documents

POST /v1/extract returns 202 with a job_id whenever the document is large or slow to process, or whenever async=true is set. Poll GET /v1/extract/{job_id} for the result, or set webhook_url and have txtfetch push it instead. No single request on this page is safe to assume will always finish synchronously.

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/report.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}

curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.

curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/report.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"

setup-steps

1. Create an S3 event notification for ObjectCreated on the bucket that receives uploads.
2. Point the event at a Lambda function.
3. In the function, generate a short-lived presigned GET URL for the uploaded object.
4. Call POST https://api.txtfetch.com/v1/extract with url set to that presigned URL and an Authorization header of Bearer, followed by your API key.
5. For a large object, pass async set to true and webhook_url set to a second Lambda's function URL, so the upload-triggered function returns right away.
6. Read extracted_text from the response, or from the webhook payload, and write it wherever the pipeline needs it next.

faq

Does txtfetch offer a Lambda layer or an S3 event integration?

No. There is no txtfetch Lambda layer or S3 integration to install. A plain Lambda function, triggered by an S3 object-created event, calls the REST API directly.

Why generate a presigned URL instead of sending the object's bytes?

A presigned GET URL lets txtfetch fetch the object directly from S3. The function never has to stream the file through its own limited memory just to forward it.

How should a Lambda function handle a document too large to process before it times out?

Pass async=true and webhook_url on the request. The upload-triggered function returns immediately, and a second, webhook-triggered Lambda picks up the finished result.

What should the upload-triggered function do if txtfetch returns an error?

Log the error.code and move the failed event to a dead-letter queue. Don't retry blindly, since a bad file or an unsupported format fails the same way on every attempt.

related-reading

• Batch and large-document ingestion →
• OCR scanned documents through one API →
• Async jobs & webhooks →
• Idempotency →
• Get an API key →