https://txtfetch.com/compare/self-hosted-tika/
Apache Tika is free. Why not run it yourself?
It is a fair question, and worth an honest answer. This page shows the build we run and the work behind it, so you can judge for yourself.
Tika is free. The parsers are not the hard part.
Apache Tika is Apache-2.0 licensed. Anyone can download it and run it for free, including the exact parser set txtfetch uses. See the full parser list for the build we run, and how it fits into our pipeline.
Parsing is not where the work is. Tika and Tesseract are mature, well-documented projects. The work is packaging them for a size- and time-limited serverless environment. It also means picking the small limits a production service needs. The sections below show that work, sourced from our own build recipe.
What you still have to build
This is the checklist we worked through to run Tika and Tesseract on AWS Lambda. Every entry below traces to a real constraint in our build.
- Trim the JDK with jlink.
- A full Corretto 21 JDK is about 350 MB unzipped. AWS Lambda caps a function's combined layers at 250 MB. jlink strips the JDK to about 100 MB, keeping every module, so no Tika parser hits a missing one.
- Compile Tesseract and Leptonica from source.
- The compiled binary must link against the exact glibc of the runtime it will execute on. We build both on Amazon Linux 2023 arm64 to match the Lambda execution environment, then copy every shared library ldd reports.
- Choose a trained-data set.
- tessdata_fast ships int8 models at about 4 MB each. tessdata_best ships float models at about 15 MB each. We chose fast, to keep cold starts short.
- Pin and verify every download.
- We check the Corretto tarball and the Tika jar against a pinned SHA-256 sum before use. This stops a swapped file at the vendor from reaching production unnoticed.
- Split the sync and async request paths.
- API Gateway caps a synchronous integration at 30 seconds. A request that could run longer needs its own job queue, a separate worker function, and either webhook delivery or a polling endpoint.
- Set your own limits.
- Memory, timeouts, input-size caps, and OCR language choice are judgment calls. Every one already has a number in this build. You would set each one again for your own traffic.
The versions we pin
These are the exact component versions our build downloads, checksums, and ships. A self-hosted build starts from the same public releases.
| Component | Version | Source |
|---|---|---|
| Java runtime | Amazon Corretto 21.0.11.10.1 | versions.env: CORRETTO_VERSION |
| Tika server | tika-server-standard 3.3.1 | versions.env: TIKA_VERSION |
| Leptonica | 1.87.0 | versions.env: LEPTONICA_VERSION |
| Tesseract | 5.5.2 | versions.env: TESSERACT_VERSION |
| Trained-data set | tessdata_fast | versions.env: TESSDATA_REPO |
| OCR languages shipped | eng, osd | versions.env: TESSDATA_LANGS |
Limits you must pick yourself
A synchronous API request through API Gateway has a hard 30-second ceiling. That is why a request above 8 MB in our system runs on an async worker path instead of blocking on the response. Every other number below is a limit we picked and now enforce. A self-hosted setup needs its own version of each one.
| Limit | Value | Source |
|---|---|---|
| Worker memory | 2,048 MB | variables.tf: worker Lambda memory_size |
| Worker timeout | 900 s | variables.tf: worker_timeout_seconds |
| Internal extraction deadline | 840,000 ms | variables.tf: worker_budget_ms |
| Max input size | 100 MB | variables.tf: worker_max_bytes |
| Sync-to-async threshold | 8 MB | variables.tf: async_threshold_bytes |
| API Gateway integration cap | 30,000 ms | apigateway.tf: timeout_milliseconds |
Where self-hosting wins
Self-hosting is the right call in real situations. We would rather name them than pretend they do not exist. txtfetch has no self-hosted or VPC-deployed artifact today. That is on the roadmap, not shipped. See /compliance for the current state.
- Your documents must never leave your own network, and you cannot wait for a VPC-deployed option to ship.
- You already run infrastructure like this, with someone to own patching, checksum verification, and scaling.
- Your volume is high enough that owning the infrastructure costs less than a per-document API bill. That is true even after you count the engineering time to build and run it.
- You need to change Tika itself: a custom parser, a different OCR engine, or trained data for a language we do not ship.
What the API adds on top of Tika
Once Tika and Tesseract are running, an extraction engine is not yet a product. txtfetch adds the parts around the engine. For large or slow documents, it adds async jobs and webhook delivery. Idempotency keys stop a retried request from double-processing a document. A typed error reference replaces a stack trace. An optional premium quality tier handles the documents Tika alone struggles with.
Frequently asked questions
- Is Apache Tika really free?
- Yes. Tika is Apache-2.0 licensed. txtfetch pays nothing to run it, and neither would you.
- What is the hardest part of self-hosting Tika?
- Not the parsers. The hard part is the operational work: trimming the JDK to fit a size limit, and compiling Tesseract for your runtime's glibc. You also pick your own memory, timeout, and input-size limits.
- Does txtfetch offer a self-hosted or VPC-deployed version?
- No, not today. It is on the roadmap, not shipped. See /compliance for the current state.
- When does self-hosting make more sense than an API?
- When your documents cannot leave your network, or your volume is high enough to justify owning the infrastructure yourself.
- Can I run Tika alone and skip Tesseract?
- Yes, if none of your documents are scans or photos. Tika alone handles PDFs, Office files, HTML, and email. Add Tesseract only when you need OCR.
Check the numbers yourself.
The benchmark runs against a committed corpus. You can re-run it.
See the benchmarks →