Does txtfetch connect directly to pgvector, Pinecone, Qdrant, Chroma, Weaviate, Elasticsearch, OpenSearch, Meilisearch, Typesense, or Postgres full-text search?

No. txtfetch ships no connector, plugin, or client for any of the ten. It returns extracted text over one HTTP call. You split the text, embed it if you need vectors, and write it with each destination's own client.

Which destination should I pick?

Start with what you build and what you already run. A RAG or LLM pipeline needs a vector store. Already on Postgres? Use pgvector. For a managed service, use Pinecone or Weaviate Cloud. To self-host open source, use Qdrant or Chroma. A search index needs a keyword engine instead. Pick Elasticsearch, OpenSearch, Meilisearch, or Typesense. Or use Postgres full-text search and run no new service. Each spoke lists its own limits.

Elasticsearch already runs Tika. Why use txtfetch?

It does, through the attachment processor. That processor runs on your own cluster nodes. It stops at 100,000 characters by default, and its docs name no OCR step. txtfetch runs Tika and Tesseract outside your cluster, behind one HTTP call. You configure no ingest pipeline.

Why does this page focus on constraints instead of features?

You split the text before you write it. A metadata limit, an ID rule, a batch limit, or a silent cut-off changes how you split. It also changes what you can store. A feature list tells you none of that.

Does txtfetch benchmark accuracy, speed, or recall for any of these destinations?

No. None of these ten is a text-extraction service, so there is no such comparison to make. See /benchmarks for txtfetch's own extraction numbers.

txtfetch gets you the text. Here's the rest of the pipeline.

Extract, split, embed if you need vectors, then write. A vector store and a keyword search engine each need something different. Here is what each one needs, with a pipeline you can run.

The four steps, one pipeline

A document goes in, and a row in a vector store comes out. txtfetch owns the first step. You own the rest, and each store below changes what "the rest" looks like.

Extract. One HTTP call turns the file or URL into plain text.
Chunk. Split the text into pieces sized for retrieval, not for a limit.
Embed. Turn each chunk into a vector with the embedding model you already run.
Upsert. Write the vector, its metadata, and often the chunk text, to the store.

A search index skips the embed step. You extract the text and split it into sections. Then you write each section to the index's bulk API. You need no embedding model.

vector stores, compared

pgvector, Pinecone, Qdrant, Chroma, and Weaviate, compared on what the write step needs
Constraint	pgvector	Pinecone	Qdrant	Chroma	Weaviate
Per-vector metadata limit	None from pgvector itself. You're bound by ordinary Postgres limits: up to 1 GB per field, with TOAST moving large text out of the row automatically.	40 KB per vector, filterable fields only. Values must be a string, number, boolean, or list of strings. No nested objects.	No fixed cap on a point's payload. Bound by the request-size ceiling instead: 32 MB per REST call by default, or a much smaller 4 MB default on the gRPC client unless you raise it.	No published byte ceiling on a document or metadata value.	No published per-property byte ceiling. Bound by the overall request size, mostly relevant over gRPC.
Stores the chunk text itself	Yes. The chunk text is just another column in the same table as the vector.	Not as a dedicated field. Put the chunk text in metadata, inside that 40 KB, or keep it in your own store, keyed by the vector id.	Yes. The payload is arbitrary JSON stored with the point, so the chunk text is a normal payload field.	Yes, as a dedicated documents field, alongside embeddings and metadatas.	Yes. The chunk text is an ordinary schema property, for example content, on the same object as the vector.
ID format	None. Use whatever primary key type the table declares: bigserial, uuid, or text.	An ASCII string, 1 to 512 characters.	An unsigned 64-bit integer or a UUID string. No other string is accepted.	Any unique string within the collection.	A UUID. Weaviate can generate one deterministically from your own fields with generate_uuid5(...) if you don't supply one.
Max vector dimensions	16,000 for a plain vector column. Building an HNSW or IVFFlat index on it caps out at 2,000 dimensions, or 4,000 for a halfvec column.	20,000 dimensions per vector.	65,535 dimensions for a dense vector.	No published ceiling. The first vector you insert fixes the collection's dimension, and every later insert must match it.	65,535 dimensions, stored as a uint16 index. Memory runs out long before that ceiling matters.
Batch upsert size	None from pgvector. You're bound by ordinary Postgres statement size. Use COPY for a bulk load.	2 MB per upsert request. Pinecone's own guidance keeps a single call near 1,000 vectors, fewer at higher dimensions.	No hard point-count cap. Qdrant's own guidance suggests 64 to 256 points per batch, with a few parallel upload threads for a big job.	Governed by client.get_max_batch_size(), a SQLite parameter ceiling that varies by build (roughly 5,000 to 44,000 records). Chroma won't split an oversized call for you.	No fixed cap. collection.data.insert_many(...) batches server-side, so you don't tune a batch size by hand.
Index type	Exact scan with no index, IVFFlat, or HNSW. HNSW is the common default today.	Serverless indexes only today. Metric is cosine, dotproduct, or euclidean, set once at index creation.	HNSW, with optional scalar or binary quantization to cut memory use.	HNSW. Distance metric (l2, cosine, or ip) is set once at collection creation and can't change after.	HNSW by default, with flat, dynamic, and hfresh alternatives, plus PQ, BQ, SQ, and RQ quantization.

Full citations, dated, live on each spoke below. No row states an accuracy, speed, or recall figure. See /benchmarks for txtfetch's own numbers.

pick a vector store

pgvector

A Postgres extension that adds a vector column type to a database you already run.

Exact scan with no index, IVFFlat, or HNSW. HNSW is the common default today.

Pinecone

A managed, serverless vector database with a hard ceiling on per-vector metadata.

Serverless indexes only today. Metric is cosine, dotproduct, or euclidean, set once at index creation.

Qdrant

An open-source and managed vector database with a strict, easy-to-miss ID format.

HNSW, with optional scalar or binary quantization to cut memory use.

Chroma

An open-source embedding database with a first-class field for the chunk text itself.

HNSW. Distance metric (l2, cosine, or ip) is set once at collection creation and can't change after.

Weaviate

An open-source and managed vector database whose client API renamed its own core method this year.

HNSW by default, with flat, dynamic, and hfresh alternatives, plus PQ, BQ, SQ, and RQ quantization.

Keyword search engines

Elasticsearch and OpenSearch can extract text themselves. Each has an attachment step that runs Apache Tika. That step runs inside your cluster and stops at 100,000 characters by default. Meilisearch, Typesense, and Postgres full-text search extract nothing. Each one also has a point where a long document stops counting. The table names it.

Elasticsearch, OpenSearch, Meilisearch, Typesense, and Postgres full-text search, compared on what the write step needs
Constraint	Elasticsearch	OpenSearch	Meilisearch	Typesense	Postgres full-text search
Where long text stops counting	100,000 characters by default, set by the attachment processor's indexed_chars option. The processor drops the rest of the text. Set indexed_chars to -1 to remove the cap. Or skip the processor and index txtfetch's text as a plain field.	100,000 characters by default, set by the plugin's indexed_chars option. The plugin drops the rest of the text. Set indexed_chars to -1 to remove the cap. Or skip the plugin and index txtfetch's text as a plain field.	65,535 word positions per attribute. Meilisearch ignores every later word in that field, with no error and no warning.	No per-document cap is documented. The documented limit is 10 GB per bulk import request. Typesense rejects a larger request. It does not truncate it.	Word positions stop at 16,383. Postgres silently sets every later position to 16,383. It also keeps at most 256 positions per lexeme and drops the rest. Only a tsvector over 1 MB raises an error.
Max request size	100 MB per HTTP request by default (http.max_content_length).	100 MB per HTTP request by default (http.max_content_length). OpenSearch kept the setting name from its fork.	100 MB per request by default. You can change it on a self-hosted instance.	10 GB per bulk import request (JSONL over HTTP).	None from full-text search. Normal Postgres statement and row limits apply. Use COPY or executemany for a bulk load.
Built-in document extraction	Yes. The attachment processor runs Apache Tika inside the cluster. Its docs name no OCR step.	Yes, through the ingest-attachment plugin, a separate install that runs Apache Tika. Its docs name no OCR step.	None. Meilisearch indexes the JSON you send. It does not extract text from files.	None. Typesense indexes the JSON documents you send. It does not extract text from files.	None. to_tsvector() parses text you already have. It does not extract text from files.
Keeps the full text	Yes, in _source by default. If you use the attachment processor, set remove_binary: true. Otherwise the base64 file stays in the index next to its text.	Yes, in _source by default, the same as any other field.	Yes. Meilisearch stores every field you send and can return it as-is.	Yes. Typesense stores every field. Set index: false on a field to keep it on disk and out of memory.	Yes. The source text is a normal column in the same row as its generated tsvector.
Document ID format	Any UTF-8 string up to 512 bytes. Elasticsearch generates one automatically if you omit it.	Any UTF-8 string up to 512 bytes. OpenSearch generates one automatically if you omit it.	A string or an integer. Use only letters, digits, hyphens, and underscores. The docs name no length limit.	A string. Typesense makes one for you if a document has no id field.	None from full-text search. Use the primary key type your table declares, such as bigserial, uuid, or text.
Bulk write API	The Bulk API (_bulk), one JSON action line per document.	The Bulk API (_bulk), one JSON action line per document.	add_documents(...). It returns a task that you can poll until the write finishes.	documents.import_(...). It takes JSONL and reports success or an error for each line.	No bulk endpoint. Use executemany(...) for a moderate batch, or COPY for a large one.

Full citations, dated, live on each spoke below. No row states an accuracy, speed, or recall figure. See /benchmarks for txtfetch's own numbers.

pick a search engine

Elasticsearch

A search engine with a built-in Tika step that stops reading at 100,000 characters by default.

Yes. The attachment processor runs Apache Tika inside the cluster. Its docs name no OCR step.

OpenSearch

A fork of Elasticsearch whose Tika plugin installs separately and stops at 100,000 characters by default.

Yes, through the ingest-attachment plugin, a separate install that runs Apache Tika. Its docs name no OCR step.

Meilisearch

A typo-tolerant search engine that ignores every word past position 65,535 in a field, with no warning.

None. Meilisearch indexes the JSON you send. It does not extract text from files.

Typesense

An open-source search engine that keeps every indexed field in RAM and publishes no per-document size cap.

None. Typesense indexes the JSON documents you send. It does not extract text from files.

Postgres full-text search

Built into Postgres, and it silently clamps every word position past 16,383.

None. to_tsvector() parses text you already have. It does not extract text from files.

No connector ships for any of them

txtfetch ships no plugin or client for pgvector, Pinecone, Qdrant, Chroma, or Weaviate. The same is true of Elasticsearch, OpenSearch, Meilisearch, Typesense, and Postgres full-text search. Each spoke below shows the whole integration. Extract with txtfetch. Then write to the destination with its own client.

More on this pipeline

RAG & LLM ingestion — the wider use case the vector-store pipeline feeds.
Search indexing — the wider use case the keyword-engine pipeline feeds.
RAG recipe: chunk, embed, index — the same vector-store pipeline through LangChain or LlamaIndex instead of plain Python.
Re-index only what changed — your documents change daily; this is how to pay for a changed file, not your whole corpus.
Chunk previewer — test your own extracted text before you write the first row.

frequently asked questions

Does txtfetch connect directly to pgvector, Pinecone, Qdrant, Chroma, Weaviate, Elasticsearch, OpenSearch, Meilisearch, Typesense, or Postgres full-text search?: No. txtfetch ships no connector, plugin, or client for any of the ten. It returns extracted text over one HTTP call. You split the text, embed it if you need vectors, and write it with each destination's own client.
Which destination should I pick?: Start with what you build and what you already run. A RAG or LLM pipeline needs a vector store. Already on Postgres? Use pgvector. For a managed service, use Pinecone or Weaviate Cloud. To self-host open source, use Qdrant or Chroma. A search index needs a keyword engine instead. Pick Elasticsearch, OpenSearch, Meilisearch, or Typesense. Or use Postgres full-text search and run no new service. Each spoke lists its own limits.
Elasticsearch already runs Tika. Why use txtfetch?: It does, through the attachment processor. That processor runs on your own cluster nodes. It stops at 100,000 characters by default, and its docs name no OCR step. txtfetch runs Tika and Tesseract outside your cluster, behind one HTTP call. You configure no ingest pipeline.
Why does this page focus on constraints instead of features?: You split the text before you write it. A metadata limit, an ID rule, a batch limit, or a silent cut-off changes how you split. It also changes what you can store. A feature list tells you none of that.
Does txtfetch benchmark accuracy, speed, or recall for any of these destinations?: No. None of these ten is a text-extraction service, so there is no such comparison to make. See /benchmarks for txtfetch's own extraction numbers.

Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the vector store upsert.

Read the RAG recipe →

Get an API key →