> Source: https://txtfetch.com/ingest/pinecone
> Plain-text twin — every page on txtfetch.com has one. https://txtfetch.com/text

---

# Extract text into Pinecone

A managed, serverless vector database with a hard ceiling on per-vector metadata. Here's what changes between the text txtfetch returns and the first row you write to Pinecone.

## The problem

Pinecone is fully managed, so there's no schema to design. But its metadata is capped at 40 KB per vector, and only holds strings, numbers, booleans, and lists of strings. A chunk record that carries its own JSON blob, the way you might store it elsewhere, doesn't fit as-is.

what pinecone needs

| Per-vector metadata limit | 40 KB per vector, filterable fields only. Values must be a string, number, boolean, or list of strings. No nested objects. |
| --- | --- |
| Stores the chunk text itself | Not as a dedicated field. Put the chunk text in metadata, inside that 40 KB, or keep it in your own store, keyed by the vector id. |
| ID format | An ASCII string, 1 to 512 characters. |
| Max vector dimensions | 20,000 dimensions per vector. |
| Batch upsert size | 2 MB per upsert request. Pinecone's own guidance keeps a single call near 1,000 vectors, fewer at higher dimensions. |
| Index type | Serverless indexes only today. Metric is cosine, dotproduct, or euclidean, set once at index creation. |

No accuracy, speed, or recall figure is measured against Pinecone on this page. See [/benchmarks](https://txtfetch.com/benchmarks) for txtfetch's own measured extraction numbers.

the pipeline

Extract, chunk, embed, and upsert. No framework, just the [txtfetch Python SDK](https://txtfetch.com/docs/quickstarts) and Pinecone's own client.

Python

```python
import os

from openai import OpenAI
from pinecone import Pinecone
from txtfetch import Txtfetch

txtfetch = Txtfetch(api_key=os.environ["TXTFETCH_KEY"])
oai = OpenAI()
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("docs")


def chunk_text(text: str, size: int = 1000, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


# 1. Extract
result = txtfetch.extract(file="whitepaper.pdf")

# 2. Chunk
chunks = chunk_text(result.extracted_text)

# 3. Embed
response = oai.embeddings.create(model="text-embedding-3-small", input=chunks)

# 4. Upsert — metadata is capped at 40 KB per vector, so only an excerpt of
# the chunk goes here. Keep the full text in your own store, keyed by this
# same id, if you need it back verbatim.
vectors = [
    {
        "id": f"whitepaper-{i}",
        "values": item.embedding,
        "metadata": {"source": "whitepaper.pdf", "text": chunks[i][:2000]},
    }
    for i, item in enumerate(response.data)
]

index.upsert(vectors=vectors, namespace="docs")
```

Every call above was resolved against pinecone 7.3.0, in September 2026. Client APIs move, so check the vendor's own docs for the version you pin.

## Chunking for Pinecone

Pinecone's 40 KB metadata ceiling is the real constraint on chunk size, not any limit on the vector itself. Store a short excerpt in metadata, not the full chunk, once your chunks run long. Check the actual token counts and boundaries your chunker produces in the chunk previewer before you commit to a size.

large batches

A large file, or a slow OCR pass, routes to an async job automatically. That returns a `202` plus a `job_id` to poll, and `?async=true` forces that path for any request. Pass `webhook_url` instead, and txtfetch calls you back when the text is ready, rather than you polling for it. See [async jobs & webhooks](https://txtfetch.com/docs/async) for the full lifecycle.

Submit (async)

```submit
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf&async=true" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}
```

Poll

```poll
curl "https://api.txtfetch.com/v1/extract/3fa85f64-5717-4562-b3fc-2c963f66afa6" \
  -H "Authorization: Bearer $TXTFETCH_KEY"

# {"status": "processing", "job_id": "..."} while running, then the same
# {"status": "success", "extracted_text": "...", "metadata": {...}} shape
# POST /v1/extract would have returned synchronously.
```

curl

```curl
curl -X POST "https://api.txtfetch.com/v1/extract?url=https://example.com/whitepaper.pdf" \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  --data-urlencode "webhook_url=https://example.com/webhooks/txtfetch"
```

A retried request, from a client timeout or a dropped response, shouldn't extract and upsert the same document twice. Send the same `Idempotency-Key` on a retry and txtfetch returns the first result instead of running the job again. See [idempotency](https://txtfetch.com/docs/idempotency) for the full guarantee.

curl

```curl
curl -X POST https://api.txtfetch.com/v1/extract \
  -H "Authorization: Bearer $TXTFETCH_KEY" \
  -H "Idempotency-Key: 3f29b6e4-9c1a-4b8e-9c2a-1e6f0a2d5b3c" \
  -F file=@contract.pdf
```

JavaScript

```javascript
// The SDK auto-generates and reuses an Idempotency-Key across its own
// retry chain. Pass your own to control it explicitly:
await txtfetch.extract({ file: "./contract.pdf", idempotencyKey: "3f29b6e4-..." });
```

Python

```python
# The SDK auto-generates and reuses an Idempotency-Key across its own
# retry chain. Pass your own to control it explicitly:
client.extract(file="contract.pdf", idempotency_key="3f29b6e4-...")
```

txtfetch ships no connector, plugin, or client for Pinecone. The pipeline above is the whole integration. Extract with txtfetch, then write to Pinecone with Pinecone's own client, the same as you would for any other source of text.

frequently asked questions

**Does txtfetch write directly to Pinecone?**: No. txtfetch returns extracted text over one HTTP call. Chunking, embedding, and the upsert itself stay a caller-side step, exactly like the pipeline above shows.

**Can I store the full chunk text in Pinecone?**: Only up to the 40 KB per-vector metadata ceiling, and metadata only holds strings, numbers, booleans, and lists of strings. A longer chunk, or a chunk record with nested fields, needs a separate store keyed by the same vector id.

**Does txtfetch benchmark accuracy or speed against Pinecone?**: No. Pinecone is a vector database, not a text-extraction service, so there's no accuracy or speed comparison to make. See /benchmarks for txtfetch's own measured extraction numbers.

sources

-   [Pinecone: Python SDK overview](https://docs.pinecone.io/reference/sdks/python/overview) Accessed 2026-09
-   [Pinecone: Upsert reference](https://docs.pinecone.io/reference/upsert) Accessed 2026-09
-   [Pinecone: Indexing overview (metadata limits and types)](https://docs.pinecone.io/guides/index-data/indexing-overview) Accessed 2026-09
-   [Pinecone: Upsert data guide (batch and request-size limits)](https://docs.pinecone.io/guides/index-data/upsert-data) Accessed 2026-09

## Related

-   [RAG & LLM ingestion](https://txtfetch.com/solutions/rag-ingestion), the wider use case this pipeline feeds.
-   [RAG recipe: chunk, embed, index](https://txtfetch.com/docs/recipe), the same pipeline through LangChain or LlamaIndex instead.
-   [Chunk previewer](https://txtfetch.com/tools/chunk-preview), to test your own extracted text before you write the first row.
-   [Async jobs & webhooks](https://txtfetch.com/docs/async), the full lifecycle for a large batch.

other stores

-   [Extract text into pgvector →](https://txtfetch.com/ingest/pgvector)
-   [Extract text into Qdrant →](https://txtfetch.com/ingest/qdrant)
-   [Extract text into Chroma →](https://txtfetch.com/ingest/chroma)
-   [Extract text into Weaviate →](https://txtfetch.com/ingest/weaviate)
-   [All destinations →](https://txtfetch.com/ingest)

## Wire the extraction step in.

The RAG recipe shows the whole path, from a file to the Pinecone upsert.

[Read the RAG recipe →](https://txtfetch.com/docs/recipe)

[Get an API key →](https://app.txtfetch.com/signup)