> Source: https://txtfetch.com/sources/web-page
> Plain-text twin — every page on txtfetch.com has one. https://txtfetch.com/text

---

# Extract text from a public web page

Already public, so there is no credential to mint and no header to send. Here's how a document actually gets from a public web page to a txtfetch response.

## The problem

A public web page needs no login and no token. Point ?url= at it, and txtfetch fetches the page on the server, on your behalf. A server-side fetch is not the same as a click in a browser, though. It can reach addresses a browser tab never sees, so txtfetch checks and blocks every hop before it connects.

?url=, and why

| Pre-authenticated URL? | Yes — nothing to authenticate, the page is already public |
| --- | --- |
| txtfetch path | ?url= |
| Link lifetime | As long as the page stays up |
| The trap | txtfetch reads what the server sends, not what a browser renders. |

A public page needs no authentication at all. ?url= reaches it with a plain GET, the same request a browser would make.

## How it works

txtfetch fetches the page over http or https, on port 80 or 443. It rejects a URL that carries a username or password, and it re-checks the address behind every redirect. txtfetch blocks a private, loopback, or link-local address. It checks the DNS answer again at connect time, so a rebind cannot slip through. A reply that is not a 2xx status comes back as a typed fetch\_failed error. The whole fetch runs under a fixed time and size budget. What comes back is the text the server sent, chrome included. txtfetch draws no line between an article and its navigation or footer.

the script

One call to txtfetch, with the page's URL in the ?url= parameter. There is nothing to fetch first, and no vendor SDK on either side.

Python

```python
import os

import requests

page_url = "https://example.com/blog/2024-annual-report"

r = requests.post(
    "https://api.txtfetch.com/v1/extract",
    headers={"Authorization": f"Bearer {os.environ['TXTFETCH_KEY']}"},
    params={"url": page_url},
)

if r.status_code != 200 or r.json().get("status") == "fetch_failed":
    # A bot block, a cookie wall, or a dead link all land here.
    raise RuntimeError(f"fetch_failed: {r.text}")

print(r.json()["extracted_text"])
```

JavaScript

```javascript
const pageUrl = "https://example.com/blog/2024-annual-report";

const endpoint = new URL("https://api.txtfetch.com/v1/extract");
endpoint.searchParams.set("url", pageUrl);

const res = await fetch(endpoint, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.TXTFETCH_KEY}` },
});

const body = await res.json();
if (!res.ok || body.status === "fetch_failed") {
  // A bot block, a cookie wall, or a dead link all land here.
  throw new Error(`fetch_failed: ${JSON.stringify(body)}`);
}

console.log(body.extracted_text);
```

txtfetch ships no connector, plugin, or client for The open web. It needs none. A public page needs no credential, so the URL is the whole request.

frequently asked questions

**Does txtfetch run a browser to fetch the page?**: No. txtfetch runs no headless browser and no JavaScript. It fetches the raw HTML a server sends and parses that. A page whose text loads only after a script runs returns just the shell the server sent.

**Does txtfetch check robots.txt before fetching a page?**: No. txtfetch checks neither robots.txt nor a site's terms of service. Following those rules for any page you fetch is the caller's responsibility.

**What happens if the URL points at a PDF instead of a web page?**: txtfetch reads it as a PDF, not as HTML. The Content-Type and Content-Disposition headers name the file, and txtfetch checks the bytes too, so a mislabeled file still parses correctly.

**Why did my fetch fail against a page that loads fine in my browser?**: A bot block, a cookie wall, and a login wall all return a non-2xx status, and txtfetch turns that into a typed fetch_failed error. A private, loopback, or link-local address fails the same way, on purpose.

sources

-   [RFC 9110: HTTP Semantics — redirection status codes](https://www.rfc-editor.org/rfc/rfc9110.html#name-redirection-3xx) Accessed 2026-09
-   [OWASP: Server-Side Request Forgery Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Server_Side_Request_Forgery_Prevention_Cheat_Sheet.html) Accessed 2026-09 directional

## Related

-   [Extract clean text from HTML and web pages](https://txtfetch.com/extract/html), how txtfetch parses the HTML this fetch returns.
-   [Free HTML to text converter](https://txtfetch.com/tools/html-to-text), paste a page's markup and see the same parser's output, with nothing uploaded.
-   [Plans and limits](https://txtfetch.com/pricing), the file-size ceiling each plan allows for a fetched page.
-   [Integrations](https://txtfetch.com/integrations), for wiring this into a no-code automation platform instead of a script.
-   [Ingest into a search or vector index](https://txtfetch.com/ingest), for where the extracted text goes next.
-   [Async jobs & webhooks](https://txtfetch.com/docs/async), for a source that's large or slow to fetch.
-   [Error reference](https://txtfetch.com/docs/errors), for what a failed `?url=` fetch returns.

other sources

-   [Extract text from Google Drive →](https://txtfetch.com/sources/google-drive)
-   [Extract text from SharePoint & OneDrive →](https://txtfetch.com/sources/sharepoint-onedrive)
-   [Extract text from Amazon S3 →](https://txtfetch.com/sources/amazon-s3)
-   [Extract text from Dropbox →](https://txtfetch.com/sources/dropbox)
-   [Extract text from Confluence →](https://txtfetch.com/sources/confluence)
-   [Extract text from Email inbox →](https://txtfetch.com/sources/email-inbox)
-   [All sources →](https://txtfetch.com/sources)

## Point it at a URL.

Pass the page's address and the text comes back. There is no credential to mint and no connector to install.

[Get an API key →](https://app.txtfetch.com/signup)

[See the URL docs →](https://txtfetch.com/docs)