The advice goes: rotate your IP on every request. For a scraper, that's right. A scraper pulls a thousand independent pages; there's no relationship between page 1 and page 900, so a fresh exit each time spreads load and dodges per-IP rate limits. Nothing is lost, because there was nothing to preserve.

An agent isn't doing that. An agent runs a flow: land on a listing, snapshot the page, click through to page two, read, paginate, extract, compare. Those steps are related. They're supposed to look like one continuous person.

Now rotate the IP between step two and step three. From the site's side, someone arrived from Chicago, and three seconds later the same session is acting from Frankfurt. That is not a bot signature — it's a hijacked-account signature, and it's treated more harshly than bot traffic. Best case you get bounced back to a re-auth. Worst case the flow just falls apart, and your agent spends its turns confused about why the page it expected isn't there.

So the requirement for agents inverts: one stable identity for the length of a task, then a fresh one for the next task.

This post is about how I wired that into an MCP server so the agent itself controls it — aethyn-browser-mcp, open source, MIT.

Sticky identity as a tool argument

The whole mechanism lives in one argument on one tool.

{
  "tool": "aethyn_launch_browser",
  "args": {
    "country": "gb",
    "session": "catalog_walk_01",
    "lifetime_min": 20,
    "tier": "premium"
  }
}

session is a task id you choose (letters, digits, underscores). Reuse the same value and you keep the same exit IP. Change it and you get a fresh identity. That's it — that's the sticky mechanism, and it's deliberately something the agent can reason about rather than a config file it can't see.

lifetime_min is how long the exit stays pinned: 1 to 1440 minutes, default 10. Size it to your task, not to your patience — a 20-minute catalog walk gets 20, not 1440.

Two naming things that will save you a debugging session:

• The token is lifetime, not ttl. If you've read a proxy doc that says ttl, that's a different product's grammar.

• session is what you pass in; session_id is what launch returns, and it's what every other tool takes. Same value, different roles. Keep them straight in your prompts.

A worked example: walking a public catalog

Let's do something a scraper-style rotation would wreck: paginate through a public listing, holding one identity the whole way.

Launch the task. One session id for the entire walk:

{ "tool": "aethyn_launch_browser",
  "args": { "country": "gb", "session": "catalog_walk_01", "lifetime_min": 20 } }

Verify where you're standing. Before trusting a single localized number on the page:

{ "tool": "aethyn_check_exit_ip", "args": { "session_id": "..." } }
→ { "ip": "…", "country": "GB", "city": "…", "org": "…", "is_residential": true }

This runs through the session's own proxy, so it reports where the page will actually be served to — not where your machine is. (is_residential is a best-effort read of the ASN/org, so treat it as a signal rather than a promise.)

Navigate, then snapshot instead of guessing selectors. This is the part I like most about the MCP ergonomics:

{ "tool": "aethyn_navigate", "args": { "session_id": "...", "url": "https://example-catalog.com/listings" } }
{ "tool": "aethyn_snapshot",  "args": { "session_id": "..." } }

aethyn_snapshot returns the accessibility tree with [ref=eNN] handles. The model reads roles and names — "link: Next page [ref=e42]" — and clicks the ref:

{ "tool": "aethyn_click", "args": { "session_id": "...", "ref": "e42" } }

No brittle CSS selectors invented by a language model that has never seen the DOM. It reads what's actually on the page, then acts on a handle. (aethyn_click and aethyn_type both accept a selector too, if you already know the page.)

Read, paginate, repeat — with aethyn_get_content (format: "markdown" keeps the context window clean) — all on the same session_id, which means the same exit IP for the whole walk. To the site, that's one visitor reading a catalog, which is exactly what it is.

Close when the task ends.

{ "tool": "aethyn_close", "args": { "session_id": "..." } }

The next task gets its own session — a fresh identity, no cross-contamination, no shared cookie jar.

Rotating deliberately, not reflexively

Sometimes a page comes back wrong. Not an error — wrong. Stripped content, a challenge interstitial, a suspiciously empty body behind a perfectly healthy 200.

The agent's move here is aethyn_new_identity:

{ "tool": "aethyn_new_identity", "args": { "session_id": "..." } }

Fresh exit IP in the same country, cookies cleared, session continues.

Now the design decision I want to defend, because it's the kind of thing people ask about:

The tool does not auto-detect blocks. It doesn't sniff for challenge markers and silently rotate behind your back. The agent reads the content with aethyn_get_content, judges that it's been soft-blocked or served something bogus, and chooses to rotate.

That's more work. It's also the right call. An agent that silently retries under the hood is an agent whose failure modes you cannot see in the trace. If the plumbing quietly swaps identities every time a page looks odd, you lose the one signal telling you the site doesn't want this traffic — and you learn nothing until you're rate-limited across the board. Make the judgement a step the model takes, in the open, where you can read it back.

The corollary: validate on content, not status. A 200 is not success. Assert that the page contains something only the real page would have before you act on it.

Two configuration traps

The 407 that isn't a credentials problem. AETHYN_DEFAULT_TIER defaults to premium → HTTP port 2099. Elite is port 5499. If your account is Elite and you leave the default in place, the browser launches perfectly, then the proxy rejects auth with a 407. It reads like a bad password; it's a port mismatch. Set AETHYN_DEFAULT_TIER=elite or pass tier: "elite" on the call.

Related: city and state targeting are Elite-only (5499). Country targeting works on both tiers.

HTTP only, by design. Chromium can't authenticate SOCKS5, so the server doesn't pretend to offer it.

It isn't locked to one provider

Defaults to Aethyn, but any HTTP proxy works — describe how your provider encodes targeting in the username and it adapts:

PROXY_HOST=...
PROXY_PORT=...
PROXY_USERNAME=...
PROXY_PASSWORD=...
PROXY_USERNAME_TEMPLATE="{username}-country-{country}[-city-{city}]-session-{session}-lifetime-{lifetime}"

[optional] segments drop out when they're empty. Fixed proxy with no geo targeting? The template is just {username}.

Scope and limits

Stated plainly, because this space overclaims constantly:

• Public data only — respect robots.txt, rate limits, and site terms.

• No CAPTCHA-solving tool exists here, and none is planned. A challenge is a signal to back off, not a puzzle to beat.

• No login or credential-wall automation.

• A residential exit fixes your network vantage point. It doesn't launder bad pacing, and it isn't an anti-bot bypass. Sticky identity is one layer — the foundational one, not the whole stack.

• The browser runs locally on your machine. Nothing is hosted for you.

Try it

npx -y aethyn-browser-mcp

Drop it into your MCP client config, and your agent can hold one identity per task on its next tool call.

Docs & runnable examples · The full writeup · GitHub · npm · Glama

Built by Aethyn — residential proxies for engineers and agents. Free account, no card.

You built a scraper. It runs beautifully on your laptop. You deploy it to AWS / GCP / a VPS, run the identical code, and suddenly everything is 403s, CAPTCHAs, or empty pages. You didn't change a line. This is the single most common scraping surprise there is — and it's almost never your code.

The cause: your laptop has been lying to you

At home, you exit through a residential IP — an address your ISP (Comcast, Jio, BT…) assigned to a real household. Anti-bot systems trust it by default; it looks like a person. In production, you exit through a datacenter IP owned by a hosting ASN (Amazon AWS, Google Cloud, DigitalOcean). Those ranges are public and known, and anti-bot vendors discount or block them on sight — because roughly nobody browses a shopping site from an AWS IP.

Same code. Different network reputation. That's the whole trick.

Prove it to yourself — check the ASN you're actually exiting from, on each machine:

import requests
info = requests.get("https://ipinfo.io/json", timeout=10).json()
print(info["ip"], info.get("org"))
# laptop -> "AS7922 Comcast Cable"        (residential, trusted)
# prod   -> "AS16509 Amazon.com, Inc."    (datacenter, flagged)

If prod prints a hosting provider's ASN, you've found your bug — and it isn't in your parser.

Four things that change between laptop and prod (in order of impact)

• IP type — the big one. Residential → datacenter. Fix: route production egress through residential proxies so requests read as real users again.

• Velocity. On your laptop you fired one request while debugging. In prod you fire hundreds per second from one IP → rate limits and bans. Fix: pace and rotate.

• Geography. Your laptop sits in the target's country; your prod region might be us-east-1 when you actually need German pricing or a UK SERP. You get different — or blocked — content. Fix: geo-targeted exits matched to the data you want.

• Retries. A failed request retried immediately from the same IP just digs the hole deeper. Fix: back off, then rotate.

The fix, as architecture

• Residential egress in production — per-request rotation for independent pages, a sticky session when a flow needs one identity.

• Pace with jittered backoff, not a tight loop.

• Geo-match the exit to the content you want.

• Validate on content, not status — a 200 can be a soft-block (a decoy or stripped page).

A minimal production-ready pattern that folds all four in:

import requests, time, random

# residential + geo-matched + rotates by default
PROXY = "http://USER-country-us:PASS@proxy.aethyn.io:2099"
proxies = {"http": PROXY, "https": PROXY}
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/150.0 Safari/537.36",
    "Accept-Language": "en-US,en;q=0.9",
}

def fetch(url, positive_token, tries=4):
    for i in range(tries):
        r = requests.get(url, proxies=proxies, headers=HEADERS, timeout=30)
        # content check — NOT just r.status_code == 200
        if r.status_code == 200 and positive_token.lower() in r.text.lower():
            return r.text
        time.sleep(min(30, 2 ** i) + random.random())  # back off; a fresh IP rotates in
    raise RuntimeError(f"blocked after {tries} tries: {url}")

html = fetch("https://example.com/product/123", positive_token="add to cart")

The honest caveat

A residential IP fixes reputation, not everything. If you're still blocked after switching, the next layers are your TLS/JA3 fingerprint (plain requests doesn't negotiate TLS like Chrome — try curl_cffi with impersonate="chrome", or a headless browser) and your pacing. Proxies are one layer of the stack, not a magic bypass — anyone selling them as a complete anti-bot solution is overselling. (More on preventing IP blocks here.)

TL;DR

It works on your laptop because your laptop has a residential IP. It dies in prod because your server has a datacenter IP that every anti-bot system recognizes on sight. Route production through geo-matched residential egress, pace and rotate, and validate on content — and your "it worked on my machine" scraper works in production too.

I work on Aethyn — residential proxies built for engineers: one unified endpoint, country/city/session targeting in the username, transparent per-GB pricing, and a free trial with no card. It's the "residential egress in production" layer from this post. Point a free trial at your prod setup →