What to do about agentic web browsers as web developers?
1 comeonreally 2 5/5/2025, 9:52:31 AM
i tried out this agentic web browser and it's really good. sure, it needs a bit of polish but i genuinely think i've seen the future.
as a web developer this got me thinking, "what happens to our interfaces and ads and revenue models?" i wanted to posit this idea of agent access tokens or AATs. basically an agentic browser would offer up a token and be authorized by the target website, which would serve it an agent-optimized version of itself. could be plain text with minimal organization or exposing a bunch of API endpoints that can be hit using the AAT.
the website owner can then use that AAT to bill the agentic browser provider per request, there'd be no standard fee.
another alternative is to confuse and block them altogether.
Some thoughts:
You're describing a new protocol layer, almost like a “robots.txt meets OAuth meets Stripe” for agents. It could allow site owners to expose structured data (APIs or structured text) and monetize access. That’s huge. It flips the current scraping arms race into an opportunity.
Revenue models would indeed be disrupted. A site’s monetization might shift from CPM/CPC ads for human attention to pay-per-query or tiered subscriptions for agent access. Kind of like how OpenAI or Google charges for API usage.
Interfaces become optional. You'd optimize for semantic structure, not aesthetics. That’s both terrifying and liberating as a dev. Imagine a site that’s just: “Here’s what we know, here’s how to query us.”
Adversarial blocking is the fallback strategy (and already happening — just ask GPTBot), but it’s a short-term play. If agents become the new UX layer for many tasks, sites will have to engage somehow — or risk total disintermediation.
One risk I see: centralization. If only a few agent providers get whitelisted, or if the billing standard becomes a walled garden, we might end up with another App Store-like gatekeeper situation.
Curious: How would you see AATs being issued — per-agent, per-session, or tied to identity in some verifiable way?
while adversarial blocking is a possibility it comes with a real financial penalty on the side of the content provider, fewer agents competing for your content does mean fewer agent providers to haggle with, ergo, a high likelihood of a bad deal.
in a perfect world they'd serve as IDs for agents similar to a person's ID card, but spoofing is a potentially massive issue. what stops a content provider capturing an agent's ID and repeatedly hitting their own endpoints?
brighter men than me will have to figure most of this out, but i wanted to put this thought out into the universe