Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures

• URL: http://arxiv.org/abs/2604.00694v1
• Date: Wed, 01 Apr 2026 09:51:46 GMT
• Title: Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures
• Abstract summary: Unbrowse is a shared route graph that transforms browser-based route discovery into a collectively maintained index.<n>The system passively learns routes from real browsing traffic and serves cached routes via direct API calls.<n>A three-path execution model -- local cache, shared graph, or browser fallback -- ensures the system is voluntary and self-correcting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous agents increasingly interact with the web, yet most websites remain designed for human browsers -- a fundamental mismatch that the emerging ``Agentic Web'' must resolve. Agents must repeatedly browse pages, inspect DOMs, and reverse-engineer callable routes -- a process that is slow, brittle, and redundantly repeated across agents. We observe that every modern website already exposes internal APIs (sometimes called \emph{shadow APIs}) behind its user interface -- first-party endpoints that power the site's own functionality. We present Unbrowse, a shared route graph that transforms browser-based route discovery into a collectively maintained index of these callable first-party interfaces. The system passively learns routes from real browsing traffic and serves cached routes via direct API calls. In a single-host live-web benchmark of equivalent information-retrieval tasks across 94 domains, fully warmed cached execution averaged 950\,ms versus 3{,}404\,ms for Playwright browser automation (3.6$\times$ mean speedup, 5.4$\times$ median), with well-cached routes completing in under 100\,ms. A three-path execution model -- local cache, shared graph, or browser fallback -- ensures the system is voluntary and self-correcting. A three-tier micropayment model via the x402 protocol charges per-query search fees for graph lookups (Tier~3), a one-time install fee for discovery documentation (Tier~1), and optional per-execution fees for site owners who opt in (Tier~2). All tiers are grounded in a necessary condition for rational adoption: an agent uses the shared graph only when the total fee is lower than the expected cost of browser rediscovery.

Related papers

• String: An Agentic OS Where Every App Is a Markdown File [33.47077247999211]
  LLM agents have become a new class of software user, but every surface they work through was designed for someone else.<n>We present String, an open-source runtime that gives this user an interface of its own and treats the job as an operating-systems problem.
  arXiv  Detail & Related papers  (2026-08-28T07:44:41Z)
• BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes [54.96895469970198]
  BrowserForge generates web interaction data at scale by driving many browser sandboxes in parallel over the open web.<n> BrowserForge couples three components: an open-web sourcing stage that exposes the agent to hundreds of thousands of real, openly reachable websites.<n>A rule-plus-model cleaning pipeline removes failed runs and rewrites the surviving reasoning into a single unified chain-of-thought style.
  arXiv  Detail & Related papers  (2026-08-25T17:35:42Z)
• WebChallenger: A Reliable and Efficient Generalist Web Agent [3.8083338863305882]
  WebChallenger is a web agent framework that addresses each gap through architecture design rather than model scale.<n>Our system achieves 56.3% on WebArena, 48.7% on VisualWebArena, 51.0% on Online-Mind2Web, and 70.9% on WorkArena.
  arXiv  Detail & Related papers  (2026-06-09T04:53:19Z)
• Skim: Speculative Execution for Fast and Efficient Web Agents [8.47432001892345]
  Skim is a speculative execution framework for web agents.<n>It exploits the predictable structure of purpose-built websites.<n>Skim reduces median per-task cost by 1.9x and latency by 33.4% with no accuracy loss.
  arXiv  Detail & Related papers  (2026-05-15T19:12:43Z)
• MolmoWeb: Open Visual Web Agent and Open Data for the Open Web [60.29597961827816]
  MolmoWebMix is a mixture of browser task demonstrations and web-GUI perception data.<n>MolmoWeb-8B is a family of fully open multimodal web agents.<n>We will release model checkpoints, training data, code, and a unified evaluation harness to enable open research on web agents.
  arXiv  Detail & Related papers  (2026-04-09T17:54:02Z)
• EmbeWebAgent: Embedding Web Agents into Any Customized UI [3.034887612600091]
  We present EmbeWebAgent, a framework for embedding agents directly into existing UIs.<n>It supports mixed-granularity actions ranging from primitives to higher-level composites.<n>Our demo shows minimal retrofitting effort and robust multi-step behaviors grounded in a live UI setting.
  arXiv  Detail & Related papers  (2026-02-16T15:59:56Z)
• GUI-360$^\circ$: A Comprehensive Dataset and Benchmark for Computer-Using Agents [59.107657859025586]
  GUI-360$circ$ is a large-scale, comprehensive dataset and benchmark suite designed to advance computer-using agents (CUAs)<n>The released corpus contains over 1.2M executed action steps across thousands of trajectories in popular Windows office applications.<n>The dataset supports three canonical tasks, GUI grounding, screen parsing, and action prediction, and a hybrid GUI+API action space.
  arXiv  Detail & Related papers  (2025-11-06T12:19:02Z)
• BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks [51.803138848305814]
  We introduce BrowserArena, a live open-web agent evaluation platform that collects user-submitted tasks.<n>We identify three consistent failure modes: captcha resolution, pop-up banner removal, and direct navigation to URLs.<n>Our findings surface both the diversity and brittleness of current web agents.
  arXiv  Detail & Related papers  (2025-10-02T15:22:21Z)
• Infogent: An Agent-Based Framework for Web Information Aggregation [59.67710556177564]
  We introduce Infogent, a novel framework for web information aggregation. Experiments on different information access settings demonstrate Infogent beats an existing SOTA multi-agent search framework by 7%.
  arXiv  Detail & Related papers  (2024-10-24T18:01:28Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-Based Agents outperform web Browsing Agents in experiments on WebArena.<n>Hybrid Agents out-perform both others nearly uniformly across tasks.<n>Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.