WebTrap Park: An Automated Platform for Systematic Security Evaluation of Web Agents

• URL: http://arxiv.org/abs/2601.08406v1
• Date: Tue, 13 Jan 2026 10:21:28 GMT
• Title: WebTrap Park: An Automated Platform for Systematic Security Evaluation of Web Agents
• Abstract summary: We present WebTrap Park, an automated platform for security evaluation of Web Agents.<n>WebTrap Park instantiates three major sources of security risk into 1,226 executable evaluation tasks.
• Score: 20.440047540224864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Web Agents are increasingly deployed to perform complex tasks in real web environments, yet their security evaluation remains fragmented and difficult to standardize. We present WebTrap Park, an automated platform for systematic security evaluation of Web Agents through direct observation of their concrete interactions with live web pages. WebTrap Park instantiates three major sources of security risk into 1,226 executable evaluation tasks and enables action based assessment without requiring agent modification. Our results reveal clear security differences across agent frameworks, highlighting the importance of agent architecture beyond the underlying model. WebTrap Park is publicly accessible at https://security.fudan.edu.cn/webagent and provides a scalable foundation for reproducible Web Agent security evaluation.

Related papers

• DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
  DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
  arXiv  Detail & Related papers  (2026-05-06T11:59:48Z)
• WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents [117.65855863464863]
  Web agents are highly vulnerable to prompt injection attacks.<n>Existing defenses, including system prompt defenses and direct fine-tuning of agents, have shown limited effectiveness.<n>We introduce WebAgentGuard, a reasoning-driven, multimodal guard model for prompt injection detection.
  arXiv  Detail & Related papers  (2026-04-14T04:50:35Z)
• WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks [10.738768990548428]
  We introduce WebSP-Eval, an evaluation framework for measuring web agent performance on website security and privacy tasks.<n>WebSP-Eval comprises 1) a manually crafted task dataset of 200 task instances across 28 websites; 2) a robust agentic system supporting account and initial state management across runs using a custom Google Chrome extension; and 3) an automated evaluator.<n>We evaluate a total of 8 web agent instantiations using state-of-the-art multimodal large language models, conducting a fine-grained analysis across websites, task categories, and UI elements.
  arXiv  Detail & Related papers  (2026-04-07T18:43:21Z)
• It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents [52.81924177620322]
  Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking.<n>Their reliance on dynamic web content makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task.<n>We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks.
  arXiv  Detail & Related papers  (2025-12-29T01:09:10Z)
• SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents [28.60912756104713]
  Large vision-language model (LVLM)-based web agents are emerging as powerful tools for automating complex online tasks.<n>Existing benchmarks provide only partial coverage, typically restricted to narrow scenarios.<n>We present tool, the first holistic benchmark for evaluating the security of LVLM-based web agents.
  arXiv  Detail & Related papers  (2025-10-11T07:18:12Z)
• Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks? [58.48689960350828]
  We show that a simple, modular and model-agnostic defense operating at the agent--tool interface achieves perfect security with high utility.<n>We employ a defense based on two firewalls: a Tool-Input Firewall (Minimizer) and a Tool-Output Firewall (Sanitizer)
  arXiv  Detail & Related papers  (2025-10-06T18:09:02Z)
• BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks [51.803138848305814]
  We introduce BrowserArena, a live open-web agent evaluation platform that collects user-submitted tasks.<n>We identify three consistent failure modes: captcha resolution, pop-up banner removal, and direct navigation to URLs.<n>Our findings surface both the diversity and brittleness of current web agents.
  arXiv  Detail & Related papers  (2025-10-02T15:22:21Z)
• WebGuard: Building a Generalizable Guardrail for Web Agents [59.31116061613742]
  WebGuard is the first dataset designed to support the assessment of web agent action risks.<n>It contains 4,939 human-annotated actions from 193 websites across 22 diverse domains.
  arXiv  Detail & Related papers  (2025-07-18T18:06:27Z)
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety [58.201189860217724]
  We introduce OpenAgentSafety, a comprehensive framework for evaluating agent behavior across eight critical risk categories.<n>Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms.<n>It combines rule-based analysis with LLM-as-judge assessments to detect both overt and subtle unsafe behaviors.
  arXiv  Detail & Related papers  (2025-07-08T16:18:54Z)
• The Hidden Dangers of Browsing AI Agents [0.0]
  This paper presents a comprehensive security evaluation of such agents, focusing on systemic vulnerabilities across multiple architectural layers.<n>Our work outlines the first end-to-end threat model for browsing agents and provides actionable guidance for securing their deployment in real-world environments.
  arXiv  Detail & Related papers  (2025-05-19T13:10:29Z)
• AdvAgent: Controllable Blackbox Red-teaming on Web Agents [22.682464365220916]
  AdvAgent is a black-box red-teaming framework for attacking web agents.<n>It employs a reinforcement learning-based pipeline to train an adversarial prompter model.<n>With careful attack design, these prompts effectively exploit agent weaknesses while maintaining stealthiness and controllability.
  arXiv  Detail & Related papers  (2024-10-22T20:18:26Z)
• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents [3.09793323158304]
  Existing benchmarks measure only whether an agent finishes a task, ignoring whether it does so safely or in a way enterprises can trust.<n>We introduce textbftextscST-WebAgentBench, a suite for evaluating web agent ST across realistic enterprise scenarios.<n>Each of its 222 tasks is paired with ST policies, concise rules that encode constraints, and is scored along six dimensions (e.g., user consent, robustness)
  arXiv  Detail & Related papers  (2024-10-09T09:13:38Z)
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models [65.18602126334716]
  Existing web agents typically only handle one input modality and are evaluated only in simplified web simulators or static web snapshots. We introduce WebVoyager, an innovative Large Multimodal Model (LMM) powered web agent that can complete user instructions end-to-end by interacting with real-world websites. We show that WebVoyager achieves a 59.1% task success rate on our benchmark, significantly surpassing the performance of both GPT-4 (All Tools) and the WebVoyager (text-only) setups.
  arXiv  Detail & Related papers  (2024-01-25T03:33:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.