What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation

• URL: http://arxiv.org/abs/2607.26935v1
• Date: Wed, 29 Jul 2026 14:05:26 GMT
• Title: What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
• Abstract summary: Bot detectors deployed at scale treat traffic as binary: human or bot.<n>We present a three-class detection framework distinguishing humans, bots, and AI agents.<n>We show that the binary-vs-agent confusion is architectural: a binary human-vs-bot detector misroutes agent sessions because its label space lacks an agent class.
• Score: 0.9626041809440635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bot detectors deployed at scale treat traffic as binary: human or bot. This assumption breaks when AI agents browse the web through browser automation, a traffic class that is neither and that binary classifiers structurally cannot represent. We present a three-class detection framework distinguishing humans, bots, and AI agents, and show that the binary-vs-agent confusion is architectural: a binary human-vs-bot detector misroutes agent sessions because its label space lacks an agent class. On our controlled benchmark, an MLP binary classifier misclassifies 39.1% of real AI agents as human and a SAINT binary transformer misclassifies 34.5%; adding an explicit agent class yields per-class agent F1 = 1.000 in all 30 runs (3 model families $\times$ 10 seeds). To measure evasion resistance, we construct a five-level evasion ladder spanning passive observation, GAN-generated trajectories, and replay of real human cursor data ($n = 2299$ evasion sessions). Across 10 seeds and 3 model families we observe zero agent misses in 22990 per-seed predictions. The discriminative signal is a browser-automation artifact, not evidence of agent reasoning: Playwright does not emit the raw pointer-move and wheel-delta streams a physical input device produces, and this absence signature survives trajectory manipulation. Exhaustive search over all feature subsets of size 1-5 (9401 GBMs) shows that two behavioral features (mouse_event_rate, teleport_click_ratio) give 100% observed agent recall at every evasion level with agent precision 0.994; five features lift macro-F1 to 0.991. The signal is redundantly encoded: removing teleport_click_ratio leaves agent detection at 100%. The single-feature regime is degenerate, flagging every agent only by collapsing the classifier to always predict "agent". Two features robustly isolate agents; five separate all three traffic classes at macro-F1 $\geq 0.99$.

Related papers

• AgentGUI: An Interface for Observing and Steering Long-Running AI Agents [3.2387066735835273]
  AgentGUI is a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents.<n>A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces.<n>In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B--9B model ladder.
  arXiv  Detail & Related papers  (2026-07-28T21:47:22Z)
• Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection [80.15317858033534]
  ForeAgent is an agentic forensics framework for AI-generated image detection with iterative self-evolution.<n>ForeAgent achieves state-of-the-art performance on the Chameleon benchmark, reaching 82.18% accuracy.<n>ForeAgent produces more consistent and causally grounded reasoning compared to GPT-5 and GPT-5-mini.
  arXiv  Detail & Related papers  (2026-06-25T02:59:33Z)
• AgentRiskBOM: A Risk-Scoping Security Bill of Materials for Agentic AI Systems [0.519554837386174]
  This paper introduces AgentRiskBOM, a security BOM for risk-scoping tool-using AI agents.<n>It is an additive layer over SBOM, AIBOM, and MLBOM, referencing them where authoritative runtime fields for authority are added.<n>Coverage analysis gives AgentRiskBOM a native-equivalent score of 14 across 16 capability dimensions, vs. 1 for SBOM, 1.5 for AIBOM and 2 for MLBOM.
  arXiv  Detail & Related papers  (2026-06-20T04:38:41Z)
• Detecting Multi-Agent Collusion Through Multi-Agent Interpretability [0.3467226901703539]
  We introduce NARCBench, a benchmark for evaluating collusion detection under environment distribution shift.<n>We propose five probing techniques that aggregate per-agent deception scores to classify scenarios at the group level.<n>No single probing technique dominates across all collusion types, suggesting that different forms of collusion manifest differently in activation space.
  arXiv  Detail & Related papers  (2026-04-01T17:08:05Z)
• Re-Evaluating EVMBench: Are AI Agents Ready for Smart Contract Security? [10.248746359119625]
  EVMbench is the first large-scale benchmark for AI agents on smart contract security.<n>Its results have fueled expectations that fully automated AI auditing is within reach.<n>These findings challenge the narrative that fully automated AI auditing is imminent.
  arXiv  Detail & Related papers  (2026-03-11T14:07:16Z)
• Behind the Prompt: The Agent-User Problem in Information Retrieval [4.563318916484434]
  User models in information retrieval rest on a foundational assumption that observed behavior reveals intent.<n>For any action an agent takes, a hidden instruction could have produced identical output - making intent non-identifiable at the individual level.<n>We investigate the agent-user problem through a large-scale corpus from an agent-native social platform.
  arXiv  Detail & Related papers  (2026-03-04T01:42:14Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents [58.79302663733703]
  Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents.<n>The impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored.<n>This paper investigates what is the resilience of various system structures under faulty agents on different downstream tasks.
  arXiv  Detail & Related papers  (2024-08-02T03:25:20Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• Malicious Agent Detection for Robust Multi-Agent Collaborative Perception [52.261231738242266]
  Multi-agent collaborative (MAC) perception is more vulnerable to adversarial attacks than single-agent perception. We propose Malicious Agent Detection (MADE), a reactive defense specific to MAC perception. We conduct comprehensive evaluations on a benchmark 3D dataset V2X-sim and a real-road dataset DAIR-V2X.
  arXiv  Detail & Related papers  (2023-10-18T11:36:42Z)
• Traffic Agent Trajectory Prediction Using Social Convolution and Attention Mechanism [57.68557165836806]
  We propose a model to predict the trajectories of target agents around an autonomous vehicle. We encode the target agent history trajectories as an attention mask and construct a social map to encode the interactive relationship between the target agent and its surrounding agents. To verify the effectiveness of our method, we widely compare with several methods on a public dataset, achieving a 20% error decrease.
  arXiv  Detail & Related papers  (2020-07-06T03:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.