How to Interpret Agent Behavior
- URL: http://arxiv.org/abs/2605.13625v1
- Date: Wed, 13 May 2026 14:52:40 GMT
- Title: How to Interpret Agent Behavior
- Abstract summary: We introduce ACT*ONOMY, a taxonomy for describing and analyzing agent behavior at runtime.<n>By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently.
- Score: 56.59836196946289
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous agents such as Claude Code and Codex now operate for hours or even days. Understanding their runtime behavior has become critical for downstream tasks such as diagnosing inefficiencies, fixing bugs, and ensuring better oversight. A primary way to gain this understanding is analyzing the reasoning trajectories and execution traces these agents generate. Yet such data remains in unstructured natural-language form, making it difficult for humans to interpret at scale. We introduce ACT*ONOMY (a combination of Action and Taxonomy), a taxonomy for describing and analyzing agent behavior at runtime. ACT*ONOMY has two components: (1) the taxonomy itself, developed through Grounded Theory and structured as a three-level hierarchy of 10 actions, 46 subactions, and 120 leaf categories; and (2) an open repository that hosts the living taxonomy, provides an automated analysis pipeline that applies it to agent trajectories analysis, and defines an extension protocol for customization and growth. Our experiments show that ACTONOMY can compare behavioral profiles across agents and characterize a single agent's behavior across diverse trajectories, surfacing patterns indicative of failure modes. By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently, enabling better oversight and control.
Related papers
- ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
arXiv Detail & Related papers (2026-08-03T15:07:46Z) - Biological Motifs for Agentic Control [0.0]
Current agentic architectures are often constructed ad-hoc, prone to cascades, infinite loops, and prompt injection attacks.<n>This paper argues that many of these failure modes can be analyzed using control motifs long studied in systems biology.<n>We develop a typed interface correspondence between Gene Regulatory Networks and agentic software systems using functors and wiring diagrams.
arXiv Detail & Related papers (2026-07-05T11:30:45Z) - Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents [12.703661629160571]
We present Agentic CLEAR, an automatic, dynamic, and easy-to-use evaluation framework.<n>It produces textual insights into the agent behavior on three levels of granularity: system, trace, and node.<n>Agentic CLEAR operates above the observability layer, enabling seamless integration and featuring an intuitive UI.
arXiv Detail & Related papers (2026-05-21T15:26:02Z) - Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
arXiv Detail & Related papers (2026-05-10T06:30:19Z) - Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
arXiv Detail & Related papers (2026-04-21T15:05:58Z) - Agent Mentor: Framing Agent Knowledge through Semantic Trajectory Analysis [4.365760422569902]
We introduce an analytics pipeline implemented as part of the Agent Mentor open-source library.<n>It monitors and incrementally adapts the system prompts defining another agent's behavior.<n>The pipeline improves performance by systematically injecting corrective instructions into the agent's knowledge.
arXiv Detail & Related papers (2026-04-12T08:02:54Z) - The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
arXiv Detail & Related papers (2026-01-21T15:22:21Z) - ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
arXiv Detail & Related papers (2026-01-11T11:05:26Z) - Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
arXiv Detail & Related papers (2025-10-06T05:03:57Z) - Understanding Software Engineering Agents Through the Lens of Traceability: An Empirical Study [15.97770416681533]
Software engineering agents (SWE agents) operate autonomously by interpreting user input and responding to environmental feedback.<n>We present the first systematic study of SWE agent behavior through the lens of execution traces.
arXiv Detail & Related papers (2025-06-10T00:41:54Z) - TRAIL: Trace Reasoning and Agentic Issue Localization [5.025960714013197]
This work articulates the need for robust and dynamic evaluation methods for agentic workflow traces.<n>We present a set of 148 large human-annotated traces (TRAIL) constructed using this taxonomy and grounded in established agentic benchmarks.<n>To ensure ecological validity, we curate traces from both single and multi-agent systems.
arXiv Detail & Related papers (2025-05-13T14:55:31Z) - AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous
Systems [7.814964460974541]
We present a visualization approach to explore detailed statuses and agents' behavior within LLMAS.
We then develop AgentLens, a visual analysis system that supports users to interactively investigate details and causes of agents' behaviors.
arXiv Detail & Related papers (2024-02-14T07:48:16Z) - A Framework for Understanding and Visualizing Strategies of RL Agents [0.0]
We present a framework for learning comprehensible models of sequential decision tasks in which agent strategies are characterized using temporal logic formulas.
We evaluate our framework on combat scenarios in StarCraft II (SC2) using traces from a handcrafted expert policy and a trained reinforcement learning agent.
arXiv Detail & Related papers (2022-08-17T21:58:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.