Automated Interpretability and Feature Discovery in Language Models with Agents

• URL: http://arxiv.org/abs/2605.01555v1
• Date: Sat, 02 May 2026 17:53:30 GMT
• Title: Automated Interpretability and Feature Discovery in Language Models with Agents
• Abstract summary: We introduce an autonomous multiagent framework for mechanistic interpretability that automates both explaining and finding internal features in large language models.<n>We show that agent-driven empirical loops yield sharper and more falsifiable explanations than one-shot labels.
• Score: 3.660409142843126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce an autonomous multiagent framework for mechanistic interpretability that automates both explaining and finding internal features in large language models. The system runs two coupled loops: (1) explanation refinement, where an agent proposes competing hypotheses and iteratively tests them with targeted prompt controls and a multi-metric evaluation; and (2) feature discovery, where an agent generates prompt sets, constructs a k-nearest-neighbor graph in activation space, and retrieves candidate features using statistical separability and semantic coherence criteria. On Gemma-2 family models and MLP neurons in weight-sparse transformers, our agent improves over one-shot auto-interpretations, discovers language-specific and safety-relevant features, and produces auditable explanation traces, showing that agent-driven empirical loops yield sharper and more falsifiable explanations than one-shot labels.

Related papers

• Declarative Skills for AI Agents in Knowledge-Grounded Tool-Use Workflows [1.9573380763700712]
  We study orchestration mechanisms for tool-using AI agents in realistic customer-service errors over an unstructured knowledge base.<n>We compare (i) a DeclarativeAgent that reads three domain-specific skill files at inference time and decides its own control flow, (ii) an ImperativeAgent based on a programmatic state machine with explicit phases, and (iii) an unscaffolded baseline agent modeled after the $$-Knowledge benchmark agent.
  arXiv  Detail & Related papers  (2026-06-05T05:38:51Z)
• How to Interpret Agent Behavior [56.59836196946289]
  We introduce ACT*ONOMY, a taxonomy for describing and analyzing agent behavior at runtime.<n>By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently.
  arXiv  Detail & Related papers  (2026-05-13T14:52:40Z)
• Agent Mentor: Framing Agent Knowledge through Semantic Trajectory Analysis [4.365760422569902]
  We introduce an analytics pipeline implemented as part of the Agent Mentor open-source library.<n>It monitors and incrementally adapts the system prompts defining another agent's behavior.<n>The pipeline improves performance by systematically injecting corrective instructions into the agent's knowledge.
  arXiv  Detail & Related papers  (2026-04-12T08:02:54Z)
• SAGE: A Service Agent Graph-guided Evaluation Benchmark [27.342044311161654]
  We propose SAGE (Service Agent Graph-guided Evaluation), a universal benchmark for automated, dual-axis assessment.<n>SAGE formalizes unstructured SOPs into Dynamic Dialogue Graphs, enabling precise verification of logical compliance.<n>We also observe Empathy Resilience'', a phenomenon where models maintain polite conversational facades despite underlying logical failures.
  arXiv  Detail & Related papers  (2026-04-10T12:55:23Z)
• When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Attribution [10.973058523304042]
  We introduce IET (Implicit Execution Tracing), a metadata-independent framework that enables token-level attribution directly from generated text.<n>During generation, agent-specific keyed signals are embedded into the token distribution, transforming the text into a self-describing execution trace detectable only with a secret key.<n>At detection time, a transition-aware scoring method identifies agent handover points and reconstructs the interaction graph.
  arXiv  Detail & Related papers  (2026-03-18T07:34:51Z)
• LinguistAgent: A Reflective Multi-Model Platform for Automated Linguistic Annotation [0.0]
  This paper introduces LinguistAgent, an integrated platform that leverages a reflective multi-model architecture to automate linguistic annotation.<n>The system implements a dual-agent workflow, comprising an Annotator and a Reviewer, to simulate a professional peer-review process.<n>We demonstrate LinguistAgent's efficacy using the task of metaphor identification as an example, providing real-time token-level evaluation.
  arXiv  Detail & Related papers  (2026-02-05T09:55:19Z)
• Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection [59.04089915447622]
  ForenAgent is an interactive IFD framework that enables MLLMs to autonomously generate, execute, and refine Python-based low-level tools around the detection objective.<n>Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication.<n>Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks.
  arXiv  Detail & Related papers  (2025-12-18T08:38:44Z)
• AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment [69.06977852423564]
  Image quality assessment (IQA) reflects both the quantification and interpretation of perceptual quality rooted in the human visual system.<n>AgenticIQA decomposes IQA into four subtasks -- distortion detection, distortion analysis, tool selection, and tool execution.<n>To support training and evaluation, we introduce AgenticIQA-200K, a large-scale instruction dataset tailored for IQA agents, and AgenticIQA-Eval, the first benchmark for assessing the planning, execution, and summarization capabilities of VLM-based IQA agents.
  arXiv  Detail & Related papers  (2025-09-30T09:37:01Z)
• MCP-Orchestrated Multi-Agent System for Automated Disinformation Detection [84.75972919995398]
  This paper presents a multi-agent system that uses relation extraction to detect disinformation in news articles.<n>The proposed Agentic AI system combines four agents: (i) a machine learning agent (logistic regression), (ii) a Wikipedia knowledge check agent, and (iv) a web-scraped data analyzer.<n>Results demonstrate that the multi-agent ensemble achieves 95.3% accuracy with an F1 score of 0.964, significantly outperforming individual agents and traditional approaches.
  arXiv  Detail & Related papers  (2025-08-13T19:14:48Z)
• A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition [71.61103962200666]
  Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora.<n>Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates.<n>We introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER.
  arXiv  Detail & Related papers  (2025-02-25T23:30:43Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.