Data-driven Circuit Discovery for Interpretability of Language Models

• URL: http://arxiv.org/abs/2605.09129v1
• Date: Sat, 09 May 2026 19:20:57 GMT
• Title: Data-driven Circuit Discovery for Interpretability of Language Models
• Abstract summary: Circuit discovery aims to explain how language models (LMs) implement a specific task by localizing and interpreting a circuit.<n>Existing methods first informally define a task with a dataset, and then apply a circuit discovery algorithm over that dataset to obtain a single circuit.<n>We propose Data-driven Circuit Discovery (DCD), a new discovery framework that drops both assumptions.
• Score: 32.70005004096789
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Circuit discovery aims to explain how language models (LMs) implement a specific task by localizing and interpreting a circuit, a computational subgraph responsible for the LM's behavior. Existing circuit discovery methods are hypothesis-driven; they first informally define a task with a dataset, and then apply a circuit discovery algorithm over that dataset to obtain a single circuit. This imposes two strong assumptions: that the LM implements the task with a single circuit, and that the dataset adequately represents the task as humans understand it. We systematically test these assumptions across four previously studied tasks and find that even minor dataset variations that preserve task semantics can produce circuits with low edge overlap and cross-dataset faithfulness. More strikingly, when applied to a mixed dataset with two distinct tasks whose separately discovered circuits have near-zero cross-faithfulness, existing methods still return a single circuit with high faithfulness across both tasks. This indicates that current methods discover dataset-specific circuits, rather than general task circuits. We propose Data-driven Circuit Discovery (DCD), a new discovery framework that drops both assumptions: instead of returning a single circuit for a dataset, DCD first clusters examples in the dataset by how similarly the model processes them and discovers a separate circuit for each group. This allows distinct mechanisms to appear separately rather than merged into a single circuit; each circuit explains its group, not the full task. Experiments show that DCD discovers multiple circuits per dataset, each more faithful to its group than a single circuit discovered by existing methods. Broadly, DCD lets the data reveal mechanistic structure within LMs, rather than relying on human-defined task boundaries that may not align with how models organize their computation.

Related papers

• EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents [64.96332056338923]
  EEVEE is the first multi-dataset test-time prompt learning framework for LLM agents.<n>It enables test-time prompt learning under real-world task streams.<n>EEVEE improves average multi-benchmark scores by 10.38 and 24.32 points over Qwen3-4B-Instruct and DeepSeek-V3.2.
  arXiv  Detail & Related papers  (2026-06-09T17:57:16Z)
• Certified Circuits: Stability Guarantees for Mechanistic Circuits [80.30622018787835]
  Certified Circuits provides provable stability guarantees for circuit discovery.<n>On ImageNet and OOD datasets, certified circuits achieve up to 91% higher accuracy.
  arXiv  Detail & Related papers  (2026-02-26T13:07:31Z)
• Finding Highly Interpretable Prompt-Specific Circuits in Language Models [4.768156759829138]
  We show that circuits are prompt-specific, even within a fixed task.<n>We introduce ACC++, refinements that extract cleaner, lower-dimensional causal signals inside attention heads from a single forward pass.<n>We develop an automated interpretability pipeline that uses ACC++ signals to surface human-interpretable features.
  arXiv  Detail & Related papers  (2026-02-13T21:41:17Z)
• Query Circuits: Explaining How Language Models Answer User Prompts [13.16677655895186]
  We introduce query circuits, which trace the information flow inside a model that maps a specific input to the output.<n>NDF is a metric to evaluate how well a discovered circuit recovers the model's decision for a specific input.<n>We find that there exist extremely sparse query circuits within the model that can recover much of its performance on single queries.
  arXiv  Detail & Related papers  (2025-09-29T13:59:02Z)
• Position-aware Automatic Circuit Discovery [59.64762573617173]
  We identify a gap in existing circuit discovery methods, treating model components as equally relevant across input positions.<n>We propose two improvements to incorporate positionality into circuits, even on tasks containing variable-length examples.<n>Our approach enables fully automated discovery of position-sensitive circuits, yielding better trade-offs between circuit size and faithfulness compared to prior work.
  arXiv  Detail & Related papers  (2025-02-07T00:18:20Z)
• A Unified Data Representation Learning for Non-parametric Two-sample Testing [50.27067977793069]
  We propose a representation-learning two-sample testing (RL-TST) framework.<n> RL-TST first performs purely self-supervised representation learning on the entire dataset.<n>A discriminative model is then trained on these IRs to learn discriminative representations (DRs)
  arXiv  Detail & Related papers  (2024-11-30T23:23:52Z)
• Transformer Circuit Faithfulness Metrics are not Robust [0.04260910081285213]
  We measure circuit 'faithfulness' by ablating portions of the model's computation. We conclude that existing circuit faithfulness scores reflect both the methodological choices of researchers as well as the actual components of the circuit. The ultimate goal of mechanistic interpretability work is to understand neural networks, so we emphasize the need for more clarity in the precise claims being made about circuits.
  arXiv  Detail & Related papers  (2024-07-11T17:59:00Z)
• Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity [18.71252449465396]
  We introduce DiscoGP, a framework for extracting self-contained modular units from neural language models (LMs)<n>Our framework identifies sheaves through a gradient-based pruning algorithm that operates on both of these in such a way that reduces the original LM to a sparse skeleton that preserves certain core capabilities.
  arXiv  Detail & Related papers  (2024-07-04T09:42:25Z)
• Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition [10.13822875330178]
  We introduce contextual decomposition for transformers (CD-T) to build interpretable circuits in large language models.<n>CD-T can produce circuits of arbitrary level of abstraction, and is the first able to produce circuits as fine-grained as attention heads.<n>We show CD-T circuits are able to perfectly replicate original models' behavior using fewer nodes than the baselines for all tasks.
  arXiv  Detail & Related papers  (2024-07-01T01:12:20Z)
• Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
  Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance. We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features. In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
  arXiv  Detail & Related papers  (2024-06-12T08:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.