Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction

• URL: http://arxiv.org/abs/2605.02234v1
• Date: Mon, 04 May 2026 05:09:21 GMT
• Title: Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction
• Abstract summary: We present a method for diagnosing interpretation in neural networks by identifying an input subspace where a proposed interpretation is highly faithful.<n>We refine this framework by partitioning the input space into well-interpreted and under-interpreted regions according to pairwise interchange-intervention behavior.
• Score: 12.945075768474618
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a method for diagnosing interpretation in neural networks by identifying an input subspace where a proposed interpretation is highly faithful. Our method is particularly useful for causal-abstraction-style interpretability, where a high-level causal hypothesis is evaluated by interchange interventions. Rather than treating interchange intervention accuracy as a single global summary, we refine this framework by partitioning the input space into well-interpreted and under-interpreted regions according to pairwise interchange-intervention behavior. This turns causal abstraction from a purely global evaluation into a more diagnostic tool: it not only measures whether an interpretation works, but also reveals where it works, where it fails, and what distinguishes the two cases. This diagnostic view also provides practical heuristics for improving interpretations. By analyzing the structure of the well-interpreted and under-interpreted regions, we can identify missing distinctions in a high-level hypothesis, discover previously unmodeled intermediate variables, and combine complementary partial interpretations into a stronger one. We instantiate this idea as a simple four-step recipe and show that it yields informative error analyses across multiple causal abstraction settings. In a toy logic task, recursively applying the recipe recovers a high-level hypothesis from scratch. More broadly, our results suggest that partitioning the input space is a useful step toward more precise, constructive, and scalable mechanistic interpretability.

Related papers

• Splitting the Difference: Interpretable Causal Forests for Treatment Effect Heterogeneity and Bias [0.0]
  We introduce an algorithm based on decision trees and random forests for estimating individual treatment effects.<n>Our algorithm is simple: it operates exactly like a standard random forest, but with a different splitting criterion.<n>It handles observational studies with varying treatment propensities without requiring separate estimation of the full propensity function.
  arXiv  Detail & Related papers  (2026-09-15T10:43:57Z)
• Causality is Key for Interpretability Claims to Generalise [35.833847356014154]
  Interpretability research on large language models (LLMs) has yielded important insights into model behaviour.<n> recurring pitfalls persist: findings that do not generalise, and causal interpretations that outrun the evidence.<n>Pearl's causal hierarchy clarifies what an interpretability study can justify.
  arXiv  Detail & Related papers  (2026-02-18T18:45:04Z)
• Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models [28.102903742881576]
  We introduce a framework for abstraction-guided reasoning that explicitly separates structural inference from lexical semantics.<n>We show that abstraction-aligned steering reduces content-driven errors and improves validity-sensitive performance.
  arXiv  Detail & Related papers  (2026-02-02T18:48:44Z)
• Schoenfeld's Anatomy of Mathematical Reasoning by Language Models [56.656180566692946]
  We adopt Schoenfeld's Episode Theory as an inductive, intermediate-scale lens and introduce ThinkARM (Anatomy of Reasoning in Models)<n>ThinkARM explicitly abstracts reasoning traces into functional reasoning steps such as Analysis, Explore, Implement, verify, etc.<n>We show that episode-level representations make reasoning steps explicit, enabling systematic analysis of how reasoning is structured, stabilized, and altered in modern language models.
  arXiv  Detail & Related papers  (2025-12-23T02:44:25Z)
• Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partition [50.68751788132789]
  We propose a fine-grained counterfactual explanation framework that generates both object-level and part-level interpretability.<n>Our approach yields explainable counterfactuals in a non-generative manner by quantifying similarity and weighting component contributions.
  arXiv  Detail & Related papers  (2025-11-11T08:38:38Z)
• InverseScope: Scalable Activation Inversion for Interpreting Large Language Models [5.670123459649656]
  InverseScope is an assumption-light and scalable framework for interpreting neural activations via input inversion.<n>To address the inefficiency of sampling in high-dimensional spaces, we propose a novel conditional generation architecture.<n>We also introduce a quantitative evaluation protocol that tests interpretability hypotheses using the feature consistency rate computed over the sampled inputs.
  arXiv  Detail & Related papers  (2025-06-09T03:59:28Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)
• Counterfactual Realizability [52.85109506684737]
  We introduce a formal definition of realizability, the ability to draw samples from a distribution, and then develop a complete algorithm to determine whether an arbitrary counterfactual distribution is realizable.<n>We illustrate the implications of this new framework for counterfactual data collection using motivating examples from causal fairness and causal reinforcement learning.
  arXiv  Detail & Related papers  (2025-03-14T20:54:27Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.