The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

• URL: http://arxiv.org/abs/2606.27510v1
• Date: Thu, 25 Jun 2026 19:52:16 GMT
• Title: The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching
• Abstract summary: Activation patching is the primary tool in mechanistic interpretability.<n>It attributes causal responsibility for a model behavior to each of its individual components by estimating its natural indirect effect.<n>It also contains interaction effects (INT) that measure how much the component's causal effect itself depends on the state of other components in the model.
• Score: 28.393865595242872
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation patching is the primary tool in mechanistic interpretability. It attributes causal responsibility for a model behavior to each of its individual components by estimating its natural indirect effect (NIE). Re-deriving the activation patching estimand from causal mediation analysis, we find that the NIE does not solely capture the causal effect through the specific component. It also contains interaction effects (INT) that measure how much the component's causal effect itself depends on the state of other components in the model. A natural response may be to try to eliminate INT by adjusting the estimator or unit of analysis, but each of these potential remedies has predictable failure modes. We demonstrate these failure modes in the GPT-2 IOI circuit; components whose causal importance is conditional on the state of other components are either invisible or artificially inflated, and INT variance explains the previously documented instability of faithfulness scores. We prove that INT scales with the distance between clean and patched component activations, is negligible when the model is locally affine, and decomposes combinatorially into pairwise and higher-order group interactions. Despite its inevitability, INT is not a nuisance to be eliminated, but rather a diagnostic for interpretability studies. Its individual and group-level magnitude and sign signal when causal conclusions are prompt-dependent, and when greedy NIE-based component ranking will miss mechanisms only discoverable through combinatorial search.

Related papers

• Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects [58.98080129016082]
  Feature-Effect Geometry Analysis (FEGA) is an unsupervised framework that removes the same active SAE feature across contexts.<n>We show that a feature can be interpretable and causally relevant without providing a stable direction for steering.
  arXiv  Detail & Related papers  (2026-07-27T16:45:08Z)
• When Are Neural Interaction Discoveries Real? Identifiability, Recoverability, and a Pre-Fit Diagnostic [0.0]
  We argue that this is fundamentally a question of identifiability, governed by the geometry of the observed input support.<n>We show that representational capacity is not identifiability.<n>We show that interaction recoverability depends on support geometry.
  arXiv  Detail & Related papers  (2026-06-07T00:52:43Z)
• Disentangled Double Machine Learning for Accurate Causal Effect Estimation [80.07136847623498]
  Confounding bias is a key challenge in causal effect estimation from observational data.<n>Double Machine Learning (DML) addresses this issue by estimating treatment and outcome nuisance functions.<n>DML often produces biased and unstable estimates in highdimensional or finite-sample scenarios.<n>We propose Disentangled Double Machine Learning (DDML), a novel algorithm that integrates two key strategies.
  arXiv  Detail & Related papers  (2026-05-24T01:40:43Z)
• CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation [0.996038809132083]
  We introduce CUE-R, a framework for measuring per-evidence-item operational utility in single-shot RAG.<n>We show that CUE-R perturbs individual evidence items via REMOVE, REPLACE, and DUPLICATE operators, then measures changes along three utility axes.<n>Our results suggest that answer-only evaluation misses important evidence effects and that intervention-based utility analysis is a practical complement for RAG evaluation.
  arXiv  Detail & Related papers  (2026-04-07T06:05:08Z)
• Causal Discovery with Mixed Latent Confounding via Precision Decomposition [0.0]
  Differentiable and score-based DAG learners can misinterpret global latent effects as causal edges, while latent-variable graphical models recover only undirected structure.<n>We propose textscDCL-DECOR, a modular, precision-led pipeline that separates these roles.
  arXiv  Detail & Related papers  (2025-12-31T08:03:41Z)
• CausalAffect: Causal Discovery for Facial Affective Understanding [26.904783570786773]
  CausalAffect is the first framework for causal graph discovery in facial affect analysis.<n>Our approach requires neither jointly annotated datasets nor handcrafted causal priors.<n>All trained models and source code will be released upon acceptance.
  arXiv  Detail & Related papers  (2025-11-29T12:07:33Z)
• A new paradigm for global sensitivity analysis [0.0]
  Current theory of global sensitivity analysis is limited in scope-for instance, the analysis is limited to the output's variance. It is shown that these important problems are solved all at once by adopting a new paradigm.
  arXiv  Detail & Related papers  (2024-09-10T07:20:51Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• CausalDialogue: Modeling Utterance-level Causality in Conversations [83.03604651485327]
  We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
  arXiv  Detail & Related papers  (2022-12-20T18:31:50Z)
• Efficient Causal Inference from Combined Observational and Interventional Data through Causal Reductions [68.6505592770171]
  Unobserved confounding is one of the main challenges when estimating causal effects. We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders. We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
  arXiv  Detail & Related papers  (2021-03-08T14:29:07Z)
• Disentangling Observed Causal Effects from Latent Confounders using Method of Moments [67.27068846108047]
  We provide guarantees on identifiability and learnability under mild assumptions. We develop efficient algorithms based on coupled tensor decomposition with linear constraints to obtain scalable and guaranteed solutions.
  arXiv  Detail & Related papers  (2021-01-17T07:48:45Z)
• CausalVAE: Structured Causal Disentanglement in Variational Autoencoder [52.139696854386976]
  The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations. We propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent factors into causal endogenous ones. Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy.
  arXiv  Detail & Related papers  (2020-04-18T20:09:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.