The Representational Limit of Scalar Interactions: An Interventional Decomposition

• URL: http://arxiv.org/abs/2606.19410v1
• Date: Wed, 17 Jun 2026 15:21:28 GMT
• Title: The Representational Limit of Scalar Interactions: An Interventional Decomposition
• Abstract summary: We prove that signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S)<n>We introduce Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference.
• Score: 9.850981754860754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S). We prove this on a minimal 3-way XOR structural causal model: faithful indices such as Shapley-Taylor return zero per pair, whereas projective indices such as Shapley Interaction spread the third-order effect into pair scalars that conflate the three mechanisms. We introduce Stochastic Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference. The estimator provides exact interventional semantics, finite-sample Monte Carlo bounds, strict variance reduction from coupled diamond sampling, and uniform finite-vocabulary convergence. Across tabular SCMs, Stochastic Hi-Fi recovers structure missed by scalar baselines (up to 411x larger interaction-magnitude recovery ratios). It also separates redundant and synergistic heads in the GPT-2 IOI circuit. On NIH ChestX-ray14, Stochastic Hi-Fi matches GradCAM on Pointing Game and improves substantially on Deletion AUC.

Related papers

• Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and a Tight Univariate Rate [47.36630149538994]
  In high-dimensional online prediction, the best predictor may depend on only a few features, so regret should scale with sparsity rather than the ambient dimension.<n>Warmuth and Amid asked at COLT 2023 whether any of three such rules admits a competitive online regret guarantee.<n>Our analysis identifies a common obstruction: cheap nuisance causes the refit to underweight the truly predictive coordinate.
  arXiv  Detail & Related papers  (2026-08-18T09:32:03Z)
• The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks [0.0]
  We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead.<n>We compare routes to exact additivity, spanning structural masks, behavioral penalties, weight decay, backfitting, the shared-section model, and bagged boosted stumps.
  arXiv  Detail & Related papers  (2026-07-22T14:23:37Z)
• CausalGate: Causal Importance Distillation for Transformer Module Pruning [0.0]
  CausalGate is an intervention-guided framework for compute-efficient runtime inference.<n>It measures semantic damage via the Kullback-Leibler divergence of the final logit distribution.<n>It consistently outperforms prominent dynamic routing and layer-skipping baselines.
  arXiv  Detail & Related papers  (2026-07-21T20:19:59Z)
• C$^{2}$R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders [56.67585330425431]
  Sparse Autoencoders (SAEs) are widely used to interpret large language models by decomposing activations into sparse, human-understandable features.<n>These issues stem from inconsistent latent assignment across samples.<n>We introduce C$2$R, which penalizes the co-activation of directionally similar latents.
  arXiv  Detail & Related papers  (2026-06-29T17:45:31Z)
• UniRank: Unified Rank Allocation for Low-Rank LLM Compression [9.742379971053806]
  We formulate global low-rank allocation as a sorting-and-truncation pipeline.<n>We verify the strong correlation between high input-output cosine similarity and low effective rank.<n>We propose rank-preserving fine-tuning, which performs direct LoRA tuning on decomposed weights.
  arXiv  Detail & Related papers  (2026-06-20T03:10:33Z)
• Deep Slice Interpolation for Reducing Through-Plane Anisotropy and Noise in Head CT [0.0]
  We present a deep learning system that synthesizes intermediate CT slices from pairs of neighboring axial slices.<n>The system improves three-dimensional visualization while simultaneously producing inherently denoised outputs.
  arXiv  Detail & Related papers  (2026-06-08T10:45:51Z)
• Fractals made Practical: Denoising Diffusion as Partitioned Iterated Function Systems [0.0]
  We show that the DDIM reverse chain operates as a Partitioned Iterated Function System (PIFS)<n>PIFS serves as a unified design language for denoising diffusion model schedules, architectures, and training objectives.
  arXiv  Detail & Related papers  (2026-03-13T15:15:50Z)
• Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction [45.25461515976432]
  Plug-and-Play diffusion prior (DP) frameworks have emerged as a powerful paradigm for imaging reconstruction.<n>We present a novel approach to resolving bias-hallucination trade-off, achieving state-of-the-art gradients with significantly accelerated convergence.
  arXiv  Detail & Related papers  (2026-02-26T16:58:43Z)
• An Odd Estimator for Shapley Values [19.262788739385012]
  The Shapley value is a framework for attribution in machine learning, encompassing feature importance, data valuation, and causal inference.<n>We prove that the Shapley value depends exclusively on the odd component of the set function, and that paired samplingizes the objective regression to filter out the irrelevant even component.<n>We propose OddSHAP, a consistent estimator that performs regression solely on the odd subspace.
  arXiv  Detail & Related papers  (2026-02-01T19:07:16Z)
• Identifiable Convex-Concave Regression via Sub-gradient Regularised Least Squares [1.9580473532948397]
  We propose a novel nonparametric regression method that models complex input-relationships as the sum of convex and concave components.<n>The method-ICCNLS-decomposes sub-constrained shape-constrained additive decomposition.
  arXiv  Detail & Related papers  (2025-06-22T15:53:12Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computation [52.73824786627612]
  This paper establishes new convergence results for textitgeodesic strongly monotone games.<n>Our key result shows that RGD attains last-iterate linear convergence in a textitgeometry-agnostic fashion.<n>Overall, this paper presents the first geometry-agnostic last-iterate convergence analysis for games beyond the Euclidean settings.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning [47.904127007515925]
  We study a variant of the classical temporal difference (TD) learning algorithm with a perturbed update direction. We prove that compressed TD algorithms, coupled with an error-feedback mechanism used widely in optimization, exhibit the same non-asymptotic approximation guarantees as their counterparts. Notably, these are the first finite-time results in RL that account for general compression operators and error-feedback in tandem with linear function approximation and Markovian sampling.
  arXiv  Detail & Related papers  (2023-01-03T04:09:38Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals [22.77704627076251]
  Learning the joint probability of random variables (RVs) is the cornerstone of statistical signal processing and machine learning. Recent work has proposed to recover the joint probability mass function (PMF) of an arbitrary number of RVs from three-dimensional marginals. accurately estimating three-dimensional marginals can still be costly in terms of sample complexity. This work puts forth a new framework for learning the joint PMF using only pairwise marginals.
  arXiv  Detail & Related papers  (2020-06-30T15:43:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.