Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression

• URL: http://arxiv.org/abs/2512.00919v1
• Date: Sun, 30 Nov 2025 14:54:03 GMT
• Title: Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression
• Authors: Dimitri Meunier, Jakub Wornbard, Vladimir R. Kostic, Antoine Moulin, Alek Fröhlich, Karim Lounici, Massimiliano Pontil, Arthur Gretton,
• Abstract summary: We introduce Augmented Spectral Feature Learning, a framework that makes the feature learning process outcome-aware.<n>We provide a theoretical analysis of this framework and validate our approach on challenging benchmarks.
• Score: 37.76825470697479
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the problem of causal effect estimation in the presence of hidden confounders using nonparametric instrumental variable (IV) regression. An established approach is to use estimators based on learned spectral features, that is, features spanning the top singular subspaces of the operator linking treatments to instruments. While powerful, such features are agnostic to the outcome variable. Consequently, the method can fail when the true causal function is poorly represented by these dominant singular functions. To mitigate, we introduce Augmented Spectral Feature Learning, a framework that makes the feature learning process outcome-aware. Our method learns features by minimizing a novel contrastive loss derived from an augmented operator that incorporates information from the outcome. By learning these task-specific features, our approach remains effective even under spectral misalignment. We provide a theoretical analysis of this framework and validate our approach on challenging benchmarks.

Related papers

• Continuous-time reinforcement learning: ellipticity enables model-free value function approximation [1.3350982138577037]
  We study off-policy reinforcement learning for controlling continuous-time Markov diffusion processes with discrete-time observations and actions.<n>We consider model-free algorithms with function approximation that learn value and advantage functions directly from data.
  arXiv  Detail & Related papers  (2026-02-06T18:25:33Z)
• Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function [0.0]
  Signed Quadratic Shrink (SQS) is an activation function designed to allow Gated Linear Units (GLUs) to learn interpretable features.<n>Our experimental results show that SQS achieves performance competitive with state-of-the-art activation functions.
  arXiv  Detail & Related papers  (2025-09-02T01:42:39Z)
• Demystifying Spectral Feature Learning for Instrumental Variable Regression [21.514407689434364]
  We derive a generalization error bound for a two-stage least squares estimator based on spectral features.<n>We show that performance depends on two key factors, leading to a clear taxonomy of outcomes.
  arXiv  Detail & Related papers  (2025-06-12T17:06:43Z)
• Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits [58.63897489864948]
  Reinforcement learning with outcome-based feedback faces a fundamental challenge.<n>How do we assign credit to the right actions?<n>This paper provides the first comprehensive analysis of this problem in online RL with general function approximation.
  arXiv  Detail & Related papers  (2025-05-26T17:44:08Z)
• Iterative Feature Boosting for Explainable Speech Emotion Recognition [17.568724398229232]
  We present a new supervised SER method based on an efficient feature engineering approach. We pay particular attention to the explainability of results to evaluate feature relevance and refine feature sets. The proposed method outperforms human-level performance (HLP) and state-of-the-art machine learning methods in emotion recognition on the TESS dataset.
  arXiv  Detail & Related papers  (2024-05-30T15:44:27Z)
• Revisit, Extend, and Enhance Hessian-Free Influence Functions [26.105554752277648]
  Influence functions serve as crucial tools for assessing sample influence in model interpretation, subset training set selection, and more. In this paper, we revisit a specific, albeit effective approximation method known as Trac. This method substitutes the inverse of the Hessian matrix with an identity matrix.
  arXiv  Detail & Related papers  (2024-05-25T03:43:36Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient [65.08966446962845]
  offline reinforcement learning, which aims at optimizing decision-making strategies with historical data, has been extensively applied in real-life applications. We take a step by considering offline reinforcement learning with differentiable function class approximation (DFA) Most importantly, we show offline differentiable function approximation is provably efficient by analyzing the pessimistic fitted Q-learning algorithm.
  arXiv  Detail & Related papers  (2022-10-03T07:59:42Z)
• Causal Inference with Treatment Measurement Error: A Nonparametric Instrumental Variable Approach [24.52459180982653]
  We propose a kernel-based nonparametric estimator for the causal effect when the cause is corrupted by error. We empirically show that our proposed method, MEKIV, improves over baselines and is robust under changes in the strength of measurement error.
  arXiv  Detail & Related papers  (2022-06-18T11:47:25Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.