Nonparametric Instrumental Variable Regression with Observed Covariates

• URL: http://arxiv.org/abs/2511.19404v1
• Date: Mon, 24 Nov 2025 18:42:49 GMT
• Title: Nonparametric Instrumental Variable Regression with Observed Covariates
• Authors: Zikai Shen, Zonghao Chen, Dimitri Meunier, Ingo Steinwart, Arthur Gretton, Zhu Li,
• Abstract summary: We study the problem of nonparametric instrumental variable regression with observed covariables, which we refer to as NPIV-O.<n> observed covariables induce a partial identity structure, which renders previous NPIV analyses inapplicable.<n>For the second challenge, we introduce a novel Fourier measure of partial smoothing.<n>We prove upper $L2$-learning rates for KIV-O and the first $L2$-minimax lower learning rates for NPIV-O.
• Score: 32.91951875577007
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of nonparametric instrumental variable regression with observed covariates, which we refer to as NPIV-O. Compared with standard nonparametric instrumental variable regression (NPIV), the additional observed covariates facilitate causal identification and enables heterogeneous causal effect estimation. However, the presence of observed covariates introduces two challenges for its theoretical analysis. First, it induces a partial identity structure, which renders previous NPIV analyses - based on measures of ill-posedness, stability conditions, or link conditions - inapplicable. Second, it imposes anisotropic smoothness on the structural function. To address the first challenge, we introduce a novel Fourier measure of partial smoothing; for the second challenge, we extend the existing kernel 2SLS instrumental variable algorithm with observed covariates, termed KIV-O, to incorporate Gaussian kernel lengthscales adaptive to the anisotropic smoothness. We prove upper $L^2$-learning rates for KIV-O and the first $L^2$-minimax lower learning rates for NPIV-O. Both rates interpolate between known optimal rates of NPIV and nonparametric regression (NPR). Interestingly, we identify a gap between our upper and lower bounds, which arises from the choice of kernel lengthscales tuned to minimize a projected risk. Our theoretical analysis also applies to proximal causal inference, an emerging framework for causal effect estimation that shares the same conditional moment restriction as NPIV-O.

Related papers

• From Tail Universality to Bernstein-von Mises: A Unified Statistical Theory of Semi-Implicit Variational Inference [0.12183405753834557]
  Semi-implicit variational inference (SIVI) constructs approximate posteriors of the form $q() = int k(| z) r(dz)$<n>This paper develops a unified "approximation-optimization-statistics'' theory for such families.
  arXiv  Detail & Related papers  (2025-12-05T19:26:25Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression [57.40108516085593]
  Deep feature instrumental variable (DFIV) regression is a nonparametric approach to IV regression using data-adaptive features learned by deep neural networks.<n>We prove that the DFIV algorithm achieves the minimax optimal learning rate when the target structural function lies in a Besov space.
  arXiv  Detail & Related papers  (2025-01-09T01:22:22Z)
• Nonparametric Instrumental Regression via Kernel Methods is Minimax Optimal [28.361133177290657]
  We study the kernel instrumental variable algorithm of citetsingh 2019.<n>We show that the kernel NPIV estimator converges to the IV solution with minimum norm.<n>We also improve the original kernel NPIV algorithm by adopting a general spectral regularization in stage 1 regression.
  arXiv  Detail & Related papers  (2024-11-29T12:18:01Z)
• Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients [0.3277163122167434]
  We show how to formulate a functional gradient descent algorithm to tackle NPIV regression by directly minimizing the populational risk.<n>We provide theoretical support in the form of bounds on the excess risk, and conduct numerical experiments showcasing our method's superior stability and competitive performance.<n>This algorithm enables flexible estimator choices, such as neural networks or kernel based methods, as well as non-quadratic loss functions.
  arXiv  Detail & Related papers  (2024-02-08T12:50:38Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• Minimax Instrumental Variable Regression and $L_2$ Convergence Guarantees without Identification or Closedness [71.42652863687117]
  We study nonparametric estimation of instrumental variable (IV) regressions. We propose a new penalized minimax estimator that can converge to a fixed IV solution. We derive a strong $L$ error rate for our estimator under lax conditions.
  arXiv  Detail & Related papers  (2023-02-10T18:08:49Z)
• Asymptotically Unbiased Instance-wise Regularized Partial AUC Optimization: Theory and Algorithm [101.44676036551537]
  One-way Partial AUC (OPAUC) and Two-way Partial AUC (TPAUC) measures the average performance of a binary classifier. Most of the existing methods could only optimize PAUC approximately, leading to inevitable biases that are not controllable. We present a simpler reformulation of the PAUC problem via distributional robust optimization AUC.
  arXiv  Detail & Related papers  (2022-10-08T08:26:22Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• Nested Nonparametric Instrumental Variable Regression [2.2984209387877628]
  Several causal parameters in short panel data models are functionals of a nested nonparametric instrumental variable regression (nested NPIV)<n>Recent examples include mediated, time varying, and long term treatment effects identified using proxy variables.<n>We provide explicit mean square rates for nested NPIV and efficient inference for recently identified causal parameters.
  arXiv  Detail & Related papers  (2021-12-28T18:29:56Z)
• On Deep Instrumental Variables Estimate [21.003925102068298]
  "Deep Instrumental Variable (IV)" is a framework based on deep neural networks to address endogeneity. We consider a two-stage estimator using deep neural networks in the linear instrumental variables model. We show that the second-stage estimator achieves the semiparametric efficiency bound.
  arXiv  Detail & Related papers  (2020-04-30T17:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.