Perturbative methods for non-parametric instrumental variable

• URL: http://arxiv.org/abs/2606.00322v1
• Date: Fri, 29 May 2026 19:56:26 GMT
• Title: Perturbative methods for non-parametric instrumental variable
• Abstract summary: We introduce a perturbative approach for nonparametric instrumental variable (NPIV) estimation.<n>We show that our first-order perturbative corrections can reduce prediction error by up to 99% in high-dimensional ill-defined cases.
• Score: 20.067317290622377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a perturbative approach for nonparametric instrumental variable (NPIV) estimation. By drawing inspiration from perturbation theory in physics, we extend standard kernel ridge methods with systematic higher perturbation order corrections that significantly improve estimation accuracy. Spectrally, the perturbation introduces mixing between different eigenmodes of the expectation integral operator, which becomes especially useful when the integral equation is ill-defined. One source for such ill-definedness can be the curse of dimensionality. Our method performs across various dimensionality regimes, particularly when the dimensionality parameter $β$ which is defined through the number of samples $n$ and dimension $d$ as $n^β= d$, becomes large. Experimental results show that our first-order perturbative corrections can reduce prediction error by up to 99\% in high-dimensional ill-defined cases ($β> 0.7$) compared to standard ridge regression approaches. The performance improvement is maintained across a wide range of dimensions, with the advantage becoming more pronounced as dimensionality increases.

Related papers

• Nonparametric Estimation of Isotropic Covariance Function [6.456477445670593]
  A nonparametric model is constructed to approximate arbitrary isotropic covariance functions valid in $mathbbRinfty$.<n>A sieve maximum likelihood (sML) estimation is then developed to estimate the unknown isotropic covaraince function valid in $mathbbRinfty$.<n>Our approach outperforms the parametric methods in reducing bias due to model misspecification and also the nonparametric methods in terms of having significantly lower values of expected $L_infty$ and $L$ norms.
  arXiv  Detail & Related papers  (2026-04-24T07:51:27Z)
• Adaptive regularization parameter selection for high-dimensional inverse problems: A Bayesian approach with Tucker low-rank constraints [1.6936594801109557]
  Method reduces computational complexity by transforming variational inference from a high-dimensional space to a lower-dimensional core tensor space via Tucker decomposition.<n>Key innovation is the introduction of per-mode precision parameters, enabling adaptive regularization for anisotropic structures.<n>The approach scales to problems with 110,000 variables and outperforms existing methods by 0.73-2.09 dB in deblurring tasks and 6.75 dB in 3D heat conduction.
  arXiv  Detail & Related papers  (2026-03-17T02:23:44Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Differentially Private Learning Beyond the Classical Dimensionality Regime [19.030954416586596]
  We study differentially private learning in the proportional dimensionality regime.<n>We provide theoretical estimates of the error of several well-studied differentially private algorithms.
  arXiv  Detail & Related papers  (2024-11-20T19:56:12Z)
• A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning [74.80956524812714]
  We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning. These problems are often formalized as Bi-Level optimizations (BLO) We introduce a novel perspective by turning a given BLO problem into a ii optimization, where the inner loss function becomes a smooth distribution, and the outer loss becomes an expected loss over the inner distribution.
  arXiv  Detail & Related papers  (2024-10-14T12:10:06Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Extending Mean-Field Variational Inference via Entropic Regularization: Theory and Computation [3.1048285287226904]
  Variational inference (VI) has emerged as a popular method for approximate inference for high-dimensional Bayesian models.<n>We propose a novel VI method that extends the naive mean field via entropic regularization.<n>We show that $Xi$-variational posteriors effectively recover the true posterior dependency.
  arXiv  Detail & Related papers  (2024-04-14T01:40:11Z)
• Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression [29.57766164934947]
  We investigate popular resampling methods for estimating the uncertainty of statistical models. We provide a tight description of the biases and variances estimated by these methods in the context of generalized linear models.
  arXiv  Detail & Related papers  (2024-02-21T08:50:33Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)
• Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectives [50.12802772165797]
  The averaging technique combines all iterative solutions into a single solution. Experiments have demonstrated trade-off and the effectiveness of averaging compared with other averaging schemes.
  arXiv  Detail & Related papers  (2020-03-09T18:14:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.