Debiased Ill-Posed Regression

• URL: http://arxiv.org/abs/2505.20787v1
• Date: Tue, 27 May 2025 06:47:33 GMT
• Title: Debiased Ill-Posed Regression
• Authors: AmirEmad Ghassami, James M. Robins, Andrea Rotnitzky,
• Abstract summary: We propose a debiased estimation strategy based on the influence function of a modification of the projected error.<n>Our proposed estimator possesses a second-order bias with respect to the involved nuisance functions.
• Score: 8.495265117285223
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In various statistical settings, the goal is to estimate a function which is restricted by the statistical model only through a conditional moment restriction. Prominent examples include the nonparametric instrumental variable framework for estimating the structural function of the outcome variable, and the proximal causal inference framework for estimating the bridge functions. A common strategy in the literature is to find the minimizer of the projected mean squared error. However, this approach can be sensitive to misspecification or slow convergence rate of the estimators of the involved nuisance components. In this work, we propose a debiased estimation strategy based on the influence function of a modification of the projected error and demonstrate its finite-sample convergence rate. Our proposed estimator possesses a second-order bias with respect to the involved nuisance functions and a desirable robustness property with respect to the misspecification of one of the nuisance functions. The proposed estimator involves a hyper-parameter, for which the optimal value depends on potentially unknown features of the underlying data-generating process. Hence, we further propose a hyper-parameter selection approach based on cross-validation and derive an error bound for the resulting estimator. This analysis highlights the potential rate loss due to hyper-parameter selection and underscore the importance and advantages of incorporating debiasing in this setting. We also study the application of our approach to the estimation of regular parameters in a specific parameter class, which are linear functionals of the solutions to the conditional moment restrictions and provide sufficient conditions for achieving root-n consistency using our debiased estimator.

Related papers

• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Existence of unbiased resilient estimators in discrete quantum systems [0.0]
  Bhattacharyya bounds offer a more robust estimation framework with respect to prior accuracy.<n>We show that when the number of constraints exceeds the number of measurement outcomes, an estimator with finite variance typically does not exist.
  arXiv  Detail & Related papers  (2024-02-23T10:12:35Z)
• Nuisance Function Tuning and Sample Splitting for Optimal Doubly Robust Estimation [5.018363990542611]
  We consider the problem of how to estimate nuisance functions to obtain optimal rates of convergence for a doubly robust nonparametric functional. We show that plug-in and first-order biased-corrected estimators can achieve minimax rates of convergence across all H"older smoothness classes of the nuisance functions.
  arXiv  Detail & Related papers  (2022-12-30T18:17:06Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Inference on Strongly Identified Functionals of Weakly Identified Functions [71.42652863687117]
  We study a novel condition for the functional to be strongly identified even when the nuisance function is not. We propose penalized minimax estimators for both the primary and debiasing nuisance functions.
  arXiv  Detail & Related papers  (2022-08-17T13:38:31Z)
• Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals [16.768606469968113]
  We consider a class of doubly robust moment functions originally introduced in (Robins et al., 2008) We demonstrate that this moment function can be used to construct estimating equations for the nuisance functions. The convergence rates of the nuisance functions are analyzed using the modern techniques in statistical learning theory.
  arXiv  Detail & Related papers  (2021-04-07T05:52:15Z)
• Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach [84.29777236590674]
  We study the estimation of causal parameters when not all confounders are observed and instead negative controls are available. Recent work has shown how these can enable identification and efficient estimation via two so-called bridge functions.
  arXiv  Detail & Related papers  (2021-03-25T17:59:19Z)
• Adaptive Sequential Design for a Single Time-Series [2.578242050187029]
  We learn an optimal, unknown choice of the controlled components of a design in order to optimize the expected outcome. We adapt the randomization mechanism for future time-point experiments based on the data collected on the individual over time.
  arXiv  Detail & Related papers  (2021-01-29T22:51:45Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)
• Orthogonal Statistical Learning [49.55515683387805]
  We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk depends on an unknown nuisance parameter. We show that if the population risk satisfies a condition called Neymanity, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order.
  arXiv  Detail & Related papers  (2019-01-25T02:21:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.