Double Machine Learning for Conditional Moment Restrictions: IV Regression, Proximal Causal Learning and Beyond

• URL: http://arxiv.org/abs/2506.14950v2
• Date: Mon, 23 Jun 2025 18:27:16 GMT
• Title: Double Machine Learning for Conditional Moment Restrictions: IV Regression, Proximal Causal Learning and Beyond
• Authors: Daqian Shao, Ashkan Soleymani, Francesco Quinzan, Marta Kwiatkowska,
• Abstract summary: Conditional moment restrictions (CMRs) are a key problem considered in statistics, causal inference, and econometrics.<n>Most CMR estimators use a two-stage approach, where the first-stage estimation is directly plugged into the second stage to estimate the function of interest.<n>We propose DML-CMR, a two-stage CMR estimator that provides an unbiased estimate with fast convergence rate guarantees.
• Score: 16.842233444365764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving conditional moment restrictions (CMRs) is a key problem considered in statistics, causal inference, and econometrics, where the aim is to solve for a function of interest that satisfies some conditional moment equalities. Specifically, many techniques for causal inference, such as instrumental variable (IV) regression and proximal causal learning (PCL), are CMR problems. Most CMR estimators use a two-stage approach, where the first-stage estimation is directly plugged into the second stage to estimate the function of interest. However, naively plugging in the first-stage estimator can cause heavy bias in the second stage. This is particularly the case for recently proposed CMR estimators that use deep neural network (DNN) estimators for both stages, where regularisation and overfitting bias is present. We propose DML-CMR, a two-stage CMR estimator that provides an unbiased estimate with fast convergence rate guarantees. We derive a novel learning objective to reduce bias and develop the DML-CMR algorithm following the double/debiased machine learning (DML) framework. We show that our DML-CMR estimator can achieve the minimax optimal convergence rate of $O(N^{-1/2})$ under parameterisation and mild regularity conditions, where $N$ is the sample size. We apply DML-CMR to a range of problems using DNN estimators, including IV regression and proximal causal learning on real-world datasets, demonstrating state-of-the-art performance against existing CMR estimators and algorithms tailored to those problems.

Related papers

• Automatic Double Reinforcement Learning in Semiparametric Markov Decision Processes with Applications to Long-Term Causal Inference [33.14076284663493]
  Markov Decision Processes (MDPs) offer a principled framework for modeling outcomes as sequences of states, actions, and rewards over time.<n>We introduce a semiparametric extension of Double Reinforcement Learning (DRL) for statistically efficient, model-robust inference on linear functionals of the Q-function.<n>We develop a novel debiased plug-in estimator based on isotonic Bellman calibration, which integrates fitted Q-it with an isotonic regression step.
  arXiv  Detail & Related papers  (2025-01-12T20:35:28Z)
• Automatic debiasing of neural networks via moment-constrained learning [0.0]
  Naively learning the regression function and taking a sample mean of the target functional results in biased estimators.<n>We propose moment-constrained learning as a new RR learning approach that addresses some shortcomings in automatic debiasing.
  arXiv  Detail & Related papers  (2024-09-29T20:56:54Z)
• Provably Efficient CVaR RL in Low-rank MDPs [58.58570425202862]
  We study risk-sensitive Reinforcement Learning (RL) We propose a novel Upper Confidence Bound (UCB) bonus-driven algorithm to balance interplay between exploration, exploitation, and representation learning in CVaR RL. We prove that our algorithm achieves a sample complexity of $epsilon$-optimal CVaR, where $H$ is the length of each episode, $A$ is the capacity of action space, and $d$ is the dimension of representations.
  arXiv  Detail & Related papers  (2023-11-20T17:44:40Z)
• Multiply Robust Estimator Circumvents Hyperparameter Tuning of Neural Network Models in Causal Inference [0.0]
  Multiply Robust (MR) estimator allows us to leverage all the first-step models in a single estimator. We show that MR is the solution to a broad class of estimating equations, and is also consistent if one of the treatment models is $sqrtn$ consistent.
  arXiv  Detail & Related papers  (2023-07-20T02:31:12Z)
• Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes [80.89852729380425]
  We propose the first computationally efficient algorithm that achieves the nearly minimax optimal regret $tilde O(dsqrtH3K)$. Our work provides a complete answer to optimal RL with linear MDPs, and the developed algorithm and theoretical tools may be of independent interest.
  arXiv  Detail & Related papers  (2022-12-12T18:58:59Z)
• Robust Causal Learning for the Estimation of Average Treatment Effects [14.96459402684986]
  We propose a Robust Causal Learning (RCL) method to offset the deficiencies of the Double/Debiased Machine Learning (DML) estimators. Empirically, the comprehensive experiments show that i) the RCL estimators give more stable estimations of the causal parameters than the DML estimators.
  arXiv  Detail & Related papers  (2022-09-05T07:35:58Z)
• On Continual Model Refinement in Out-of-Distribution Data Streams [64.62569873799096]
  Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams. Existing continual learning (CL) problem setups cannot cover such a realistic and complex scenario. We propose a new CL problem formulation dubbed continual model refinement (CMR)
  arXiv  Detail & Related papers  (2022-05-04T11:54:44Z)
• Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning [59.02006924867438]
  Off-policy evaluation and learning (OPE/L) use offline observational data to make better decisions. Recent work proposed distributionally robust OPE/L (DROPE/L) to remedy this, but the proposal relies on inverse-propensity weighting. We propose the first DR algorithms for DROPE/L with KL-divergence uncertainty sets.
  arXiv  Detail & Related papers  (2022-02-19T20:00:44Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Robust Algorithms for GMM Estimation: A Finite Sample Viewpoint [30.839245814393724]
  A generic method of solving moment conditions is the Generalized Method of Moments (GMM) We develop a GMM estimator that can tolerate a constant $ell$ recovery guarantee of $O(sqrtepsilon)$. Our algorithm and assumptions apply to instrumental variables linear and logistic regression.
  arXiv  Detail & Related papers  (2021-10-06T21:06:22Z)
• Enhanced Doubly Robust Learning for Debiasing Post-click Conversion Rate Estimation [29.27760413892272]
  Post-click conversion, as a strong signal indicating the user preference, is salutary for building recommender systems. Currently, most existing methods utilize counterfactual learning to debias recommender systems. We propose a novel double learning approach for the MRDR estimator, which can convert the error imputation into the general CVR estimation.
  arXiv  Detail & Related papers  (2021-05-28T06:59:49Z)
• SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression [68.66245730450915]
  We develop an improved method for debiasing predictions and estimating frequentist uncertainty for practical datasets. Our main contribution is SLOE, an estimator of the signal strength with convergence guarantees that reduces the computation time of estimation and inference by orders of magnitude.
  arXiv  Detail & Related papers  (2021-03-23T17:48:56Z)
• Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond [69.83813153444115]
  We consider an efficient estimating equation for the (local) quantile treatment effect ((L)QTE) in causal inference. Debiased machine learning (DML) is a data-splitting approach to estimating high-dimensional nuisances. We propose localized debiased machine learning (LDML), which avoids this burdensome step.
  arXiv  Detail & Related papers  (2019-12-30T14:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.