Covariate Balancing and Riesz Regression Should Be Guided by the Neyman Orthogonal Score in Debiased Machine Learning

• URL: http://arxiv.org/abs/2605.06386v1
• Date: Thu, 07 May 2026 15:02:47 GMT
• Title: Covariate Balancing and Riesz Regression Should Be Guided by the Neyman Orthogonal Score in Debiased Machine Learning
• Abstract summary: We advocate regressor balancing, implemented by Riesz regression with basis functions of $X$, as the general balancing principle for DML.<n>The position is not that covariate balancing is invalid, but that covariate balancing should be understood as the special case that is appropriate.
• Score: 6.44705221140412
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This position paper argues that, in debiased machine learning, balancing functions should be derived from the Neyman orthogonal score, not chosen only as functions of covariates. Covariate balancing is effective when the regression error entering the score can be represented by functions of covariates alone, and it is the natural finite-dimensional approximation for targets such as ATT counterfactual means. For ATE estimation under treatment effect heterogeneity, however, the score error generally contains treatment-specific components because the outcome regression is a function of the full regressor $X=(D,Z)$. In that case, balancing common functions of $Z$ can leave the treatment-specific component unbalanced. We therefore advocate regressor balancing, implemented by Riesz regression with basis functions of $X$, as the general balancing principle for DML. The position is not that covariate balancing is invalid, but that covariate balancing should be understood as the special case that is appropriate when the score-relevant regression error is a function of covariates alone.

Related papers

• TILT: Target-induced loss tilting under covariate shift [14.526961596109361]
  We introduce and analyze Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation.<n>Based on a novel objective function that decomposes the source predictor as $f+b$, fits $f+b$ on labeled source data while simultaneously penalizing the auxiliary component $b$ on unlabeled target inputs.<n>We show that this target-side penalty implicitly induces relative importance weighting at the population level, but in terms of an estimand $b*_f$ that is self-localized to the current error, and remains uniformly bounded for any source-target pair.
  arXiv  Detail & Related papers  (2026-05-14T02:26:34Z)
• DARTS: Targeting Prognostic Covariates in Budget-Constrained Sequential Experiments [47.61857875238484]
  We introduce Dynamic Adaptive Rerandomization via Thompson Sampling (DARTS)<n>DARTS treats covariate acquisition as a sequential optimization problem embedded within a design-based causal inference task.<n>We derive a Bayes risk bound for the acquisition layer that matches the minimax lower bound up to logarithmic factors.
  arXiv  Detail & Related papers  (2026-05-07T17:27:51Z)
• A Unified Theory for Causal Inference: Direct Debiased Machine Learning via Bregman-Riesz Regression [6.44705221140412]
  This note introduces a unified theory for causal inference that integrates Riesz regression, covariate balancing, density-ratio estimation (DRE), and the matching estimator in average treatment effect (ATE) estimation.<n>In ATE estimation, the balancing weights and the regression functions of the outcome play important roles, where the balancing weights are referred to as the Riesz representer.
  arXiv  Detail & Related papers  (2025-10-30T17:56:47Z)
• Direct Debiased Machine Learning via Bregman Divergence Minimization [6.44705221140412]
  We develop a direct debiased machine learning framework with an end-to-end algorithm.<n>We formulate estimation of the nuisance parameters, the regression function and the Riesz representer.<n>Neyman targeted estimation includes Riesz representer estimation, and we measure discrepancies using the Bregman divergence.
  arXiv  Detail & Related papers  (2025-10-27T17:10:43Z)
• Regression-Based Estimation of Causal Effects in the Presence of Selection Bias and Confounding [52.1068936424622]
  We consider the problem of estimating the expected causal effect $E[Y|do(X)]$ for a target variable $Y$ when treatment $X$ is set by intervention.<n>In settings without selection bias or confounding, $E[Y|do(X)] = E[Y|X]$, which can be estimated using standard regression methods.<n>We propose a framework that incorporates both selection bias and confounding.
  arXiv  Detail & Related papers  (2025-03-26T13:43:37Z)
• TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression [109.69084997173196]
  Deepscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. Recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of the negative log-likelihood.
  arXiv  Detail & Related papers  (2023-10-29T09:54:03Z)
• Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing [11.291393872745951]
  Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. We propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR) VIR performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct.
  arXiv  Detail & Related papers  (2023-06-11T06:27:06Z)
• Mean Parity Fair Regression in RKHS [43.98593032593897]
  We study the fair regression problem under the notion of Mean Parity (MP) fairness. We address this problem by leveraging reproducing kernel Hilbert space (RKHS) We derive a corresponding regression function that can be implemented efficiently and provides interpretable tradeoffs.
  arXiv  Detail & Related papers  (2023-02-21T02:44:50Z)
• Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency [53.90687548731265]
  We study optimal procedures for estimating a linear functional based on observational data. For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
  arXiv  Detail & Related papers  (2023-01-16T02:57:37Z)
• SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression [68.66245730450915]
  We develop an improved method for debiasing predictions and estimating frequentist uncertainty for practical datasets. Our main contribution is SLOE, an estimator of the signal strength with convergence guarantees that reduces the computation time of estimation and inference by orders of magnitude.
  arXiv  Detail & Related papers  (2021-03-23T17:48:56Z)
• Kernel Ridge Riesz Representers: Generalization, Mis-specification, and the Counterfactual Effective Dimension [2.7152798636894193]
  I interpret kernel balancing weights as kernel ridge Riesz representers (KRRR) KRRR is an exact generalization of kernel ridge regression. I use KRRR to quantify uncertainty for heterogeneous treatment effects, by age, of 401(k) eligibility on assets.
  arXiv  Detail & Related papers  (2021-02-22T14:46:23Z)
• Online nonparametric regression with Sobolev kernels [99.12817345416846]
  We derive the regret upper bounds on the classes of Sobolev spaces $W_pbeta(mathcalX)$, $pgeq 2, beta>fracdp$. The upper bounds are supported by the minimax regret analysis, which reveals that in the cases $beta> fracd2$ or $p=infty$ these rates are (essentially) optimal.
  arXiv  Detail & Related papers  (2021-02-06T15:05:14Z)
• Piecewise Linear Regression via a Difference of Convex Functions [50.89452535187813]
  We present a new piecewise linear regression methodology that utilizes fitting a difference of convex functions (DC functions) to the data. We empirically validate the method, showing it to be practically implementable, and to have comparable performance to existing regression/classification methods on real-world datasets.
  arXiv  Detail & Related papers  (2020-07-05T18:58:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.