GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes

• URL: http://arxiv.org/abs/2509.22953v1
• Date: Fri, 26 Sep 2025 21:35:28 GMT
• Title: GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes
• Authors: Valentyn Melnychuk, Stefan Feuerriegel,
• Abstract summary: We introduce a general suite of generative Neyman-orthogonal learners that estimate conditional distributions of potential outcomes.<n>Our proposed GDR-learners are flexible and can be instantiated with many state-of-the-art deep generative models.<n>Unlike the existing methods, our GDR-learners possess the properties of quasi-oracle efficiency and double robustness.
• Score: 50.228749840286895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various deep generative models have been proposed to estimate potential outcomes distributions from observational data. However, none of them have the favorable theoretical property of general Neyman-orthogonality and, associated with it, quasi-oracle efficiency and double robustness. In this paper, we introduce a general suite of generative Neyman-orthogonal (doubly-robust) learners that estimate the conditional distributions of potential outcomes. Our proposed GDR-learners are flexible and can be instantiated with many state-of-the-art deep generative models. In particular, we develop GDR-learners based on (a) conditional normalizing flows (which we call GDR-CNFs), (b) conditional generative adversarial networks (GDR-CGANs), (c) conditional variational autoencoders (GDR-CVAEs), and (d) conditional diffusion models (GDR-CDMs). Unlike the existing methods, our GDR-learners possess the properties of quasi-oracle efficiency and rate double robustness, and are thus asymptotically optimal. In a series of (semi-)synthetic experiments, we demonstrate that our GDR-learners are very effective and outperform the existing methods in estimating the conditional distributions of potential outcomes.

Related papers

• Distributional Evaluation of Generative Models via Relative Density Ratio [12.663086000741872]
  We propose a functional evaluation metric for generative models based on the relative density ratio (RDR)<n>We show that the RDR as a functional summary of the goodness-of-fit for the generative model, possesses several desirable theoretical properties.<n>We show that the estimated RDR not only allows for an effective comparison of the overall performance of competing generative models, but it can also offer a convenient means of revealing the nature of the underlying goodness-of-fit.
  arXiv  Detail & Related papers  (2025-10-29T13:31:35Z)
• Does Generative Retrieval Overcome the Limitations of Dense Retrieval? [117.49076770892583]
  Generative retrieval (GR) has emerged as a new paradigm in neural information retrieval.<n>We investigate how GR fundamentally diverges from dense retrieval (DR) in both learning objectives and representational capacity.
  arXiv  Detail & Related papers  (2025-09-26T09:38:01Z)
• Brain-like variational inference [0.0]
  Inference in both brains and machines can be formalized by maximizing the evidence lower bound (ELBO) in machine learning, or minimizing variational free energy (F) in neuroscience (ELBO = -F)<n>Here, we show that online natural gradient descent on F, under Poisson assumptions, leads to a recurrent spiking neural network that performs variational inference via membrane potential dynamics.<n>The resulting model -- the iterative Poisson variational autoencoder (iP-VAE) -- replaces the encoder network with local updates derived from natural gradient descent on F.
  arXiv  Detail & Related papers  (2024-10-25T06:00:18Z)
• A Non-negative VAE:the Generalized Gamma Belief Network [49.970917207211556]
  The gamma belief network (GBN) has demonstrated its potential for uncovering multi-layer interpretable latent representations in text data. We introduce the generalized gamma belief network (Generalized GBN) in this paper, which extends the original linear generative model to a more expressive non-linear generative model. We also propose an upward-downward Weibull inference network to approximate the posterior distribution of the latent variables.
  arXiv  Detail & Related papers  (2024-08-06T18:18:37Z)
• Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models [32.52492468276371]
  We propose regularized deep generative model (Reg-DGM) to reduce the variance of generative modeling with limited data. Reg-DGM uses a pre-trained model to optimize a weighted sum of a certain divergence and the expectation of an energy function. Empirically, with various pre-trained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data.
  arXiv  Detail & Related papers  (2022-08-30T10:28:50Z)
• BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery [97.79015388276483]
  A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG) Recent advances enabled effective maximum-likelihood point estimation of DAGs from observational data. We propose BCD Nets, a variational framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM.
  arXiv  Detail & Related papers  (2021-12-06T03:35:21Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Analytical Probability Distributions and EM-Learning for Deep Generative Networks [29.319553019103868]
  Deep Generative Networks (DGNs) with probabilistic modeling of their output and latent space are currently trained via Variational Autoencoders (VAEs) In the absence of a known analytical form for the posterior and likelihood expectation, VAEs resort to approximations, including (Amortized) Variational Inference (AVI) and Monte-Carlo sampling. We exploit the Continuous Piecewise Affine (CPA) property of modern DGNs to derive their posterior and marginal distributions. We demonstrate empirically that EM training of DGNs produces greater likelihood than VAE training.
  arXiv  Detail & Related papers  (2020-06-17T17:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.