Normalizing Flows for Interventional Density Estimation

• URL: http://arxiv.org/abs/2209.06203v5
• Date: Tue, 20 Jun 2023 12:24:53 GMT
• Title: Normalizing Flows for Interventional Density Estimation
• Authors: Valentyn Melnychuk, Dennis Frauen, Stefan Feuerriegel
• Abstract summary: We propose a novel, fully-parametric deep learning method called Interventional Normalizing Flows. We combine two normalizing flows, namely (i) a nuisance flow for estimating nuisance parameters and (ii) a target flow for parametric estimation of the density of potential outcomes.
• Score: 18.640006398066188
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing machine learning methods for causal inference usually estimate quantities expressed via the mean of potential outcomes (e.g., average treatment effect). However, such quantities do not capture the full information about the distribution of potential outcomes. In this work, we estimate the density of potential outcomes after interventions from observational data. For this, we propose a novel, fully-parametric deep learning method called Interventional Normalizing Flows. Specifically, we combine two normalizing flows, namely (i) a nuisance flow for estimating nuisance parameters and (ii) a target flow for parametric estimation of the density of potential outcomes. We further develop a tractable optimization objective based on a one-step bias correction for efficient and doubly robust estimation of the target flow parameters. As a result, our Interventional Normalizing Flows offer a properly normalized density estimator. Across various experiments, we demonstrate that our Interventional Normalizing Flows are expressive and highly effective, and scale well with both sample size and high-dimensional confounding. To the best of our knowledge, our Interventional Normalizing Flows are the first proper fully-parametric, deep learning method for density estimation of potential outcomes.

Related papers

• Deep conditional distribution learning via conditional Föllmer flow [3.227277661633986]
  We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional F"ollmer Flow. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network.
  arXiv  Detail & Related papers  (2024-02-02T14:52:10Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Learned harmonic mean estimation of the marginal likelihood with normalizing flows [6.219412541001482]
  We introduce the use of normalizing flows to represent the importance sampling target distribution. The code implementing the learned harmonic mean, which is publicly available, has been updated to now support normalizing flows.
  arXiv  Detail & Related papers  (2023-06-30T18:00:02Z)
• Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning [5.346298077607419]
  We propose density ratio estimation-based Bayesian optimization with semi-supervised learning to solve this challenge. We show the empirical results of our methods and several baseline methods in two distinct scenarios with unlabeled point sampling and a fixed-size pool.
  arXiv  Detail & Related papers  (2023-05-24T23:01:56Z)
• Joint Manifold Learning and Density Estimation Using Normalizing Flows [4.939777212813711]
  We introduce two approaches, namely per-pixel penalized log-likelihood and hierarchical training, to answer the question. We propose a single-step method for joint manifold learning and density estimation by disentangling the transformed space. Results validate the superiority of the proposed methods in simultaneous manifold learning and density estimation.
  arXiv  Detail & Related papers  (2022-06-07T13:35:14Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)
• Continuous Wasserstein-2 Barycenter Estimation without Minimax Optimization [94.18714844247766]
  Wasserstein barycenters provide a geometric notion of the weighted average of probability measures based on optimal transport. We present a scalable algorithm to compute Wasserstein-2 barycenters given sample access to the input measures.
  arXiv  Detail & Related papers  (2021-02-02T21:01:13Z)
• Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation [99.92568326314667]
  We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation. Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle. We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-11-05T08:04:34Z)
• Neural Methods for Point-wise Dependency Estimation [129.93860669802046]
  We focus on estimating point-wise dependency (PD), which quantitatively measures how likely two outcomes co-occur. We demonstrate the effectiveness of our approaches in 1) MI estimation, 2) self-supervised representation learning, and 3) cross-modal retrieval task.
  arXiv  Detail & Related papers  (2020-06-09T23:26:15Z)
• On Deep Instrumental Variables Estimate [21.003925102068298]
  "Deep Instrumental Variable (IV)" is a framework based on deep neural networks to address endogeneity. We consider a two-stage estimator using deep neural networks in the linear instrumental variables model. We show that the second-stage estimator achieves the semiparametric efficiency bound.
  arXiv  Detail & Related papers  (2020-04-30T17:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.