Bidirectional Generative Modeling Using Adversarial Gradient Estimation

• URL: http://arxiv.org/abs/2002.09161v3
• Date: Tue, 30 Jun 2020 03:59:02 GMT
• Title: Bidirectional Generative Modeling Using Adversarial Gradient Estimation
• Authors: Xinwei Shen, Tong Zhang, Kani Chen
• Abstract summary: We show that different divergences induce similar algorithms in terms of gradient evaluation. This paper gives a general recipe for a class of principled $f$-divergence based generative modeling methods.
• Score: 15.270525239234072
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper considers the general $f$-divergence formulation of bidirectional generative modeling, which includes VAE and BiGAN as special cases. We present a new optimization method for this formulation, where the gradient is computed using an adversarially learned discriminator. In our framework, we show that different divergences induce similar algorithms in terms of gradient evaluation, except with different scaling. Therefore this paper gives a general recipe for a class of principled $f$-divergence based generative modeling methods. Theoretical justifications and extensive empirical studies are provided to demonstrate the advantage of our approach over existing methods.

Related papers

• Revisiting Extragradient-Type Methods -- Part 1: Generalizations and Sublinear Convergence Rates [6.78476672849813]
  This paper presents a comprehensive analysis of the well-known extragradient (EG) method for solving both equations and inclusions. We analyze both sublinear best-iterate'' and last-iterate'' convergence rates for the entire class of algorithms. We extend our EG framework above to monotone'' inclusions, introducing a new class of algorithms and its corresponding convergence results.
  arXiv  Detail & Related papers  (2024-09-25T12:14:05Z)
• A Unified Theory of Stochastic Proximal Point Methods without Smoothness [52.30944052987393]
  Proximal point methods have attracted considerable interest owing to their numerical stability and robustness against imperfect tuning. This paper presents a comprehensive analysis of a broad range of variations of the proximal point method (SPPM)
  arXiv  Detail & Related papers  (2024-05-24T21:09:19Z)
• Generalizing Backpropagation for Gradient-Based Interpretability [103.2998254573497]
  We show that the gradient of a model is a special case of a more general formulation using semirings. This observation allows us to generalize the backpropagation algorithm to efficiently compute other interpretable statistics.
  arXiv  Detail & Related papers  (2023-07-06T15:19:53Z)
• Optimal Discriminant Analysis in High-Dimensional Latent Factor Models [1.4213973379473654]
  In high-dimensional classification problems, a commonly used approach is to first project the high-dimensional features into a lower dimensional space. We formulate a latent-variable model with a hidden low-dimensional structure to justify this two-step procedure. We propose a computationally efficient classifier that takes certain principal components (PCs) of the observed features as projections.
  arXiv  Detail & Related papers  (2022-10-23T21:45:53Z)
• Efficiently Disentangle Causal Representations [37.1087310583588]
  We approximate the difference with models' generalization abilities so that it fits in the standard machine learning framework. In contrast to the state-of-the-art approach, which relies on the learner's adaptation speed to new distribution, the proposed approach only requires evaluating the model's generalization ability.
  arXiv  Detail & Related papers  (2022-01-06T07:12:36Z)
• Learning Gaussian Graphical Models with Latent Confounders [74.72998362041088]
  We compare and contrast two strategies for inference in graphical models with latent confounders. While these two approaches have similar goals, they are motivated by different assumptions about confounding. We propose a new method, which combines the strengths of these two approaches.
  arXiv  Detail & Related papers  (2021-05-14T00:53:03Z)
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology [66.06153115971732]
  We present a method for quantifying disentanglement that only uses the generative model. We empirically evaluate several state-of-the-art models across multiple datasets.
  arXiv  Detail & Related papers  (2020-06-05T20:54:11Z)
• There and Back Again: Revisiting Backpropagation Saliency Methods [87.40330595283969]
  Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample. A popular class of such methods is based on backpropagating a signal and analyzing the resulting gradient. We propose a single framework under which several such methods can be unified.
  arXiv  Detail & Related papers  (2020-04-06T17:58:08Z)
• Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs [71.26657499537366]
  We propose a simple literature-based method for the efficient approximation of gradients in neural ODE models. We compare it with the reverse dynamic method to train neural ODEs on classification, density estimation, and inference approximation tasks.
  arXiv  Detail & Related papers  (2020-03-11T13:15:57Z)
• Differential Similarity in Higher Dimensional Spaces: Theory and Applications [0.0]
  We develop an algorithm for clustering and coding that combines a geometric model with a probabilistic model in a principled way. We evaluate the solution strategies and the estimation techniques by applying them to two familiar real-world examples.
  arXiv  Detail & Related papers  (2019-02-10T20:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.