A Convenient Infinite Dimensional Framework for Generative Adversarial
Learning
- URL: http://arxiv.org/abs/2011.12087v3
- Date: Fri, 3 Dec 2021 15:23:48 GMT
- Title: A Convenient Infinite Dimensional Framework for Generative Adversarial
Learning
- Authors: Hayk Asatryan, Hanno Gottschalk, Marieke Lippert and Matthias Rottmann
- Abstract summary: We propose an infinite dimensional theoretical framework for generative adversarial learning.
In our framework the Jensen-Shannon divergence between the distribution induced by the generator from the adversarial learning procedure and the data generating distribution converges to zero.
- Score: 4.396860522241306
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, generative adversarial networks (GANs) have demonstrated
impressive experimental results while there are only a few works that foster
statistical learning theory for GANs. In this work, we propose an infinite
dimensional theoretical framework for generative adversarial learning. Assuming
the class of uniformly bounded $k$-times $\alpha$-H\"older differentiable and
uniformly positive densities, we show that the Rosenblatt transformation
induces an optimal generator, which is realizable in the hypothesis space of
$\alpha$-H\"older differentiable generators. With a consistent definition of
the hypothesis space of discriminators, we further show that in our framework
the Jensen-Shannon divergence between the distribution induced by the generator
from the adversarial learning procedure and the data generating distribution
converges to zero. Under sufficiently strict regularity assumptions on the
density of the data generating process, we also provide rates of convergence
based on concentration and chaining.
Related papers
- Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers.
We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.
This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
arXiv Detail & Related papers (2024-10-17T16:42:12Z) - Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian
Mixture Models [59.331993845831946]
Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties.
This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
arXiv Detail & Related papers (2024-03-03T23:15:48Z) - Distributed Markov Chain Monte Carlo Sampling based on the Alternating
Direction Method of Multipliers [143.6249073384419]
In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers.
We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art.
In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
arXiv Detail & Related papers (2024-01-29T02:08:40Z) - Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative
Models [49.81937966106691]
We develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models.
In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach.
arXiv Detail & Related papers (2023-06-15T16:30:08Z) - The Representation Jensen-Shannon Divergence [0.0]
Quantifying the difference between probability distributions is crucial in machine learning.
This work proposes the representation Jensen-Shannon divergence (RJSD), a novel measure inspired by the traditional Jensen-Shannon divergence.
Our results demonstrate RJSD's superiority in two-sample testing, distribution shift detection, and unsupervised domain adaptation.
arXiv Detail & Related papers (2023-05-25T19:44:36Z) - Structure-preserving GANs [6.438897276587413]
We introduce structure-preserving GANs as a data-efficient framework for learning distributions.
We show that we can reduce the discriminator space to its projection on the invariant discriminator space.
We contextualize our framework by building symmetry-preserving GANs for distributions with intrinsic group symmetry.
arXiv Detail & Related papers (2022-02-02T16:40:04Z) - $(f,\Gamma)$-Divergences: Interpolating between $f$-Divergences and
Integral Probability Metrics [6.221019624345409]
We develop a framework for constructing information-theoretic divergences that subsume both $f$-divergences and integral probability metrics (IPMs)
We show that they can be expressed as a two-stage mass-redistribution/mass-transport process.
Using statistical learning as an example, we demonstrate their advantage in training generative adversarial networks (GANs) for heavy-tailed, not-absolutely continuous sample distributions.
arXiv Detail & Related papers (2020-11-11T18:17:09Z) - GANs with Variational Entropy Regularizers: Applications in Mitigating
the Mode-Collapse Issue [95.23775347605923]
Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples.
GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution.
We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
arXiv Detail & Related papers (2020-09-24T19:34:37Z) - Beyond $\mathcal{H}$-Divergence: Domain Adaptation Theory With
Jensen-Shannon Divergence [21.295136514836788]
We reveal the incoherence between the widely-adopted empirical domain adversarial training and its generally-assumed theoretical counterpart based on $mathcalH$-divergence.
We establish a new theoretical framework by directly proving the upper and lower target risk bounds based on joint distributional Jensen-Shannon divergence.
arXiv Detail & Related papers (2020-07-30T16:19:59Z) - Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling
by Exploring Energy of the Discriminator [85.68825725223873]
Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data.
We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator.
We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
arXiv Detail & Related papers (2020-04-05T01:50:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.