Training Generative Adversarial Networks via stochastic Nash games

• URL: http://arxiv.org/abs/2010.10013v3
• Date: Fri, 21 May 2021 10:20:32 GMT
• Title: Training Generative Adversarial Networks via stochastic Nash games
• Authors: Barbara Franci, Sergio Grammatico
• Abstract summary: Generative adversarial networks (GANs) are a class of generative models with two antagonistic neural networks: a generator and a discriminator. We show convergence to an exact solution when an increasing number of data is available. We also show convergence of an averaged variant of the SRFB algorithm to a neighborhood of the solution when only few samples are available.
• Score: 2.995087247817663
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative adversarial networks (GANs) are a class of generative models with two antagonistic neural networks: a generator and a discriminator. These two neural networks compete against each other through an adversarial process that can be modeled as a stochastic Nash equilibrium problem. Since the associated training process is challenging, it is fundamental to design reliable algorithms to compute an equilibrium. In this paper, we propose a stochastic relaxed forward-backward (SRFB) algorithm for GANs and we show convergence to an exact solution when an increasing number of data is available. We also show convergence of an averaged variant of the SRFB algorithm to a neighborhood of the solution when only few samples are available. In both cases, convergence is guaranteed when the pseudogradient mapping of the game is monotone. This assumption is among the weakest known in the literature. Moreover, we apply our algorithm to the image generation problem.

Related papers

• Computing Systemic Risk Measures with Graph Neural Networks [1.6874375111244329]
  This paper investigates systemic risk measures for financial networks of explicitly modelled bilateral liabilities. We study numerical methods for the approximation of systemic risk and optimal random allocations.
  arXiv  Detail & Related papers  (2024-09-30T10:18:13Z)
• LinSATNet: The Positive Linear Satisfiability Neural Networks [116.65291739666303]
  This paper studies how to introduce the popular positive linear satisfiability to neural networks. We propose the first differentiable satisfiability layer based on an extension of the classic Sinkhorn algorithm for jointly encoding multiple sets of marginal distributions.
  arXiv  Detail & Related papers  (2024-07-18T22:05:21Z)
• Stochastic Unrolled Federated Learning [85.6993263983062]
  We introduce UnRolled Federated learning (SURF), a method that expands algorithm unrolling to federated learning. Our proposed method tackles two challenges of this expansion, namely the need to feed whole datasets to the unrolleds and the decentralized nature of federated learning.
  arXiv  Detail & Related papers  (2023-05-24T17:26:22Z)
• Generative Adversarial Learning of Sinkhorn Algorithm Initializations [0.0]
  We show that meticulously training a neural network to learn initializations to the algorithm via the entropic OT dual problem can significantly speed up convergence. We show that our network can even be used as a standalone OT solver to approximate regularized transport distances to a few percent error.
  arXiv  Detail & Related papers  (2022-11-30T21:56:09Z)
• Conjugate Gradient Method for Generative Adversarial Networks [0.0]
  It is not feasible to calculate the Jensen-Shannon divergence of the density function of the data and the density function of the model of deep neural networks. Generative adversarial networks (GANs) can be used to formulate this problem as a discriminative problem with two models, a generator and a discriminator. We propose to apply the conjugate gradient method to solve the local Nash equilibrium problem in GANs.
  arXiv  Detail & Related papers  (2022-03-28T04:44:45Z)
• The Separation Capacity of Random Neural Networks [78.25060223808936]
  We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
  arXiv  Detail & Related papers  (2021-07-31T10:25:26Z)
• On the Reproducibility of Neural Network Predictions [52.47827424679645]
  We study the problem of churn, identify factors that cause it, and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classification tasks. We propose using emphminimum entropy regularizers to increase prediction confidences. We present empirical results showing the effectiveness of both techniques in reducing churn while improving the accuracy of the underlying model.
  arXiv  Detail & Related papers  (2021-02-05T18:51:01Z)
• Neural Thompson Sampling [94.82847209157494]
  We propose a new algorithm, called Neural Thompson Sampling, which adapts deep neural networks for both exploration and exploitation. At the core of our algorithm is a novel posterior distribution of the reward, where its mean is the neural network approximator, and its variance is built upon the neural tangent features of the corresponding neural network.
  arXiv  Detail & Related papers  (2020-10-02T07:44:09Z)
• A game-theoretic approach for Generative Adversarial Networks [2.995087247817663]
  Generative adversarial networks (GANs) are a class of generative models, known for producing accurate samples. Main bottleneck for their implementation is that the neural networks are very hard to train. We propose a relaxed forward-backward algorithm for GANs. We prove that when the pseudogradient mapping of the game is monotone, we have convergence to an exact solution or in a neighbourhood of it.
  arXiv  Detail & Related papers  (2020-03-30T17:14:41Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.