Adversarial Training Improves Joint Energy-Based Generative Modelling

• URL: http://arxiv.org/abs/2207.08950v1
• Date: Mon, 18 Jul 2022 21:30:03 GMT
• Title: Adversarial Training Improves Joint Energy-Based Generative Modelling
• Authors: Rostislav Korst, Arip Asadulaev
• Abstract summary: We propose the novel framework for generative modelling using hybrid energy-based models. In our method we combine the interpretable input gradients of the robust classifier and Langevin Dynamics for sampling.
• Score: 1.0878040851638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose the novel framework for generative modelling using hybrid energy-based models. In our method we combine the interpretable input gradients of the robust classifier and Langevin Dynamics for sampling. Using the adversarial training we improve not only the training stability, but robustness and generative modelling of the joint energy-based models.

Related papers

• Structured Generative Modeling with the Thermodynamic Kolmogorov-Arnold Model [0.0]
  We propose a novel adaptation of the Kolmogorov-Arnold representation theorem for generative modeling.<n>We introduce the Thermodynamic Kolmogorov-Arnold Model (T-KAM) as incorporating a new framework for structural and inductive biases.<n>T-KAM provides an elegant balance among common trade-offs in generative modeling, offering fast inference, high sample quality, and stable training.
  arXiv  Detail & Related papers  (2025-06-17T04:07:32Z)
• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• MGE: A Training-Free and Efficient Model Generation and Enhancement Scheme [10.48591131837771]
  This paper proposes a Training-Free and Efficient Model Generation and Enhancement Scheme (MGE) It considers two aspects during the model generation process: the distribution of model parameters and model performance. Experiments result shows that generated models are comparable to models obtained through normal training, and even superior in some cases.
  arXiv  Detail & Related papers  (2024-02-27T13:12:00Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• Latent Diffusion Energy-Based Model for Interpretable Text Modeling [104.85356157724372]
  We introduce a novel symbiosis between the diffusion models and latent space EBMs in a variational learning framework. We develop a geometric clustering-based regularization jointly with the information bottleneck to further improve the quality of the learned latent space.
  arXiv  Detail & Related papers  (2022-06-13T03:41:31Z)
• A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training [64.71254710803368]
  Adversarial Training (AT) is an effective approach to enhance the robustness of deep neural networks. We demystify this phenomenon by developing a unified probabilistic framework, called Contrastive Energy-based Models (CEM) We propose a principled method to develop adversarial learning and sampling methods.
  arXiv  Detail & Related papers  (2022-03-25T05:33:34Z)
• Integrating Physics-Based Modeling with Machine Learning for Lithium-Ion Batteries [4.946066838162504]
  This paper proposes two new frameworks to integrate physics-based models with machine learning to achieve high-precision modeling for LiBs. The frameworks are characterized by informing the machine learning model of the state information of the physical model. The study further expands to conduct aging-aware hybrid modeling, leading to the design of a hybrid model conscious of the state-of-health to make prediction.
  arXiv  Detail & Related papers  (2021-12-24T07:39:02Z)
• Hybrid modeling of the human cardiovascular system using NeuralFMUs [0.0]
  We show that the hybrid modeling process is more comfortable, needs less system knowledge and is less error-prone compared to modeling solely based on first principle. The resulting hybrid model has improved in computation performance, compared to a pure first principle white-box model. The considered use-case can serve as example for other modeling and simulation applications in and beyond the medical domain.
  arXiv  Detail & Related papers  (2021-09-10T13:48:43Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)
• No MCMC for me: Amortized sampling for fast and stable training of energy-based models [62.1234885852552]
  Energy-Based Models (EBMs) present a flexible and appealing way to represent uncertainty. We present a simple method for training EBMs at scale using an entropy-regularized generator to amortize the MCMC sampling. Next, we apply our estimator to the recently proposed Joint Energy Model (JEM), where we match the original performance with faster and stable training.
  arXiv  Detail & Related papers  (2020-10-08T19:17:20Z)
• Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy [6.767885381740952]
  Our method constructs an energy function on the latent variable space that yields an energy function on samples produced by the pre-trained generative model. We show that using our proposed method, we can greatly improve the sample quality of popular likelihood based generative models.
  arXiv  Detail & Related papers  (2020-06-15T02:58:43Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.