Your Autoregressive Generative Model Can be Better If You Treat It as an
Energy-Based One
- URL: http://arxiv.org/abs/2206.12840v1
- Date: Sun, 26 Jun 2022 10:58:41 GMT
- Title: Your Autoregressive Generative Model Can be Better If You Treat It as an
Energy-Based One
- Authors: Yezhen Wang, Tong Che, Bo Li, Kaitao Song, Hengzhi Pei, Yoshua Bengio,
Dongsheng Li
- Abstract summary: We propose a unique method termed E-ARM for training autoregressive generative models.
E-ARM takes advantage of a well-designed energy-based learning objective.
We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
- Score: 83.5162421521224
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Autoregressive generative models are commonly used, especially for those
tasks involving sequential data. They have, however, been plagued by a slew of
inherent flaws due to the intrinsic characteristics of chain-style conditional
modeling (e.g., exposure bias or lack of long-range coherence), severely
limiting their ability to model distributions properly. In this paper, we
propose a unique method termed E-ARM for training autoregressive generative
models that takes advantage of a well-designed energy-based learning objective.
By leveraging the extra degree of freedom of the softmax operation, we are
allowed to make the autoregressive model itself be an energy-based model for
measuring the likelihood of input without introducing any extra parameters.
Furthermore, we show that E-ARM can be trained efficiently and is capable of
alleviating the exposure bias problem and increase temporal coherence for
autoregressive generative models. Extensive empirical results, covering
benchmarks like language modeling, neural machine translation, and image
generation, demonstrate the effectiveness of the proposed approach.
Related papers
- Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL.
We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
arXiv Detail & Related papers (2024-05-30T03:57:29Z) - Generative Marginalization Models [24.694121731706314]
marginalization models (MaMs) are a new family of generative models for high-dimensional discrete data.
They offer scalable and flexible generative modeling with tractable likelihoods.
For energy-based training tasks, MaMs enable any-order generative modeling of high-dimensional problems.
arXiv Detail & Related papers (2023-10-19T17:14:29Z) - Exploring Model Transferability through the Lens of Potential Energy [78.60851825944212]
Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models.
Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels.
We present an insightful physics-inspired approach named PED to address these challenges.
arXiv Detail & Related papers (2023-08-29T07:15:57Z) - RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment [32.752633250862694]
Generative foundation models are susceptible to implicit biases that can arise from extensive unsupervised training data.
We introduce a new framework, Reward rAnked FineTuning, designed to align generative models effectively.
arXiv Detail & Related papers (2023-04-13T18:22:40Z) - Controllable and Compositional Generation with Latent-Space Energy-Based
Models [60.87740144816278]
Controllable generation is one of the key requirements for successful adoption of deep generative models in real-world applications.
In this work, we use energy-based models (EBMs) to handle compositional generation over a set of attributes.
By composing energy functions with logical operators, this work is the first to achieve such compositionality in generating photo-realistic images of resolution 1024x1024.
arXiv Detail & Related papers (2021-10-21T03:31:45Z) - Autoregressive Dynamics Models for Offline Policy Evaluation and
Optimization [60.73540999409032]
We show that expressive autoregressive dynamics models generate different dimensions of the next state and reward sequentially conditioned on previous dimensions.
We also show that autoregressive dynamics models are useful for offline policy optimization by serving as a way to enrich the replay buffer.
arXiv Detail & Related papers (2021-04-28T16:48:44Z) - Dynamic Model Pruning with Feedback [64.019079257231]
We propose a novel model compression method that generates a sparse trained model without additional overhead.
We evaluate our method on CIFAR-10 and ImageNet, and show that the obtained sparse models can reach the state-of-the-art performance of dense models.
arXiv Detail & Related papers (2020-06-12T15:07:08Z) - Maximum Entropy Model Rollouts: Fast Model Based Policy Optimization
without Compounding Errors [10.906666680425754]
We propose a Dyna-style model-based reinforcement learning algorithm, which we called Maximum Entropy Model Rollouts (MEMR)
To eliminate the compounding errors, we only use our model to generate single-step rollouts.
arXiv Detail & Related papers (2020-06-08T21:38:15Z) - DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications [0.0]
One of the limitations of deep learning models with sparse features today stems from the predefined nature of their input.
We show that the resulting models are able to perform better and efficiently run at a much larger scale.
arXiv Detail & Related papers (2020-04-17T17:43:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.