SparseGAN: Sparse Generative Adversarial Network for Text Generation

• URL: http://arxiv.org/abs/2103.11578v2
• Date: Mon, 24 Jul 2023 06:53:10 GMT
• Title: SparseGAN: Sparse Generative Adversarial Network for Text Generation
• Authors: Liping Yuan, Jiehang Zeng, Xiaoqing Zheng
• Abstract summary: We propose a SparseGAN that generates semantic-interpretable, but sparse sentence representations as inputs to the discriminator. With such semantic-rich representations, we not only reduce unnecessary noises for efficient adversarial training, but also make the entire training process fully differentiable.
• Score: 8.634962333084724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is still a challenging task to learn a neural text generation model under the framework of generative adversarial networks (GANs) since the entire training process is not differentiable. The existing training strategies either suffer from unreliable gradient estimations or imprecise sentence representations. Inspired by the principle of sparse coding, we propose a SparseGAN that generates semantic-interpretable, but sparse sentence representations as inputs to the discriminator. The key idea is that we treat an embedding matrix as an over-complete dictionary, and use a linear combination of very few selected word embeddings to approximate the output feature representation of the generator at each time step. With such semantic-rich representations, we not only reduce unnecessary noises for efficient adversarial training, but also make the entire training process fully differentiable. Experiments on multiple text generation datasets yield performance improvements, especially in sequence-level metrics, such as BLEU.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.