Adversarial Mutual Information for Text Generation

• URL: http://arxiv.org/abs/2007.00067v1
• Date: Tue, 30 Jun 2020 19:11:51 GMT
• Title: Adversarial Mutual Information for Text Generation
• Authors: Boyuan Pan, Yazheng Yang, Kaizhao Liang, Bhavya Kailkhura, Zhongming Jin, Xian-Sheng Hua, Deng Cai, Bo Li
• Abstract summary: We propose Adversarial Mutual Information (AMI): a text generation framework. AMI is formed as a novel saddle point (min-max) optimization aiming to identify joint interactions between the source and target. We show that AMI has potential to lead to a tighter lower bound of maximum mutual information.
• Score: 62.974883143784616
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in maximizing mutual information (MI) between the source and target have demonstrated its effectiveness in text generation. However, previous works paid little attention to modeling the backward network of MI (i.e., dependency from the target to the source), which is crucial to the tightness of the variational information maximization lower bound. In this paper, we propose Adversarial Mutual Information (AMI): a text generation framework which is formed as a novel saddle point (min-max) optimization aiming to identify joint interactions between the source and target. Within this framework, the forward and backward networks are able to iteratively promote or demote each other's generated instances by comparing the real and synthetic data distributions. We also develop a latent noise sampling strategy that leverages random variations at the high-level semantic space to enhance the long term dependency in the generation process. Extensive experiments based on different text generation tasks demonstrate that the proposed AMI framework can significantly outperform several strong baselines, and we also show that AMI has potential to lead to a tighter lower bound of maximum mutual information for the variational information maximization problem.

Related papers

• Graph-Augmented Relation Extraction Model with LLMs-Generated Support Document [7.0421339410165045]
  This study introduces a novel approach to sentence-level relation extraction (RE) It integrates Graph Neural Networks (GNNs) with Large Language Models (LLMs) to generate contextually enriched support documents. Our experiments, conducted on the CrossRE dataset, demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-30T20:48:34Z)
• Generative Example-Based Explanations: Bridging the Gap between Generative Modeling and Explainability [42.50219822975012]
  We propose a novel probabilistic framework for local example-based explanations. Our aim is to facilitate communication, foster rigor and transparency, and improve the quality of peer discussion and research progress.
  arXiv  Detail & Related papers  (2024-10-28T10:18:07Z)
• A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks [43.98557963966335]
  Model Inversion (MI) attacks aim to reconstruct privacy-sensitive training data from released models by utilizing output information. Recent advances in generative adversarial networks (GANs) have contributed significantly to the improved performance of MI attacks. We propose a novel method, Intermediate Features enhanced Generative Model Inversion (IF-GMI), which disassembles the GAN structure and exploits features between intermediate blocks.
  arXiv  Detail & Related papers  (2024-07-18T19:16:22Z)
• IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition [0.19791587637442667]
  Cross modality transfer approaches convert existing datasets from a source modality, such as video, to a target modality (IMU) We introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%.
  arXiv  Detail & Related papers  (2024-02-01T22:37:33Z)
• Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis [89.04041100520881]
  This research proposes to retrieve textual and visual evidence based on the object, sentence, and whole image. We develop a novel approach to synthesize the object-level, image-level, and sentence-level information for better reasoning between the same and different modalities.
  arXiv  Detail & Related papers  (2023-05-25T15:26:13Z)
• Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning [66.83982926437547]
  We argue that mitigating the long-tailed distribution problem is crucial for DocRE in the real-world scenario. Motivated by the long-tailed distribution problem, we propose an Easy Relation Augmentation(ERA) method for improving DocRE.
  arXiv  Detail & Related papers  (2022-05-21T06:15:11Z)
• Cross-Modal Generative Augmentation for Visual Question Answering [34.9601948665926]
  This paper introduces a generative model for data augmentation by leveraging the correlations among multiple modalities. The proposed model is able to quantify the confidence of augmented data by its generative probability, and can be jointly updated with a downstream pipeline.
  arXiv  Detail & Related papers  (2021-05-11T04:51:26Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding [67.61357003974153]
  We propose a novel data augmentation framework dubbed CoDA. CoDA synthesizes diverse and informative augmented examples by integrating multiple transformations organically. A contrastive regularization objective is introduced to capture the global relationship among all the data samples.
  arXiv  Detail & Related papers  (2020-10-16T23:57:03Z)
• Enhancing Dialogue Generation via Multi-Level Contrastive Learning [57.005432249952406]
  We propose a multi-level contrastive learning paradigm to model the fine-grained quality of the responses with respect to the query. A Rank-aware (RC) network is designed to construct the multi-level contrastive optimization objectives. We build a Knowledge Inference (KI) component to capture the keyword knowledge from the reference during training and exploit such information to encourage the generation of informative words.
  arXiv  Detail & Related papers  (2020-09-19T02:41:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.