Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder

• URL: http://arxiv.org/abs/2006.08101v1
• Date: Mon, 15 Jun 2020 02:59:52 GMT
• Title: Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder
• Authors: Daya Guo, Duyu Tang, Nan Duan, Jian Yin, Daxin Jiang and Ming Zhou
• Abstract summary: We propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach provides state-of-the-art performance on both Event2Mind and ATOMIC datasets.
• Score: 104.25716317141321
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context that is not explicitly provided, resulting in a context-independent semantic representation that struggles to support the generation. To address this, we propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach works in an encoder-decoder manner and is equipped with a Vector Quantised-Variational Autoencoder, where the encoder outputs representations from a distribution over discrete variables. Such discrete representations enable automatically selecting relevant evidence, which not only facilitates evidence-aware generation, but also provides a natural way to uncover rationales behind the generation. Our approach provides state-of-the-art performance on both Event2Mind and ATOMIC datasets. More importantly, we find that with discrete representations, our model selectively uses evidence to generate different inferential texts.

Related papers

• IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognition [5.525052547053668]
  Scene text recognition has attracted more and more attention due to its diverse applications. Most state-of-the-art methods adopt an encoder-decoder framework with the attention mechanism, autoregressively generating text from left to right. We propose an alternative solution, using a parallel and iterative decoder that adopts an easy-first decoding strategy.
  arXiv  Detail & Related papers  (2023-12-19T08:03:19Z)
• Language Models As Semantic Indexers [78.83425357657026]
  We introduce LMIndexer, a self-supervised framework to learn semantic IDs with a generative language model. We show the high quality of the learned IDs and demonstrate their effectiveness on three tasks including recommendation, product search, and document retrieval.
  arXiv  Detail & Related papers  (2023-10-11T18:56:15Z)
• Semantic Autoencoder and Its Potential Usage for Adversarial Attack [25.315265827962875]
  We propose an enhanced autoencoder architecture named semantic autoencoder. We consider adversarial attacks to learning algorithms that rely on the latent representation obtained via autoencoders.
  arXiv  Detail & Related papers  (2022-05-31T08:10:07Z)
• Unsupervised and Distributional Detection of Machine-Generated Text [1.552214657968262]
  The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. We propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams. Our experiments show that leveraging that signal allows us to rank suspicious documents accurately.
  arXiv  Detail & Related papers  (2021-11-04T14:07:46Z)
• Rethinking Text Line Recognition Models [57.47147190119394]
  We consider two decoder families (Connectionist Temporal Classification and Transformer) and three encoder modules (Bidirectional LSTMs, Self-Attention, and GRCLs) We compare their accuracy and performance on widely used public datasets of scene and handwritten text. Unlike the more common Transformer-based models, this architecture can handle inputs of arbitrary length.
  arXiv  Detail & Related papers  (2021-04-15T21:43:13Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)
• Commonsense Evidence Generation and Injection in Reading Comprehension [57.31927095547153]
  We propose a Commonsense Evidence Generation and Injection framework in reading comprehension, named CEGI. The framework injects two kinds of auxiliary commonsense evidence into comprehensive reading to equip the machine with the ability of rational thinking. Experiments on the CosmosQA dataset demonstrate that the proposed CEGI model outperforms the current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-05-11T16:31:08Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.