GPT-too: A language-model-first approach for AMR-to-text generation

• URL: http://arxiv.org/abs/2005.09123v2
• Date: Wed, 27 May 2020 11:24:29 GMT
• Title: GPT-too: A language-model-first approach for AMR-to-text generation
• Authors: Manuel Mager, Ramon Fernandez Astudillo, Tahira Naseem, Md Arafat Sultan, Young-Suk Lee, Radu Florian and Salim Roukos
• Abstract summary: We propose an approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques.
• Score: 22.65728041544785
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequence-to-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.

Related papers

• AMR Parsing with Causal Hierarchical Attention and Pointers [54.382865897298046]
  We introduce new target forms of AMR parsing and a novel model, CHAP, which is equipped with causal hierarchical attention and the pointer mechanism. Experiments show that our model outperforms baseline models on four out of five benchmarks in the setting of no additional data.
  arXiv  Detail & Related papers  (2023-10-18T13:44:26Z)
• Grounded Image Text Matching with Mismatched Relation Reasoning [39.524420144738684]
  Grounded Image Text Matching with Mismatched Relation (GITM-MR) is a novel visual-linguistic joint task. GITM-MR requires a model to first determine if an expression describes an image, then localize referred objects or ground the mismatched parts of the text. We propose the Relation-sensitive Correspondence Reasoning Network (RCRN), which incorporates relation-aware reasoning via bi-directional message propagation.
  arXiv  Detail & Related papers  (2023-08-02T15:44:36Z)
• Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
  We make empirical studies of state-of-the-art UniDA methods using foundation models. We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models. Although simple, our method outperforms previous approaches in most benchmark tasks.
  arXiv  Detail & Related papers  (2023-05-18T16:28:29Z)
• A Survey : Neural Networks for AMR-to-Text [2.3924114046608627]
  AMR-to-Text is one of the key techniques in the NLP community that aims at generating sentences from the Abstract Meaning Representation (AMR) graphs. Since AMR was proposed in 2013, the study on AMR-to-Text has become increasingly prevalent as an essential branch of structured data to text.
  arXiv  Detail & Related papers  (2022-06-15T07:20:28Z)
• Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing [20.67024416678313]
  We explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach. We propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning. We show that the proposed parsing architecture retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new state of the art for AMR 2.0, without the need for graph re-categorization.
  arXiv  Detail & Related papers  (2021-10-29T04:36:31Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• Improving AMR Parsing with Sequence-to-Sequence Pre-training [39.33133978535497]
  In this paper, we focus on sequence-to-sequence (seq2seq) AMR parsing. We propose a seq2seq pre-training approach to build pre-trained models in both single and joint way. Experiments show that both the single and joint pre-trained models significantly improve the performance.
  arXiv  Detail & Related papers  (2020-10-05T04:32:47Z)
• Dynamic Data Selection and Weighting for Iterative Back-Translation [116.14378571769045]
  We propose a curriculum learning strategy for iterative back-translation models. We evaluate our models on domain adaptation, low-resource, and high-resource MT settings. Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines.
  arXiv  Detail & Related papers  (2020-04-07T19:49:58Z)
• Document Ranking with a Pretrained Sequence-to-Sequence Model [56.44269917346376]
  We show how a sequence-to-sequence model can be trained to generate relevance labels as "target words" Our approach significantly outperforms an encoder-only model in a data-poor regime.
  arXiv  Detail & Related papers  (2020-03-14T22:29:50Z)
• Learning by Semantic Similarity Makes Abstractive Summarization Better [13.324006587838522]
  We compare the generated summaries from recent LM, BART, and the reference summaries from a benchmark dataset, CNN/DM. Interestingly, model-generated summaries receive higher scores relative to reference summaries.
  arXiv  Detail & Related papers  (2020-02-18T17:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.