Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness

• URL: http://arxiv.org/abs/2205.01889v1
• Date: Wed, 4 May 2022 04:40:39 GMT
• Title: Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness
• Authors: Yun-Zhu Song and Yi-Syuan Chen and Hong-Han Shuai
• Abstract summary: A notable challenge in Multi-Document Summarization (MDS) is the extremely-long length of the input. We present an extract-then-abstract Transformer framework to overcome the problem. We propose a loss weighting mechanism that makes the model aware of the unequal importance for the sentences not in the pseudo extraction oracle.
• Score: 21.037841262371355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A notable challenge in Multi-Document Summarization (MDS) is the extremely-long length of the input. In this paper, we present an extract-then-abstract Transformer framework to overcome the problem. Specifically, we leverage pre-trained language models to construct a hierarchical extractor for salient sentence selection across documents and an abstractor for rewriting the selected contents as summaries. However, learning such a framework is challenging since the optimal contents for the abstractor are generally unknown. Previous works typically create pseudo extraction oracle to enable the supervised learning for both the extractor and the abstractor. Nevertheless, we argue that the performance of such methods could be restricted due to the insufficient information for prediction and inconsistent objectives between training and testing. To this end, we propose a loss weighting mechanism that makes the model aware of the unequal importance for the sentences not in the pseudo extraction oracle, and leverage the fine-tuned abstractor to generate summary references as auxiliary signals for learning the extractor. Moreover, we propose a reinforcement learning method that can efficiently apply to the extractor for harmonizing the optimization between training and testing. Experiment results show that our framework substantially outperforms strong baselines with comparable model sizes and achieves the best results on the Multi-News, Multi-XScience, and WikiCatSum corpora.

Related papers

• Learning to Extract Structured Entities Using Language Models [52.281701191329]
  Recent advances in machine learning have significantly impacted the field of information extraction. We reformulate the task to be entity-centric, enabling the use of diverse metrics. We contribute to the field by introducing Structured Entity Extraction and proposing the Approximate Entity Set OverlaP metric.
  arXiv  Detail & Related papers  (2024-02-06T22:15:09Z)
• Leveraging Knowledge Graph Embeddings to Enhance Contextual Representations for Relation Extraction [0.0]
  We propose a relation extraction approach based on the incorporation of pretrained knowledge graph embeddings at the corpus scale into the sentence-level contextual representation. We conducted a series of experiments which revealed promising and very interesting results for our proposed approach.
  arXiv  Detail & Related papers  (2023-06-07T07:15:20Z)
• Learning with Rejection for Abstractive Text Summarization [42.15551472507393]
  We propose a training objective for abstractive summarization based on rejection learning. We show that our method considerably improves the factuality of generated summaries in automatic and human evaluations.
  arXiv  Detail & Related papers  (2023-02-16T19:07:08Z)
• Active Learning for Abstractive Text Summarization [50.79416783266641]
  We propose the first effective query strategy for Active Learning in abstractive text summarization. We show that using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores.
  arXiv  Detail & Related papers  (2023-01-09T10:33:14Z)
• Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization [44.835811239393244]
  Sentence summarization shortens given texts while maintaining core contents of the texts. Unsupervised approaches have been studied to summarize texts without human-written summaries. We devise an abstractive model based on reinforcement learning without ground-truth summaries.
  arXiv  Detail & Related papers  (2022-12-21T08:34:28Z)
• Salience Allocation as Guidance for Abstractive Summarization [61.31826412150143]
  We propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON) SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness.
  arXiv  Detail & Related papers  (2022-10-22T02:13:44Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• Topic-Guided Abstractive Text Summarization: a Joint Learning Approach [19.623946402970933]
  We introduce a new approach for abstractive text summarization, Topic-Guided Abstractive Summarization. The idea is to incorporate neural topic modeling with a Transformer-based sequence-to-sequence (seq2seq) model in a joint learning framework.
  arXiv  Detail & Related papers  (2020-10-20T14:45:25Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)
• Combination of abstractive and extractive approaches for summarization of long scientific texts [0.0]
  We present a method to generate summaries of long scientific documents using both extractive and abstractive approaches. Our experiments showed that using extractive and abstractive models jointly significantly improves summarization results and ROUGE scores.
  arXiv  Detail & Related papers  (2020-06-09T15:38:21Z)
• A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis [73.74885246830611]
  We propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network (DreGcn) Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-04T14:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.