Extract-and-Abstract: Unifying Extractive and Abstractive Summarization within Single Encoder-Decoder Framework

• URL: http://arxiv.org/abs/2409.11827v1
• Date: Wed, 18 Sep 2024 09:21:25 GMT
• Title: Extract-and-Abstract: Unifying Extractive and Abstractive Summarization within Single Encoder-Decoder Framework
• Authors: Yuping Wu, Hao Li, Hongbo Zhu, Goran Nenadic, Xiao-Jun Zeng,
• Abstract summary: We propose ExtAbs, which jointly and seamlessly performs Extractive and Abstractive summarization tasks within single encoder-decoder model. In ExtAbs, the vanilla encoder is augmented to extract salients, and the vanilla decoder is modified with the proposed saliency mask to generate summaries. Experiments show that ExtAbs can achieve superior performance than baselines on the extractive task and performs comparable, or even better than the vanilla models on the abstractive task.
• Score: 24.97672212363703
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Extract-then-Abstract is a naturally coherent paradigm to conduct abstractive summarization with the help of salient information identified by the extractive model. Previous works that adopt this paradigm train the extractor and abstractor separately and introduce extra parameters to highlight the extracted salients to the abstractor, which results in error accumulation and additional training costs. In this paper, we first introduce a parameter-free highlight method into the encoder-decoder framework: replacing the encoder attention mask with a saliency mask in the cross-attention module to force the decoder to focus only on salient parts of the input. A preliminary analysis compares different highlight methods, demonstrating the effectiveness of our saliency mask. We further propose the novel extract-and-abstract paradigm, ExtAbs, which jointly and seamlessly performs Extractive and Abstractive summarization tasks within single encoder-decoder model to reduce error accumulation. In ExtAbs, the vanilla encoder is augmented to extract salients, and the vanilla decoder is modified with the proposed saliency mask to generate summaries. Built upon BART and PEGASUS, experiments on three datasets show that ExtAbs can achieve superior performance than baselines on the extractive task and performs comparable, or even better than the vanilla models on the abstractive task.

Related papers

• How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval? [120.49126407479717]
  We propose a sketch-based image retrieval framework capable of handling sketch abstraction at varied levels. For granularity-level abstraction understanding, we dictate that the retrieval model should not treat all abstraction-levels equally. Our Acc.@q loss uniquely allows a sketch to narrow/broaden its focus in terms of how stringent the evaluation should be.
  arXiv  Detail & Related papers  (2024-03-11T23:08:29Z)
• Generalizable Entity Grounding via Assistance of Large Language Model [77.07759442298666]
  We propose a novel approach to densely ground visual entities from a long caption. We leverage a large multimodal model to extract semantic nouns, a class-a segmentation model to generate entity-level segmentation, and a multi-modal feature fusion module to associate each semantic noun with its corresponding segmentation mask.
  arXiv  Detail & Related papers  (2024-02-04T16:06:05Z)
• Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning [36.93582300019002]
  We propose an efficient and interpretable compressive summarisation method using unsupervised dual-agent reinforcement learning. Our model achieves promising performance and a significant improvement on Newsroom in terms of the ROUGE metric.
  arXiv  Detail & Related papers  (2023-06-06T05:30:49Z)
• Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization [76.57699934689468]
  We propose a fine-grained Token-level retrieval-augmented mechanism (Tram) on the decoder side to enhance the performance of neural models. To overcome the challenge of token-level retrieval in capturing contextual code semantics, we also propose integrating code semantics into individual summary tokens.
  arXiv  Detail & Related papers  (2023-05-18T16:02:04Z)
• A Simple Framework for Open-Vocabulary Segmentation and Detection [85.21641508535679]
  We present OpenSeeD, a simple Open-vocabulary and Detection framework that jointly learns from different segmentation and detection datasets. We first introduce a pre-trained text encoder to encode all the visual concepts in two tasks and learn a common semantic space for them. After pre-training, our model exhibits competitive or stronger zero-shot transferability for both segmentation and detection.
  arXiv  Detail & Related papers  (2023-03-14T17:58:34Z)
• An Extractive-and-Abstractive Framework for Source Code Summarization [28.553366270065656]
  Code summarization aims to automatically generate summaries/comments for a given code snippet in the form of natural language. We propose a novel extractive-and-abstractive framework to generate human-written-like summaries with preserved factual details.
  arXiv  Detail & Related papers  (2022-06-15T02:14:24Z)
• Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness [21.037841262371355]
  A notable challenge in Multi-Document Summarization (MDS) is the extremely-long length of the input. We present an extract-then-abstract Transformer framework to overcome the problem. We propose a loss weighting mechanism that makes the model aware of the unequal importance for the sentences not in the pseudo extraction oracle.
  arXiv  Detail & Related papers  (2022-05-04T04:40:39Z)
• HydraSum -- Disentangling Stylistic Features in Text Summarization using Multi-Decoder Models [12.070474521259776]
  We introduce HydraSum, a new summarization architecture that extends the single decoder framework of current models. Our proposed model encourages each expert, i.e. decoder, to learn and generate stylistically-distinct summaries. A guided version of the training process can explicitly govern which summary style is partitioned between decoders.
  arXiv  Detail & Related papers  (2021-10-08T22:49:49Z)
• Contrastive Triple Extraction with Generative Transformer [72.21467482853232]
  We introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object.
  arXiv  Detail & Related papers  (2020-09-14T05:29:24Z)
• Combination of abstractive and extractive approaches for summarization of long scientific texts [0.0]
  We present a method to generate summaries of long scientific documents using both extractive and abstractive approaches. Our experiments showed that using extractive and abstractive models jointly significantly improves summarization results and ROUGE scores.
  arXiv  Detail & Related papers  (2020-06-09T15:38:21Z)
• Exploring Explainable Selection to Control Abstractive Summarization [51.74889133688111]
  We develop a novel framework that focuses on explainability. A novel pair-wise matrix captures the sentence interactions, centrality, and attribute scores. A sentence-deployed attention mechanism in the abstractor ensures the final summary emphasizes the desired content.
  arXiv  Detail & Related papers  (2020-04-24T14:39:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.