Controllable Topic-Focused Abstractive Summarization

• URL: http://arxiv.org/abs/2311.06724v1
• Date: Sun, 12 Nov 2023 03:51:38 GMT
• Title: Controllable Topic-Focused Abstractive Summarization
• Authors: Seyed Ali Bahrainian, Martin Jaggi, Carsten Eickhoff
• Abstract summary: Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects. This paper presents a new Transformer-based architecture capable of producing topic-focused summaries.
• Score: 57.8015120583044
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects by shifting the distribution of generated text towards a desired style, e.g., a set of topics. Subsequently, the resulting summaries may be tailored to user-defined requirements. This paper presents a new Transformer-based architecture capable of producing topic-focused summaries. The architecture modifies the cross-attention mechanism of the Transformer to bring topic-focus control to the generation process while not adding any further parameters to the model. We show that our model sets a new state of the art on the NEWTS dataset in terms of topic-focused abstractive summarization as well as a topic-prevalence score. Moreover, we show via extensive experiments that our proposed topical cross-attention mechanism can be plugged into various Transformer models, such as BART and T5, improving their performance on the CNN/Dailymail and XSum benchmark datasets for abstractive summarization. This is achieved via fine-tuning, without requiring training from scratch. Finally, we show through human evaluation that our model generates more faithful summaries outperforming the state-of-the-art Frost model.

Related papers

• Multi-View Unsupervised Image Generation with Cross Attention Guidance [23.07929124170851]
  This paper introduces a novel pipeline for unsupervised training of a pose-conditioned diffusion model on single-category datasets. We identify object poses by clustering the dataset through comparing visibility and locations of specific object parts. Our model, MIRAGE, surpasses prior work in novel view synthesis on real images.
  arXiv  Detail & Related papers  (2023-12-07T14:55:13Z)
• Enriching and Controlling Global Semantics for Text Summarization [11.037667460077813]
  Transformer-based models have been proven effective in the abstractive summarization task by creating fluent and informative summaries. We introduce a neural topic model empowered with normalizing flow to capture the global semantics of the document, which are then integrated into the summarization model. Our method outperforms state-of-the-art summarization models on five common text summarization datasets.
  arXiv  Detail & Related papers  (2021-09-22T09:31:50Z)
• Aspect-Controllable Opinion Summarization [58.5308638148329]
  We propose an approach that allows the generation of customized summaries based on aspect queries. Using a review corpus, we create a synthetic training dataset of (review, summary) pairs enriched with aspect controllers. We fine-tune a pretrained model using our synthetic dataset and generate aspect-specific summaries by modifying the aspect controllers.
  arXiv  Detail & Related papers  (2021-09-07T16:09:17Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)
• Topic-Guided Abstractive Text Summarization: a Joint Learning Approach [19.623946402970933]
  We introduce a new approach for abstractive text summarization, Topic-Guided Abstractive Summarization. The idea is to incorporate neural topic modeling with a Transformer-based sequence-to-sequence (seq2seq) model in a joint learning framework.
  arXiv  Detail & Related papers  (2020-10-20T14:45:25Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)
• Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling [81.33107307509718]
  We propose a topic adaptive storyteller to model the ability of inter-topic generalization. We also propose a prototype encoding structure to model the ability of intra-topic derivation. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model.
  arXiv  Detail & Related papers  (2020-08-11T03:55:11Z)
• Exploring Explainable Selection to Control Abstractive Summarization [51.74889133688111]
  We develop a novel framework that focuses on explainability. A novel pair-wise matrix captures the sentence interactions, centrality, and attribute scores. A sentence-deployed attention mechanism in the abstractor ensures the final summary emphasizes the desired content.
  arXiv  Detail & Related papers  (2020-04-24T14:39:34Z)
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising [44.384730968526156]
  We propose a transformer-based unsupervised abstractive summarization system with pretraining on large-scale data. We first leverage the lead bias in news articles to pretrain the model on millions of unlabeled corpora. We finetune TED on target domains through theme modeling and a denoising autoencoder to enhance the quality of generated summaries.
  arXiv  Detail & Related papers  (2020-01-03T05:15:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.