PromptSum: Parameter-Efficient Controllable Abstractive Summarization

• URL: http://arxiv.org/abs/2308.03117v1
• Date: Sun, 6 Aug 2023 13:54:14 GMT
• Title: PromptSum: Parameter-Efficient Controllable Abstractive Summarization
• Authors: Mathieu Ravaut, Hailin Chen, Ruochen Zhao, Chengwei Qin, Shafiq Joty, Nancy Chen
• Abstract summary: We introduce PromptSum, a method combining PT with a multi-task objective and discrete entity prompts for abstractive summarization. Our model competitive ROUGE results on popular abstractive summarization benchmarks coupled with a strong level of controllability through entities.
• Score: 4.145362426026615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prompt tuning (PT), a parameter-efficient technique that only tunes the additional prompt embeddings while keeping the backbone pre-trained language model (PLM) frozen, has shown promising results in language understanding tasks, especially in low-resource scenarios. However, effective prompt design methods suitable for generation tasks such as summarization are still lacking. At the same time, summarization guided through instructions (discrete prompts) can achieve a desirable double objective of high quality and controllability in summary generation. Towards a goal of strong summarization performance under the triple conditions of parameter-efficiency, data-efficiency, and controllability, we introduce PromptSum, a method combining PT with a multi-task objective and discrete entity prompts for abstractive summarization. Our model achieves competitive ROUGE results on popular abstractive summarization benchmarks coupled with a strong level of controllability through entities, all while only tuning several orders of magnitude less parameters.

Related papers

• Guide-to-Explain for Controllable Summarization [11.904090197598505]
  controllable summarization with large language models (LLMs) remains underexplored. We propose a guide-to-explain framework (GTE) for controllable summarization. Our framework enables the model to identify misaligned attributes in the initial draft and guides it in explaining errors in the previous output.
  arXiv  Detail & Related papers  (2024-11-19T12:36:02Z)
• Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization [33.67670065326008]
  This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs) Our results show that fine-tuned Llama-2-7B-based summarization model performs on-par with GPT-4 in terms of factual accuracy, completeness and conciseness.
  arXiv  Detail & Related papers  (2024-10-24T10:32:10Z)
• Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning [55.265138447400744]
  Statement-Tuning is a technique that models discriminative tasks as a set of finite statements and trains an encoder model to discriminate between the potential statements to determine the label. Experimental results demonstrate that Statement-Tuning achieves competitive performance compared to state-of-the-art LLMs with significantly fewer parameters. The study investigates the impact of several design choices on few-shot and zero-shot generalization, revealing that Statement-Tuning can achieve strong performance with modest training data.
  arXiv  Detail & Related papers  (2024-04-19T14:05:03Z)
• Towards Abstractive Timeline Summarisation using Preference-based Reinforcement Learning [3.6640004265358477]
  This paper introduces a novel pipeline for summarising timelines of events reported by multiple news sources. Transformer-based models for abstractive summarisation generate coherent and concise summaries of long documents. While extractive summaries are more faithful to their sources, they may be less readable and contain redundant or unnecessary information.
  arXiv  Detail & Related papers  (2022-11-14T18:24:13Z)
• MACSum: Controllable Summarization with Mixed Attributes [56.685735509260276]
  MACSum is the first human-annotated summarization dataset for controlling mixed attributes. We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization.
  arXiv  Detail & Related papers  (2022-11-09T17:17:37Z)
• COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization [84.70895015194188]
  We propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score.
  arXiv  Detail & Related papers  (2022-09-29T06:11:21Z)
• Controllable Summarization with Constrained Markov Decision Process [50.04321779376415]
  We study controllable text summarization which allows users to gain control on a particular attribute. We propose a novel training framework based on Constrained Markov Decision Process (CMDP) Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness.
  arXiv  Detail & Related papers  (2021-08-07T09:12:53Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)
• An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [84.49035467829819]
  We show that it is possible to better manage this trade-off by optimizing a bound on the Information Bottleneck (IB) objective. Our fully unsupervised approach jointly learns an explainer that predicts sparse binary masks over sentences, and an end-task predictor that considers only the extracted rationale.
  arXiv  Detail & Related papers  (2020-05-01T23:26:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.