Summarization with Precise Length Control

• URL: http://arxiv.org/abs/2305.05171v1
• Date: Tue, 9 May 2023 04:45:24 GMT
• Title: Summarization with Precise Length Control
• Authors: Lesly Miculicich, Yujia Xie, Song Wang, Pengcheng He
• Abstract summary: We present a framework to generate summaries with precisely the specified number of tokens or sentences. We jointly train the models to predict the lengths, so our model can generate summaries with optimal length.
• Score: 23.688834410051
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many applications of text generation such as summarization benefit from accurately controlling the text length. Existing approaches on length-controlled summarization either result in degraded performance or can only control the length approximately. In this work, we present a framework to generate summaries with precisely the specified number of tokens or sentences, while maintaining or even improving the text quality. In addition, we jointly train the models to predict the lengths, so our model can generate summaries with optimal length. We evaluate the proposed framework on the CNNDM dataset and show improved performance compared to existing methods.

Related papers

• A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers [32.53051395472311]
  Length-control summarization aims to condense long texts into a short one within a certain length limit. Previous approaches often use autoregressive (AR) models and treat the length requirement as a soft constraint. Our approach allows for multiple plausible sequence fragments and predicts a emphpath to connect them.
  arXiv  Detail & Related papers  (2025-02-06T22:12:55Z)
• Length Controlled Generation for Black-box LLMs [70.57649832433451]
  Large language models (LLMs) have demonstrated impressive instruction following capabilities, but struggle to accurately manage the length of generated text. We propose a novel iterative sampling framework for text length control, integrating the Metropolis-Hastings algorithm with an importance sampling acceleration strategy. Our framework achieves almost 100% success rates of length control on Llama3.1 for tasks such as length-controlled abstractive summarization.
  arXiv  Detail & Related papers  (2024-12-19T09:07:38Z)
• LongAlign: A Recipe for Long Context Alignment of Large Language Models [61.85923382850057]
  LongAlign is a recipe of the instruction data, training, and evaluation for long context alignment. We construct a long instruction-following dataset using Self-Instruct. We adopt the packing and sorted strategies to speed up supervised fine-tuning on data with varied length distributions.
  arXiv  Detail & Related papers  (2024-01-31T18:29:39Z)
• Effective Long-Context Scaling of Foundation Models [90.57254298730923]
  We present a series of long-context LLMs that support effective context windows of up to 32,768 tokens. Our models achieve consistent improvements on most regular tasks and significant improvements on long-context tasks over Llama 2.
  arXiv  Detail & Related papers  (2023-09-27T21:41:49Z)
• Hierarchical3D Adapters for Long Video-to-text Summarization [79.01926022762093]
  multimodal information offers superior performance over more memory-heavy and fully fine-tuned textual summarization methods. Our experiments demonstrate that multimodal information offers superior performance over more memory-heavy and fully fine-tuned textual summarization methods.
  arXiv  Detail & Related papers  (2022-10-10T16:44:36Z)
• Adapting Pretrained Text-to-Text Models for Long Text Sequences [39.62224414485055]
  We adapt an existing pretrained text-to-text model for long-sequence inputs. We build a long-context model that achieves competitive performance on long-text QA tasks.
  arXiv  Detail & Related papers  (2022-09-21T00:41:07Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Reinforced Abstractive Summarization with Adaptive Length Controlling [12.793451906532223]
  Controllable summarization, especially of the length, is an important issue for some practical applications. We propose an textbfAdaptive textbfLength textbfControlling textbfOptimization (textbfALCO) method to leverage two-stage abstractive summarization model.
  arXiv  Detail & Related papers  (2021-12-14T16:48:47Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• LenAtten: An Effective Length Controlling Unit For Text Summarization [5.554982420311913]
  Fixed length summarization aims at generating summaries with a preset number of words or characters. Most recent researches incorporate length information with word embeddings as the input to the recurrent decoding unit. We present an effective length controlling unit Length Attention (LenAtten) to break this trade-off.
  arXiv  Detail & Related papers  (2021-06-01T08:45:41Z)
• Length-controllable Abstractive Summarization by Guiding with Summary Prototype [27.094797760775297]
  We propose a new length-controllable abstractive summarization model. Our model generates a summary in two steps. Experiments with the CNN/Daily Mail dataset and the NEWSROOM dataset show that our model outperformed previous models in length-controlled settings.
  arXiv  Detail & Related papers  (2020-01-21T04:01:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.