AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content

• URL: http://arxiv.org/abs/2305.14806v2
• Date: Thu, 16 Nov 2023 11:47:05 GMT
• Title: AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content
• Authors: Shuyang Cao and Lu Wang
• Abstract summary: Long document summarization systems are critical for domains with lengthy and jargonladen text. AWESOME produces summaries with improved informativeness, faithfulness, and coherence than competitive baselines on longer documents.
• Score: 16.458279293804285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long document summarization systems are critical for domains with lengthy and jargonladen text, yet they present significant challenges to researchers and developers with limited computing resources. Existing solutions mainly focus on efficient attentions or divide-and-conquer strategies. The former reduces theoretical time complexity, but is still memory-heavy. The latter methods sacrifice global context, leading to uninformative and incoherent summaries. This work aims to leverage the memory-efficient nature of divide-and-conquer methods while preserving global context. Concretely, our framework AWESOME uses two novel mechanisms: (1) External memory mechanisms track previously encoded document segments and their corresponding summaries, to enhance global document understanding and summary coherence. (2) Global salient content is further identified beforehand to augment each document segment to support its summarization. Extensive experiments on diverse genres of text, including government reports, transcripts, scientific papers, and novels, show that AWESOME produces summaries with improved informativeness, faithfulness, and coherence than competitive baselines on longer documents, while having a smaller GPU memory footprint.

Related papers

• LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models [73.13933847198395]
  We propose a training-free framework for processing long texts, utilizing a divide-and-conquer strategy to achieve comprehensive document understanding. The proposed LLM$times$MapReduce framework splits the entire document into several chunks for LLMs to read and then aggregates the intermediate answers to produce the final output.
  arXiv  Detail & Related papers  (2024-10-12T03:13:44Z)
• Graph Neural Network and NER-Based Text Summarization [1.5850926890180461]
  This project introduces an innovative approach to text summarization, leveraging the capabilities of Graph Neural Networks (GNNs) and Named Entity Recognition (NER) systems. Our method aims to enhance the efficiency of summarization and also tries to ensures a high degree relevance in the condensed content.
  arXiv  Detail & Related papers  (2024-02-05T03:00:44Z)
• RET-LLM: Towards a General Read-Write Memory for Large Language Models [53.288356721954514]
  RET-LLM is a novel framework that equips large language models with a general write-read memory unit. Inspired by Davidsonian semantics theory, we extract and save knowledge in the form of triplets. Our framework exhibits robust performance in handling temporal-based question answering tasks.
  arXiv  Detail & Related papers  (2023-05-23T17:53:38Z)
• Document-Level Abstractive Summarization [0.0]
  We study how efficient Transformer techniques can be used to improve the automatic summarization of very long texts. We propose a novel retrieval-enhanced approach which reduces the cost of generating a summary of the entire document by processing smaller chunks.
  arXiv  Detail & Related papers  (2022-12-06T14:39:09Z)
• Dynamic Global Memory for Document-level Argument Extraction [63.314514124716936]
  We introduce a new global neural generation-based framework for document-level event argument extraction. We use a document memory store to record the contextual event information and leverage it to implicitly and explicitly help with decoding of arguments for later events. Empirical results show that our framework outperforms prior methods substantially.
  arXiv  Detail & Related papers  (2022-09-18T23:45:25Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Unsupervised Summarization with Customized Granularities [76.26899748972423]
  We propose the first unsupervised multi-granularity summarization framework, GranuSum. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-01-29T05:56:35Z)
• Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents [30.09742243490895]
  FacetSum is a faceted summarization benchmark built on Emerald journal articles. Analyses and empirical results on our dataset reveal the importance of bringing structure into summaries. We believe FacetSum will spur further advances in summarization research and foster the development of NLP systems.
  arXiv  Detail & Related papers  (2021-05-31T22:58:38Z)
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries [46.183289748907804]
  We propose SOE, a pipelined system that outlines, outlining and elaborating for long text generation. SOE produces long texts with significantly better quality, along with faster convergence speed.
  arXiv  Detail & Related papers  (2020-10-14T13:22:20Z)
• From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information [77.89755281215079]
  Text summarization is the research area aiming at creating a short and condensed version of the original document. In real-world applications, most of the data is not in a plain text format. This paper focuses on the survey of these new summarization tasks and approaches in the real-world application.
  arXiv  Detail & Related papers  (2020-05-10T14:59:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.