MemSum: Extractive Summarization of Long Documents using Multi-step Episodic Markov Decision Processes

• URL: http://arxiv.org/abs/2107.08929v1
• Date: Mon, 19 Jul 2021 14:41:31 GMT
• Title: MemSum: Extractive Summarization of Long Documents using Multi-step Episodic Markov Decision Processes
• Authors: Nianlong Gu, Elliott Ash, Richard H.R. Hahnloser
• Abstract summary: We introduce MemSum, a reinforcement-learning-based extractive summarizer enriched at any given time step with information on the current extraction history. Our innovation is in considering a broader information set when summarizing that would intuitively also be used by humans in this task.
• Score: 6.585259903186036
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce MemSum (Multi-step Episodic Markov decision process extractive SUMmarizer), a reinforcement-learning-based extractive summarizer enriched at any given time step with information on the current extraction history. Similar to previous models in this vein, MemSum iteratively selects sentences into the summary. Our innovation is in considering a broader information set when summarizing that would intuitively also be used by humans in this task: 1) the text content of the sentence, 2) the global text context of the rest of the document, and 3) the extraction history consisting of the set of sentences that have already been extracted. With a lightweight architecture, MemSum nonetheless obtains state-of-the-art test-set performance (ROUGE score) on long document datasets (PubMed, arXiv, and GovReport). Supporting analysis demonstrates that the added awareness of extraction history gives MemSum robustness against redundancy in the source document.

Related papers

• Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization [48.57273563299046]
  We propose the task of Stepwise Summarization, which aims to generate a new appended summary each time a new document is proposed. The appended summary should not only summarize the newly added content but also be coherent with the previous summary. We show that SSG achieves state-of-the-art performance in terms of both automatic metrics and human evaluations.
  arXiv  Detail & Related papers  (2024-06-08T05:37:26Z)
• On Context Utilization in Summarization with Large Language Models [83.84459732796302]
  Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. We conduct the first comprehensive study on context utilization and position bias in summarization.
  arXiv  Detail & Related papers  (2023-10-16T16:45:12Z)
• GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state [6.4805900740861]
  We propose GoSum, a reinforcement-learning-based extractive model for long-paper summarization. GoSum encodes states by building a heterogeneous graph from different discourse levels for each input document. We evaluate the model on two datasets of scientific articles summarization: PubMed and arXiv.
  arXiv  Detail & Related papers  (2022-11-18T14:07:29Z)
• Text Summarization with Oracle Expectation [88.39032981994535]
  Extractive summarization produces summaries by identifying and concatenating the most important sentences in a document. Most summarization datasets do not come with gold labels indicating whether document sentences are summary-worthy. We propose a simple yet effective labeling algorithm that creates soft, expectation-based sentence labels.
  arXiv  Detail & Related papers  (2022-09-26T14:10:08Z)
• A Survey on Neural Abstractive Summarization Methods and Factual Consistency of Summarization [18.763290930749235]
  summarization is the process of shortening a set of textual data computationally, to create a subset (a summary) Existing summarization methods can be roughly divided into two types: extractive and abstractive. An extractive summarizer explicitly selects text snippets from the source document, while an abstractive summarizer generates novel text snippets to convey the most salient concepts prevalent in the source.
  arXiv  Detail & Related papers  (2022-04-20T14:56:36Z)
• Reinforcing Semantic-Symmetry for Document Summarization [15.113768658584979]
  Document summarization condenses a long document into a short version with salient information and accurate semantic descriptions. This paper introduces a new textbfreinforcing stextbfemantic-textbfsymmetry learning textbfmodel is proposed for document summarization. A series of experiments have been conducted on two wildly used benchmark datasets CNN/Daily Mail and BigPatent.
  arXiv  Detail & Related papers  (2021-12-14T17:41:37Z)
• Leveraging Information Bottleneck for Scientific Document Summarization [26.214930773343887]
  This paper presents an unsupervised extractive approach to summarize scientific long documents. Inspired by previous work which uses the Information Bottleneck principle for sentence compression, we extend it to document level summarization.
  arXiv  Detail & Related papers  (2021-10-04T09:43:47Z)
• On Generating Extended Summaries of Long Documents [16.149617108647707]
  We present a new method for generating extended summaries of long papers. Our method exploits hierarchical structure of the documents and incorporates it into an extractive summarization model. Our analysis shows that our multi-tasking approach can adjust extraction probability distribution to the favor of summary-worthy sentences.
  arXiv  Detail & Related papers  (2020-12-28T08:10:28Z)
• SupMMD: A Sentence Importance Model for Extractive Summarization using Maximum Mean Discrepancy [92.5683788430012]
  SupMMD is a novel technique for generic and update summarization based on the maximum discrepancy from kernel two-sample testing. We show the efficacy of SupMMD in both generic and update summarization tasks by meeting or exceeding the current state-of-the-art on the DUC-2004 and TAC-2009 datasets.
  arXiv  Detail & Related papers  (2020-10-06T09:26:55Z)
• Extractive Summarization as Text Matching [123.09816729675838]
  This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. We formulate the extractive summarization task as a semantic text matching problem. We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
  arXiv  Detail & Related papers  (2020-04-19T08:27:57Z)
• At Which Level Should We Extract? An Empirical Analysis on Extractive Document Summarization [110.54963847339775]
  We show that unnecessity and redundancy issues exist when extracting full sentences. We propose extracting sub-sentential units based on the constituency parsing tree.
  arXiv  Detail & Related papers  (2020-04-06T13:35:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.