On the Trade-off between Redundancy and Local Coherence in Summarization

• URL: http://arxiv.org/abs/2205.10192v2
• Date: Thu, 6 Jun 2024 13:27:20 GMT
• Title: On the Trade-off between Redundancy and Local Coherence in Summarization
• Authors: Ronald Cardenas, Matthias Galle, Shay B. Cohen,
• Abstract summary: We investigate the trade-offs incurred when aiming to control for inter-sentential cohesion and redundancy in extracted summaries. We find that the proposed unsupervised systems manage to extract highly cohesive summaries across varying levels of document redundancy.
• Score: 20.16107829497668
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extractive summaries are usually presented as lists of sentences with no expected cohesion between them and with plenty of redundant information if not accounted for. In this paper, we investigate the trade-offs incurred when aiming to control for inter-sentential cohesion and redundancy in extracted summaries, and their impact on their informativeness. As case study, we focus on the summarization of long, highly redundant documents and consider two optimization scenarios, reward-guided and with no supervision. In the reward-guided scenario, we compare systems that control for redundancy and cohesion during sentence scoring. In the unsupervised scenario, we introduce two systems that aim to control all three properties -- informativeness, redundancy, and cohesion -- in a principled way. Both systems implement a psycholinguistic theory that simulates how humans keep track of relevant content units and how cohesion and non-redundancy constraints are applied in short-term memory during reading. Extensive automatic and human evaluations reveal that systems optimizing for -- among other properties -- cohesion are capable of better organizing content in summaries compared to systems that optimize only for redundancy, while maintaining comparable informativeness. We find that the proposed unsupervised systems manage to extract highly cohesive summaries across varying levels of document redundancy, although sacrificing informativeness in the process. Finally, we lay evidence as to how simulated cognitive processes impact the trade-off between the analyzed summary properties.

Related papers

• Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence [51.54175067684008]
  This paper introduces a Transformer-based integrative feature and cost aggregation network designed for dense matching tasks. We first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal the potential for substantial benefits stemming from the judicious use of both aggregation processes. Our framework is evaluated on standard benchmarks for semantic matching, and also applied to geometric matching, where we show that our approach achieves significant improvements compared to existing methods.
  arXiv  Detail & Related papers  (2024-03-17T07:02:55Z)
• `Keep it Together': Enforcing Cohesion in Extractive Summaries by Simulating Human Memory [22.659031563705245]
  In this paper, we aim to enforce cohesion whilst controlling for informativeness and redundancy in summaries. Our sentence selector simulates human memory to keep track of topics. It is possible to extract highly cohesive summaries that nevertheless read as informative to humans.
  arXiv  Detail & Related papers  (2024-02-16T12:43:26Z)
• Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization [56.94741578760294]
  We propose the task of fine-grained inconsistency detection, the goal of which is to predict the fine-grained types of factual errors in a summary. Motivated by how humans inspect factual inconsistency in summaries, we propose an interpretable fine-grained inconsistency detection model, FineGrainFact.
  arXiv  Detail & Related papers  (2023-05-23T22:11:47Z)
• Enhancing Coherence of Extractive Summarization with Multitask Learning [40.349019691412465]
  This study proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and coherent discriminator module. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in the extracted summaries.
  arXiv  Detail & Related papers  (2023-05-22T09:20:58Z)
• How to Find Strong Summary Coherence Measures? A Toolbox and a Comparative Study for Summary Coherence Measure Evaluation [3.434197496862117]
  We conduct a large-scale investigation of various methods for summary coherence modelling on an even playing field. We introduce two novel analysis measures, intra-system correlation and bias matrices, that help identify biases in coherence measures and provide robustness against system-level confounders. While none of the currently available automatic coherence measures are able to assign reliable coherence scores to system summaries across all evaluation metrics, large-scale language models show promising results, as long as fine-tuning takes into account that they need to generalize across different summary lengths.
  arXiv  Detail & Related papers  (2022-09-14T09:42:19Z)
• SNaC: Coherence Error Detection for Narrative Summarization [73.48220043216087]
  We introduce SNaC, a narrative coherence evaluation framework rooted in fine-grained annotations for long summaries. We develop a taxonomy of coherence errors in generated narrative summaries and collect span-level annotations for 6.6k sentences across 150 book and movie screenplay summaries. Our work provides the first characterization of coherence errors generated by state-of-the-art summarization models and a protocol for eliciting coherence judgments from crowd annotators.
  arXiv  Detail & Related papers  (2022-05-19T16:01:47Z)
• Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics [64.81682222169113]
  How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by system-level correlations. We identify two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice.
  arXiv  Detail & Related papers  (2022-04-21T15:52:14Z)
• Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video [128.41392860714635]
  We introduce Weakly-Supervised Snoma-Temporally Detection (WSSTAD) in surveillance video. WSSTAD aims to localize a-temporal tube (i.e. sequence of bounding boxes at consecutive times) that encloses abnormal event. We propose a dual-branch network which takes as input proposals with multi-granularities in both spatial-temporal domains.
  arXiv  Detail & Related papers  (2021-08-09T06:11:14Z)
• Unsupervised Extractive Summarization using Pointwise Mutual Information [5.544401446569243]
  We propose new metrics of relevance and redundancy using pointwise mutual information (PMI) between sentences. We show that our method outperforms similarity-based methods on datasets in a range of domains including news, medical journal articles, and personal anecdotes.
  arXiv  Detail & Related papers  (2021-02-11T21:05:50Z)
• Systematically Exploring Redundancy Reduction in Summarizing Long Documents [6.812554384019158]
  We explore and compare different ways to deal with redundancy when summarizing long documents. In a series of experiments, we show that our proposed methods achieve the state-of-the-art with respect to ROUGE scores on two scientific paper datasets.
  arXiv  Detail & Related papers  (2020-11-30T19:07:27Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.