P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models

• URL: http://arxiv.org/abs/2311.09741v2
• Date: Thu, 4 Apr 2024 09:10:34 GMT
• Title: P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models
• Authors: Yuhan Liu, Shangbin Feng, Xiaochuang Han, Vidhisha Balachandran, Chan Young Park, Sachin Kumar, Yulia Tsvetkov,
• Abstract summary: We find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries. We propose P3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. Experiments on three news summarization datasets demonstrate that P3SUM outperforms state-of-the-art summarization systems.
• Score: 57.571395694391654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we take a first step towards designing summarization systems that are faithful to the author's intent, not only the semantic content of the article. Focusing on a case study of preserving political perspectives in news summarization, we find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries, misrepresenting the intent and perspectives of the news authors. We thus propose P^3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. In P^3SUM, the political leaning of a generated summary is iteratively evaluated at each decoding step, and any drift from the article's original stance incurs a loss back-propagated to the embedding layers, steering the political stance of the summary at inference time. Extensive experiments on three news summarization datasets demonstrate that P^3SUM outperforms state-of-the-art summarization systems and large language models by up to 13.7% in terms of the success rate of stance preservation, with competitive performance on standard metrics of summarization quality. Our findings present a first analysis of preservation of pragmatic features in summarization, highlight the lacunae in existing summarization models -- that even state-of-the-art models often struggle to preserve author's intents -- and develop new summarization systems that are more faithful to author's perspectives.

Related papers

• Summarization of Opinionated Political Documents with Varied Perspectives [11.399915001583059]
  Models capable of generating accurate summaries of diverse perspectives can help reduce such polarization by exposing users to alternative perspectives. We introduce a novel dataset and task for independently summarizing each political perspective in a set of passages from opinionated news articles. We benchmark 10 models of varying sizes and architectures through both automatic and human evaluation.
  arXiv  Detail & Related papers  (2024-11-06T18:14:48Z)
• Fair Abstractive Summarization of Diverse Perspectives [103.08300574459783]
  A fair summary should provide a comprehensive coverage of diverse perspectives without underrepresenting certain groups. We first formally define fairness in abstractive summarization as not underrepresenting perspectives of any groups of people. We propose four reference-free automatic metrics by measuring the differences between target and source perspectives.
  arXiv  Detail & Related papers  (2023-11-14T03:38:55Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• News Summarization and Evaluation in the Era of GPT-3 [73.48220043216087]
  We study how GPT-3 compares against fine-tuned models trained on large summarization datasets. We show that not only do humans overwhelmingly prefer GPT-3 summaries, prompted using only a task description, but these also do not suffer from common dataset-specific issues such as poor factuality.
  arXiv  Detail & Related papers  (2022-09-26T01:04:52Z)
• NEWTS: A Corpus for News Topic-Focused Summarization [9.872518517174498]
  This paper introduces the first topical summarization corpus, based on the well-known CNN/Dailymail dataset. We evaluate a range of existing techniques and analyze the effectiveness of different prompting methods.
  arXiv  Detail & Related papers  (2022-05-31T10:01:38Z)
• Enriching and Controlling Global Semantics for Text Summarization [11.037667460077813]
  Transformer-based models have been proven effective in the abstractive summarization task by creating fluent and informative summaries. We introduce a neural topic model empowered with normalizing flow to capture the global semantics of the document, which are then integrated into the summarization model. Our method outperforms state-of-the-art summarization models on five common text summarization datasets.
  arXiv  Detail & Related papers  (2021-09-22T09:31:50Z)
• What's New? Summarizing Contributions in Scientific Literature [85.95906677964815]
  We introduce a new task of disentangled paper summarization, which seeks to generate separate summaries for the paper contributions and the context of the work. We extend the S2ORC corpus of academic articles by adding disentangled "contribution" and "context" reference labels. We propose a comprehensive automatic evaluation protocol which reports the relevance, novelty, and disentanglement of generated outputs.
  arXiv  Detail & Related papers  (2020-11-06T02:23:01Z)
• Multi-Fact Correction in Abstractive Text Summarization [98.27031108197944]
  Span-Fact is a suite of two factual correction models that leverages knowledge learned from question answering models to make corrections in system-generated summaries via span selection. Our models employ single or multi-masking strategies to either iteratively or auto-regressively replace entities in order to ensure semantic consistency w.r.t. the source text. Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-10-06T02:51:02Z)
• Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization [22.611879349101596]
  We evaluate modern neural models for abstractive summarization of relevant article abstracts from systematic reviews. We find that modern summarization systems yield consistently fluent and relevant synopses, but that they are not always factual.
  arXiv  Detail & Related papers  (2020-08-25T22:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.