Polarity Calibration for Opinion Summarization

• URL: http://arxiv.org/abs/2404.01706v1
• Date: Tue, 2 Apr 2024 07:43:12 GMT
• Title: Polarity Calibration for Opinion Summarization
• Authors: Yuanyuan Lei, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang, Ruihong Huang, Dong Yu,
• Abstract summary: Polarity calibration aims to align the polarity of output summary with that of input text. We evaluate our model on two types of opinions summarization tasks: summarizing product reviews and political opinions articles.
• Score: 46.83053173308394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Opinion summarization is automatically generating summaries from a variety of subjective information, such as product reviews or political opinions. The challenge of opinions summarization lies in presenting divergent or even conflicting opinions. We conduct an analysis of previous summarization models, which reveals their inclination to amplify the polarity bias, emphasizing the majority opinions while ignoring the minority opinions. To address this issue and make the summarizer express both sides of opinions, we introduce the concept of polarity calibration, which aims to align the polarity of output summary with that of input text. Specifically, we develop a reinforcement training approach for polarity calibration. This approach feeds the polarity distance between output summary and input text as reward into the summarizer, and also balance polarity calibration with content preservation and language naturality. We evaluate our Polarity Calibration model (PoCa) on two types of opinions summarization tasks: summarizing product reviews and political opinions articles. Automatic and human evaluation demonstrate that our approach can mitigate the polarity mismatch between output summary and input text, as well as maintain the content semantic and language quality.

Related papers

• Summarization of Opinionated Political Documents with Varied Perspectives [11.399915001583059]
  Models capable of generating accurate summaries of diverse perspectives can help reduce such polarization by exposing users to alternative perspectives. We introduce a novel dataset and task for independently summarizing each political perspective in a set of passages from opinionated news articles. We benchmark 10 models of varying sizes and architectures through both automatic and human evaluation.
  arXiv  Detail & Related papers  (2024-11-06T18:14:48Z)
• P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models [57.571395694391654]
  We find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries. We propose P3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. Experiments on three news summarization datasets demonstrate that P3SUM outperforms state-of-the-art summarization systems.
  arXiv  Detail & Related papers  (2023-11-16T10:14:28Z)
• Mitigating Framing Bias with Polarity Minimization Loss [56.24404488440295]
  Framing bias plays a significant role in exacerbating political polarization by distorting the perception of actual events. We propose a new loss function that encourages the model to minimize the polarity difference between the polarized input articles to reduce framing bias.
  arXiv  Detail & Related papers  (2023-11-03T09:50:23Z)
• Bias in News Summarization: Measures, Pitfalls and Corpora [4.917075909999548]
  We introduce definitions for biased behaviours in summarization models, along with practical operationalizations. We measure gender bias in English summaries generated by both purpose-built summarization models and general purpose chat models. We find content selection in single document summarization to be largely unaffected by gender bias, while hallucinations exhibit evidence of bias.
  arXiv  Detail & Related papers  (2023-09-14T22:20:27Z)
• Biased processing and opinion polarization: experimental refinement of argument communication theory in the context of the energy debate [0.0]
  We combine experimental research on biased argument processing with a computational theory of group deliberation. The experiment reveals a strong tendency to consider arguments aligned with the current attitude more persuasive and to downgrade those speaking against it. We derive a mathematical model that allows to relate the strength of biased processing to expected attitude changes.
  arXiv  Detail & Related papers  (2022-12-20T09:35:53Z)
• Human-in-the-loop Abstractive Dialogue Summarization [61.4108097664697]
  We propose to incorporate different levels of human feedback into the training process. This will enable us to guide the models to capture the behaviors humans care about for summaries.
  arXiv  Detail & Related papers  (2022-12-19T19:11:27Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• Learning Opinion Summarizers by Selecting Informative Reviews [81.47506952645564]
  We collect a large dataset of summaries paired with user reviews for over 31,000 products, enabling supervised training. The content of many reviews is not reflected in the human-written summaries, and, thus, the summarizer trained on random review subsets hallucinates. We formulate the task as jointly learning to select informative subsets of reviews and summarizing the opinions expressed in these subsets.
  arXiv  Detail & Related papers  (2021-09-09T15:01:43Z)
• Detecting Polarized Topics in COVID-19 News Using Partisanship-aware Contextualized Topic Embeddings [3.9761027576939405]
  Growing polarization of the news media has been blamed for fanning disagreement, controversy and even violence. We propose Partisanship-aware Contextualized Topic Embeddings (PaCTE), a method to automatically detect polarized topics from partisan news sources.
  arXiv  Detail & Related papers  (2021-04-15T23:05:52Z)
• Political Depolarization of News Articles Using Attribute-aware Word Embeddings [7.411577497708497]
  Political polarization in the US is on the rise. This polarization negatively affects the public sphere by contributing to the creation of ideological echo chambers. We introduce a framework for depolarizing news articles.
  arXiv  Detail & Related papers  (2021-01-05T07:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.