An Empirical Analysis of Diversity in Argument Summarization

• URL: http://arxiv.org/abs/2402.01535v2
• Date: Wed, 14 Feb 2024 10:51:08 GMT
• Title: An Empirical Analysis of Diversity in Argument Summarization
• Authors: Michiel van der Meer, Piek Vossen, Catholijn M. Jonker, Pradeep K. Murukannaiah
• Abstract summary: We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis.
• Score: 4.128725138940779
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Presenting high-level arguments is a crucial task for fostering participation in online societal discussions. Current argument summarization approaches miss an important facet of this task -- capturing diversity -- which is important for accommodating multiple perspectives. We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis, which shows how these approaches struggle to (1) represent arguments shared by few people, (2) deal with data from various sources, and (3) align with subjectivity in human-provided annotations. We find that both general-purpose LLMs and dedicated KPA models exhibit this behavior, but have complementary strengths. Further, we observe that diversification of training data may ameliorate generalization. Addressing diversity in argument summarization requires a mix of strategies to deal with subjectivity.

Related papers

• PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis [74.41260927676747]
  This paper bridges the gaps by introducing a multimodal conversational Sentiment Analysis (ABSA) To benchmark the tasks, we construct PanoSent, a dataset annotated both manually and automatically, featuring high quality, large scale, multimodality, multilingualism, multi-scenarios, and covering both implicit and explicit sentiment elements. To effectively address the tasks, we devise a novel Chain-of-Sentiment reasoning framework, together with a novel multimodal large language model (namely Sentica) and a paraphrase-based verification mechanism.
  arXiv  Detail & Related papers  (2024-08-18T13:51:01Z)
• Overview of PerpectiveArg2024: The First Shared Task on Perspective Argument Retrieval [56.66761232081188]
  We present a novel dataset covering demographic and socio-cultural (socio) variables, such as age, gender, and political attitude, representing minority and majority groups in society. We find substantial challenges in incorporating perspectivism, especially when aiming for personalization based solely on the text of arguments without explicitly providing socio profiles. While we bootstrap perspective argument retrieval, further research is essential to optimize retrieval systems to facilitate personalization and reduce polarization.
  arXiv  Detail & Related papers  (2024-07-29T03:14:57Z)
• PAKT: Perspectivized Argumentation Knowledge Graph and Tool for Deliberation Analysis (with Supplementary Materials) [18.436817251174357]
  We propose PAKT, a Perspectivized Argumentation Knowledge Graph and Tool. The graph structures the argumentative space across diverse topics, where arguments are divided into premises and conclusions. We show how to construct PAKT and conduct case studies on the obtained multifaceted argumentation graph.
  arXiv  Detail & Related papers  (2024-04-16T13:47:19Z)
• SocraSynth: Multi-LLM Reasoning with Conditional Statistics [2.5200794639628032]
  Large language models (LLMs) face criticisms for biases, hallucinations, and a lack of reasoning capability. This paper introduces Socra Synth, a multi-LLM agent reasoning platform developed to mitigate these issues.
  arXiv  Detail & Related papers  (2024-01-19T07:16:21Z)
• GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives [18.574420136899978]
  We propose GRASP, a comprehensive disagreement analysis framework to measure group association in perspectives among different rater sub-groups. Our framework reveals specific rater groups that have significantly different perspectives than others on certain tasks, and helps identify demographic axes that are crucial to consider in specific task contexts.
  arXiv  Detail & Related papers  (2023-11-09T00:12:21Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization [73.89239820192894]
  We argue that automated counterfactual generation should regard several aspects of the produced adversarial instances. We present a novel framework for the generation of counterfactual examples.
  arXiv  Detail & Related papers  (2022-05-20T15:02:53Z)
• Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis [17.875273745811775]
  We develop a method for automatic extraction of key points, which enables fully automatic analysis. We demonstrate that the applicability of key point analysis goes well beyond argumentation data. An additional contribution is an in-depth evaluation of argument-to-key point matching models.
  arXiv  Detail & Related papers  (2020-10-11T23:01:51Z)
• Simultaneous Relevance and Diversity: A New Recommendation Inference Approach [81.44167398308979]
  We propose a new approach, which extends the general collaborative filtering (CF) by introducing a new way of CF inference, negative-to-positive. Our approach is applicable to a wide range of recommendation scenarios/use-cases at various sophistication levels. Our analysis and experiments on public datasets and real-world production data show that our approach outperforms existing methods on relevance and diversity simultaneously.
  arXiv  Detail & Related papers  (2020-09-27T22:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.