Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation

• URL: http://arxiv.org/abs/2604.19331v1
• Date: Tue, 21 Apr 2026 11:01:53 GMT
• Title: Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation
• Abstract summary: Large Language Models (LLMs) have been shown to enable automated summarisation at scale.<n>Existing automated summarisation metrics have been shown to correlate poorly with human judgements of consistency.<n>We propose a formal framework for evaluating parliamentary debate summaries that grounds argument structures in the contested proposals up for debate.
• Score: 12.06334493909551
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how policy is debated and justified in parliament is a fundamental aspect of the democratic process. However, the volume and complexity of such debates mean that outside audiences struggle to engage. Meanwhile, Large Language Models (LLMs) have been shown to enable automated summarisation at scale. While summaries of debates can make parliamentary procedures more accessible, evaluating whether these summaries faithfully communicate argumentative content remains challenging. Existing automated summarisation metrics have been shown to correlate poorly with human judgements of consistency (i.e., faithfulness or alignment between summary and source). In this work, we propose a formal framework for evaluating parliamentary debate summaries that grounds argument structures in the contested proposals up for debate. Our novel approach, driven by computational argumentation, focuses the evaluation on formal properties concerning the faithful preservation of the reasoning presented to justify or oppose policy outcomes. We demonstrate our methods using a case-study of debates from the European Parliament and associated LLM-driven summaries.

Related papers

• A Theory of Post-hoc Debate Judgement [29.17703958536647]
  We develop and test a novel theory of debate judgement applicable to all settings where agents engage in debates.<n>We identify a number of formal properties that debate judgement may be required to satisfy in general, as concerns, robustness, groundedness and explainability.<n>We show that the two methods give similar accuracy performances but the former may lack formal guarantees.
  arXiv  Detail & Related papers  (2026-08-19T15:03:44Z)
• LLM-based Argument Mining meets Argumentation and Description Logics: a Unified Framework for Reasoning about Debates [18.314315278861073]
  Large Language Models (LLMs) achieve strong performance in analyzing and generating text.<n>They struggle with explicit, transparent, and verifiable reasoning over complex texts such as those containing debates.<n>We propose a framework that integrates learning-based argument mining with quantitative reasoning.
  arXiv  Detail & Related papers  (2026-03-03T11:06:23Z)
• SAD: A Large-Scale Strategic Argumentative Dialogue Dataset [60.33125467375306]
  In practice, argumentation is often realized as multi-turn dialogue.<n>We present the first large-scale textbfStrategic textbfArgumentative textbfDialogue dataset, consisting of 392,822 examples.
  arXiv  Detail & Related papers  (2026-01-12T11:11:37Z)
• R-Debater: Retrieval-Augmented Debate Generation through Argumentative Memory [18.007379464461312]
  We present R-Debater, an agentic framework for generating multi-turn debates built on argumentative memory.<n>R-Debater integrates a debate knowledge base for retrieving case-like evidence and prior debate moves with a role-based agent that composes coherent utterances across turns.
  arXiv  Detail & Related papers  (2025-12-31T07:33:12Z)
• ARQUSUMM: Argument-aware Quantitative Summarization of Online Conversations [11.33923212079359]
  We propose a novel task of argument-aware quantitative summarization to reveal the claim-reason structure of arguments in conversations.<n>For quantitative summarization, ARQUSUMM employs argument structure-aware clustering algorithms to aggregate arguments and quantify their support.
  arXiv  Detail & Related papers  (2025-11-21T06:37:32Z)
• Implicit Reasoning in Large Language Models: A Comprehensive Survey [67.53966514728383]
  Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks.<n>Recent studies have shifted attention from explicit chain-of-thought prompting toward implicit reasoning.<n>This survey introduces a taxonomy centered on execution paradigms, shifting the focus from representational forms to computational strategies.
  arXiv  Detail & Related papers  (2025-09-02T14:16:02Z)
• Identifying Algorithmic and Domain-Specific Bias in Parliamentary Debate Summarisation [13.982513887272683]
  This paper explores the use of large language models (LLMs) to summarise plenary debates from the European Parliament.<n>We propose a structured, multi-stage summarisation framework that improves textual coherence and content fidelity.<n>We find evidence of consistent positional and partisan biases, with certain speakers systematically under-represented or misattributed.
  arXiv  Detail & Related papers  (2025-07-16T11:49:33Z)
• Few-shot Policy (de)composition in Conversational Question Answering [54.259440408606515]
  We propose a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting.<n>We show that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies.<n>We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning.
  arXiv  Detail & Related papers  (2025-01-20T08:40:15Z)
• Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM [51.43102092480804]
  Debatrix is an automated debate judge based on Large Language Models (LLMs) To align with real-world debate scenarios, we introduced the PanelBench benchmark, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation.
  arXiv  Detail & Related papers  (2024-03-12T18:19:47Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• A Unifying Framework for Learning Argumentation Semantics [47.84663434179473]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way.<n>Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• DEBACER: a method for slicing moderated debates [55.705662163385966]
  Partitioning debates into blocks with the same subject is essential for understanding. We propose a new algorithm, DEBACER, which partitions moderated debates.
  arXiv  Detail & Related papers  (2021-12-10T10:39:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.