Retrieving Relations, Detecting Fallacies: A RAG Approach to Political Debate Analysis

• URL: http://arxiv.org/abs/2608.27471v1
• Date: Tue, 18 Aug 2026 09:39:13 GMT
• Title: Retrieving Relations, Detecting Fallacies: A RAG Approach to Political Debate Analysis
• Abstract summary: We introduce a guided retrieval-augmented methodology for fallacy detection and classification.<n>We evaluate our approach on the ElecDeb60to20 benchmark across 42 retrieval configurations and 14 models.<n>Our results show that incorporating external knowledge significantly enhances fallacy detection and classification when retrieval is argumentatively guided.
• Score: 9.600892324769037
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Fallacies are arguments that employ invalid reasoning, making their automatic detection critical in sensitive contexts such as high-stakes political debates, where public opinion is shaped. Spotting a fallacious argument requires contextual knowledge beyond its pure surface text. This entails world knowledge pertaining to the subject matter under discussion, as well as knowledge of the relationships that exist between arguments within the argumentative discourse. Prior work on fallacy analysis has shown that argumentative discourse structure can beneficially improve classification performance. However, such structure is typically encoded only as static classifier features, limiting its flexibility. Building on this intuition while addressing this limitation, we introduce a guided retrieval-augmented methodology for fallacy detection and classification that leverages argumentative relations of support and attack to dynamically steer the extraction of relevant documents. We evaluate our approach on the ElecDeb60to20 benchmark across 42 retrieval configurations and 14 models, performing retrieval over a 15GB knowledge base of collected political-related documents. Our approach improves macro-F1 up to 0.864 for fallacy detection and up to 0.725 for classification over non-retrieval baselines. These results show that incorporating external knowledge significantly enhances fallacy detection and classification when retrieval is argumentatively guided.

Related papers

• Embedding Models for Stance-Aware Argument Retrieval [16.736290000291188]
  We show how dense embedding models (hereafter, models) can serve as the basis of semantic search incorporating this dual assessment.<n>We show experimentally that existing models struggle with asymmetric reasoning, exhibiting a strong bias toward topical overlap while ignoring instructional stance.<n>We thus introduce diagnostic word-ablation metrics to quantify this phenomenon and propose a data-centric solution.
  arXiv  Detail & Related papers  (2026-08-28T12:47:45Z)
• R$^2$-Searcher: Calibrating Retrieval and Reasoning Boundaries for Agentic Search [59.034878430183305]
  R$2$-Searcher explicitly explores and calibrates the retrieval and reasoning boundaries.<n>We show that R$2$-Searcher significantly outperforms state-of-the-art agentic search methods in answer accuracy and retrieval-reasoning quality.
  arXiv  Detail & Related papers  (2026-06-26T19:44:34Z)
• Validating Political Position Predictions of Arguments [3.3571381688392488]
  We construct a large-scale knowledge base of political position predictions using 22 language models.<n>Pointwise evaluation shows moderate human-model agreement, reflecting intrinsic subjectivity.<n>Pairwise validation reveals substantially stronger alignment between human- and model-derived rankings.
  arXiv  Detail & Related papers  (2026-02-20T17:03:44Z)
• Unsupervised dense retrieval with conterfactual contrastive learning [16.679649921935482]
  We propose to improve the robustness of dense retrieval models by enhancing their sensitivity of fine-graned relevance signals.<n>A model achieving sensitivity in this context should exhibit high variances when documents' key passages determining their relevance to queries have been modified.<n>Motivated by causality and counterfactual analysis, we propose a series of counterfactual regularization methods.
  arXiv  Detail & Related papers  (2024-12-30T07:01:34Z)
• CASA: Causality-driven Argument Sufficiency Assessment [79.13496878681309]
  We propose CASA, a zero-shot causality-driven argument sufficiency assessment framework. PS measures how likely introducing the premise event would lead to the conclusion when both the premise and conclusion events are absent. Experiments on two logical fallacy detection datasets demonstrate that CASA accurately identifies insufficient arguments.
  arXiv  Detail & Related papers  (2024-01-10T16:21:18Z)
• Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognition [49.38757847011105]
  computational fallacy recognition faces challenges due to diverse genres, domains, and types of fallacies found in datasets. We aim to enhance existing models for fallacy recognition by incorporating additional context and by leveraging large language models to generate synthetic data. Our evaluation results demonstrate consistent improvements across fallacy types, datasets, and generators.
  arXiv  Detail & Related papers  (2023-11-16T04:17:47Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• A Unifying Framework for Learning Argumentation Semantics [47.84663434179473]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way.<n>Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• Ranking-based Argumentation Semantics Applied to Logical Argumentation (full version) [2.9005223064604078]
  We investigate the behaviour of ranking-based semantics for structured argumentation. We show that a wide class of ranking-based semantics gives rise to so-called culpability measures.
  arXiv  Detail & Related papers  (2023-07-31T15:44:33Z)
• Case-Based Reasoning with Language Models for Classification of Logical Fallacies [3.511369967593153]
  We propose a Case-Based Reasoning method that classifies new cases of logical fallacy. Our experiments indicate that Case-Based Reasoning improves the accuracy and generalizability of language models.
  arXiv  Detail & Related papers  (2023-01-27T17:49:16Z)
• Exploring Discourse Structures for Argument Impact Classification [48.909640432326654]
  This paper empirically shows that the discourse relations between two arguments along the context path are essential factors for identifying the persuasive power of an argument. We propose DisCOC to inject and fuse the sentence-level structural information with contextualized features derived from large-scale language models.
  arXiv  Detail & Related papers  (2021-06-02T06:49:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.