Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs

• URL: http://arxiv.org/abs/2510.09970v1
• Date: Sat, 11 Oct 2025 03:02:11 GMT
• Title: Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
• Authors: Olivia Peiyu Wang, Tashvi Bansal, Ryan Bai, Emily M. Chui, Leilani H. Gilpin,
• Abstract summary: Large Language Models (LLMs) suffer from critical reasoning gaps.<n>This limitation stems from their default System 1 processing, which is fast and intuitive.<n>We explore a low-cost, instruction-based intervention to bridge this gap.
• Score: 2.3488056916440856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) suffer from critical reasoning gaps, including a tendency to hallucinate and poor accuracy in classifying logical fallacies. This limitation stems from their default System 1 processing, which is fast and intuitive, whereas reliable reasoning requires the deliberate, effortful System 2 approach (Kahneman, 2011; Li et al., 2025). Since full System 2 training is often prohibitively expensive, we explore a low-cost, instruction-based intervention to bridge this gap. Our methodology introduces a novel stepwise instruction dataset that decomposes fallacy classification into a series of atomic procedural steps (simple binary questions). We further augment this with a final verification step where models consult a relational knowledge graph of related fallacies. This procedural, rule-based intervention yields a significant improvement in LLM logical fallacy classification. Crucially, the approach also provides enhanced transparency into the LLMs' decision-making, highlighting a practical pathway for Neuro-symbolic architectures to address LLM reasoning deficits.

Related papers

• PathMind: A Retrieve-Prioritize-Reason Framework for Knowledge Graph Reasoning with Large Language Models [18.868091458925928]
  We propose PathMind, a framework designed to enhance faithful and interpretable reasoning by selectively guiding LLMs with important reasoning paths.<n>Extensive experiments on benchmark datasets demonstrate that PathMind consistently outperforms competitive baselines.
  arXiv  Detail & Related papers  (2025-11-18T08:45:16Z)
• Prejudge-Before-Think: Enhancing Large Language Models at Test-Time by Process Prejudge Reasoning [13.865037985388575]
  We introduce a new emphprocess prejudge strategy in LLM reasoning.<n>We define a prejudge node in the rationale, which represents a reasoning step.<n>We present an automated reasoning framework with a dynamic tree-searching strategy.
  arXiv  Detail & Related papers  (2025-04-18T06:42:30Z)
• R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization [86.32257216965229]
  We propose a new online reinforcement learning framework that enables MLLMs to self-improve reasoning ability via simple, effective and dense step-wise rewarding.<n>StepGRPO introduces two novel rule-based reasoning rewards: Step-wise Reasoning Accuracy Reward (StepRAR) and Step-wise Reasoning Validity Reward (StepRVR)<n>With the proposed StepGRPO, we introduce R1-VL, a series of MLLMs with outstanding capabilities in step-by-step reasoning.
  arXiv  Detail & Related papers  (2025-03-17T08:51:44Z)
• Stepwise Informativeness Search for Efficient and Effective LLM Reasoning [49.85349030928302]
  Recent studies show that Large Language Models (LLMs) tend to lose focus over the middle of long contexts.<n>We propose guiding LLMs to generate more accurate and concise step-by-step rationales.
  arXiv  Detail & Related papers  (2025-02-21T09:39:27Z)
• Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback [94.25162866972077]
  Step-KTO is a training framework that combines process-level and outcome-level binary feedback.<n>Our experiments show that Step-KTO significantly improves both final answer accuracy and the quality of intermediate reasoning steps.
  arXiv  Detail & Related papers  (2025-01-18T15:38:03Z)
• System-2 Mathematical Reasoning via Enriched Instruction Tuning [13.672967091915181]
  Enriched Instruction Tuning (EIT) is a method that enriches existing human-annotated mathematical datasets by synergizing human and AI feedback.<n>EIT achieves an accuracy of 84.1% on GSM8K and 32.5% on MATH, surpassing state-of-the-art fine-tuning and prompting methods.
  arXiv  Detail & Related papers  (2024-12-22T10:49:27Z)
• Disentangling Memory and Reasoning Ability in Large Language Models [97.26827060106581]
  We propose a new inference paradigm that decomposes the complex inference process into two distinct and clear actions.<n>Our experiment results show that this decomposition improves model performance and enhances the interpretability of the inference process.
  arXiv  Detail & Related papers  (2024-11-20T17:55:38Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making.<n>We present a process-based benchmark MR-Ben that demands a meta-reasoning skill.<n>Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning [73.77088902676306]
  We take a closer look at the self-verification abilities of large language models (LLMs) in the context of logical reasoning. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods.
  arXiv  Detail & Related papers  (2023-11-14T07:13:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.