Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation

• URL: http://arxiv.org/abs/2606.01020v1
• Date: Sun, 31 May 2026 05:24:57 GMT
• Title: Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation
• Abstract summary: We introduce LFTutor, an intelligent tutoring system which uses Large Language Models to tutor laypeople and help them learn about logical fallacies.<n>Through both automatic and human evaluations, we demonstrate that LFTutor significantly outperforms baseline LLMs.<n>This work highlights the promise of combining LLMs with pedagogical scaffolding to foster critical thinking and argument literacy in the age of AI.
• Score: 49.177047539037
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifying logical fallacies in everyday discourse is challenging for many people. This challenge is amplified in the era of Large Language Models (LLMs), where malicious agents can deploy fallacious arguments to disseminate misinformation at scale. In this work, we explore the potential of LLMs as part of the solution. We introduce LFTutor, an intelligent tutoring system which uses LLMs to tutor laypeople and help them learn about logical fallacies. LFTutor integrates intent-driven Socratic questioning and critical argumentation principles to actively engage learners to reflect on their reasoning. Through both automatic and human evaluations, we demonstrate that LFTutor significantly outperforms baseline LLMs lacking these pedagogical strategies. This work highlights the promise of combining LLMs with pedagogical scaffolding to foster critical thinking and argument literacy in the age of AI.

Related papers

• Truth or Sophistry? LoFa: A Benchmark for LLM Robustness Against Logical Fallacies [51.39569894108518]
  Large Language Models (LLMs) exhibit strong semantic capabilities, yet their resilience to manipulative linguistic patterns such as logical fallacies remains underexplored.<n>We introduce LoFa (Logical Fallacy), a benchmark for evaluating LLM robustness against fallacies.<n>We also propose Logical Fallacy Resistance at k (LFR@k), a metric that quantifies resistance to fallacious attacks.
  arXiv  Detail & Related papers  (2026-06-30T02:17:45Z)
• Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident [0.8564319625930894]
  Large language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice reasoning and metacognitive judgments?<n>We evaluate LLMs as novices using 630 think-aloud utterances from chemistry tutoring problems with problem-solving logs of student hint use, attempts, and problem context.<n>We compare LLM-generated reasoning to human learner utterances under minimal and extended contextual prompting, and assess the models' ability to predict step-level learner success.
  arXiv  Detail & Related papers  (2026-02-01T04:46:38Z)
• Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding [40.2816930342597]
  Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks. But they still struggle with some complicated reasoning tasks including logical reasoning. We propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW in this paper.
  arXiv  Detail & Related papers  (2024-04-04T08:38:03Z)
• LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models [63.14196038655506]
  We introduce LogicAsker, a novel approach for evaluating and enhancing the logical reasoning capabilities of large language models (LLMs) Our methodology reveals significant gaps in LLMs' learning of logical rules, with identified reasoning failures ranging from 29% to 90% across different models. We leverage these findings to construct targeted demonstration examples and fine-tune data, notably enhancing logical reasoning in models like GPT-4o by up to 5%.
  arXiv  Detail & Related papers  (2024-01-01T13:53:53Z)
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning [73.77088902676306]
  We take a closer look at the self-verification abilities of large language models (LLMs) in the context of logical reasoning. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods.
  arXiv  Detail & Related papers  (2023-11-14T07:13:10Z)
• Language Models can be Logical Solvers [99.40649402395725]
  We introduce LoGiPT, a novel language model that directly emulates the reasoning processes of logical solvers. LoGiPT is fine-tuned on a newly constructed instruction-tuning dataset derived from revealing and refining the invisible reasoning process of deductive solvers.
  arXiv  Detail & Related papers  (2023-11-10T16:23:50Z)
• Improving Large Language Models in Event Relation Logical Prediction [33.88499005859982]
  Event relation extraction is a challenging task that demands thorough semantic understanding and rigorous logical reasoning. In this paper, we conduct an in-depth investigation to systematically explore the capability of LLMs in understanding and applying event relation logic. Our study reveals that LLMs are not logically consistent reasoners, which results in their suboptimal performance on tasks that need rigorous reasoning.
  arXiv  Detail & Related papers  (2023-10-13T14:53:06Z)
• Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models [56.34029644009297]
  Large language models (LLMs) have demonstrated the ability to overcome various limitations of formal Knowledge Representation (KR) systems. LLMs excel most in abductive reasoning, followed by deductive reasoning, while they are least effective at inductive reasoning. We study single-task training, multi-task training, and "chain-of-thought" knowledge distillation fine-tuning technique to assess the performance of model.
  arXiv  Detail & Related papers  (2023-10-02T01:00:50Z)
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate [85.3444184685235]
  We propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution. Our framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation.
  arXiv  Detail & Related papers  (2023-05-30T15:25:45Z)
• Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate [19.887103433032774]
  Large language models (LLMs) have shown impressive performance in complex reasoning tasks. This work explores testing LLMs' reasoning by engaging with them in a debate-like conversation. We find that despite their impressive performance, LLMs like ChatGPT cannot maintain their beliefs in truth for a significant portion of examples.
  arXiv  Detail & Related papers  (2023-05-22T15:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.