The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions

• URL: http://arxiv.org/abs/2606.23205v1
• Date: Mon, 22 Jun 2026 11:51:17 GMT
• Title: The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions
• Abstract summary: We investigate automatic detection of hidden misconceptions using 20,964 real student responses from the Eedi mathematics platform.<n>We present a graduated assessment that separates answer correctness from method validity, and propose a detect-verify-escalate pipeline.<n>Two deployment modes adapt the pipeline: a teacher dashboard where the system filters a review queue, and an autonomous tutor where flags trigger low-cost formative follow-up.
• Score: 2.6352293843915606
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Automated feedback systems that rely on answer correctness will reinforce, rather than address, misconceptions when students reach the correct answer through flawed reasoning. We investigate automatic detection of these hidden misconceptions using 20,964 real student responses from the Eedi mathematics platform. Fine-tuned classifiers detect only 57% of these hidden misconceptions, and standard ML interventions do not improve on this. An open-weight reasoning model detects 84%, but at realistic prevalence, false alarms outnumber genuine detections roughly 8 to 1. We present a graduated assessment rubric that separates answer correctness from method validity, and propose a detect-verify-escalate pipeline that routes uncertain cases to diagnostic follow-up questions rather than directly to teachers. Two deployment modes adapt the pipeline: a teacher dashboard where the system filters a review queue, and an autonomous tutor where flags trigger low-cost formative follow-up.

Related papers

• AutoVerifier: Residual-Guided Non-Parametric Optimization for Reference-Based Answer Verification [5.990350746292239]
  AutoVerifier learns implicit assumptions from recurring verifier errors.<n>Experiments on four verifier benchmarks demonstrate that AutoVerifier outperforms state-of-the-art verifiers by a large margin.
  arXiv  Detail & Related papers  (2026-08-26T11:06:40Z)
• Teaching a Large Language Model Tutor to Withhold the Answer: A Supervisor Architecture and an Evidence-Driven Method for Tuning Socratic Behavior [0.0]
  An effective large language model (LLM) tutor must often decline to give an answer it could easily produce.<n>We report a deployed tutoring system that enforces answer-withholding as a per-turn, machine-checkable contract.<n>We tune the behavior with an automated evaluation that uses no human subjects.
  arXiv  Detail & Related papers  (2026-08-12T17:35:58Z)
• Cybersecurity Detection Classification with Reasoning-enabled Language Models [50.72675435043546]
  A major issue in Security Operations Centers (SOCs) is alert fatigue.<n>Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.<n>We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections.
  arXiv  Detail & Related papers  (2026-07-30T16:22:13Z)
• Two Regimes of Chain-of-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong [1.2744523252873352]
  Black-box (behavioral) detection of unfaithful CoT against FaithCoT-Bench's human annotations.<n>We find answer correctness structures the problem at every level.
  arXiv  Detail & Related papers  (2026-07-26T04:43:53Z)
• Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation [25.235899757379844]
  Two responses can receive the same final-answer score while differing in whether the trace explicitly flags injected biasing content.<n>We introduce a minimal trace-level diagnostic with two axes: emphsusceptibility (whether the bias breaks a previously correct answer) and emphacknowledgment (whether the trace contains a rubric-defined surface reference to the injected content.
  arXiv  Detail & Related papers  (2026-06-13T05:41:57Z)
• Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation [14.101636730819175]
  Chain-of-thought (CoT) distillation trains a smaller model to imitate a teacher's reasoning trace.<n>We ask whether gains in answer quality are accompanied by improvements in the trace.
  arXiv  Detail & Related papers  (2026-05-27T10:55:24Z)
• Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning [2.6352293843915606]
  We study a failure mode we call the correct answer trap (CAT): models under-detect misconceptions when students reach a correct answer via flawed reasoning.<n>Analysing real student responses from the Eedi mathematics platform, we show that 71% of these failures concentrate in just two question types.
  arXiv  Detail & Related papers  (2026-04-20T10:53:48Z)
• GATES: Self-Distillation under Privileged Context with Consensus Gating [89.62339954332248]
  We study self-distillation in settings where supervision is unreliable.<n>We focus on document-grounded question answering with asymmetric context.<n>We derive supervision online from tutor consensus by sampling multiple document-grounded reasoning traces.
  arXiv  Detail & Related papers  (2026-02-24T05:56:20Z)
• PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering [71.15346406323827]
  We introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification.<n>We find that current verifiers frequently fail to detect derivation flaws.<n>We propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
  arXiv  Detail & Related papers  (2026-02-12T04:45:01Z)
• Probing for Arithmetic Errors in Language Models [86.8227317662622]
  Internal activations in language models can be used to detect arithmetic errors.<n>We show that simple probes can accurately decode both the model's predicted output and the correct answer from hidden states.<n>We train lightweight error detectors that predict model correctness with over 90% accuracy.
  arXiv  Detail & Related papers  (2025-07-16T16:27:50Z)
• Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
  Large language models (LLMs) present an opportunity to scale high-quality personalized education to all. LLMs struggle to precisely detect student's errors and tailor their feedback to these errors. Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
  arXiv  Detail & Related papers  (2024-07-12T10:11:40Z)
• PANACEA: An Automated Misinformation Detection System on COVID-19 [49.83321665982157]
  PANACEA is a web-based misinformation detection system on COVID-19 related claims. It has two modules, fact-checking and rumour detection.
  arXiv  Detail & Related papers  (2023-02-28T21:53:48Z)
• Claim Check-Worthiness Detection as Positive Unlabelled Learning [53.24606510691877]
  Claim check-worthiness detection is a critical component of fact checking systems. We illuminate a central challenge in claim check-worthiness detection underlying all of these tasks. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning.
  arXiv  Detail & Related papers  (2020-03-05T16:06:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.