Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators

• URL: http://arxiv.org/abs/2605.12748v1
• Date: Tue, 12 May 2026 20:55:23 GMT
• Title: Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators
• Abstract summary: Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators.<n>Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction.<n>We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception.
• Score: 55.617099475539305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators. Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction. We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception. Central to our framework is a misconception-contrastive feedback protocol that compares targeted feedback against two controls: misaligned feedback (targeting a different but plausible misconception) and generic feedback (only identifying answer is wrong). We propose Selective Flip Score (SFS), which quantifies how much more often a simulator flips its answer under targeted feedback than under contrastive controls. Across seven LLMs (4B-120B), multiple datasets, and prompting strategies, simulators exhibit near-zero SFS, correcting their answers at similarly high rates regardless of feedback relevance. Further analyses reveal a sycophantic failure mode: models behave less like students with misconceptions but more like problem-solvers who treat any corrective signal as a cue to abandon the simulated belief and re-solve from internal knowledge. To address this, we develop a post-training pipeline spanning supervised fine-tuning (SFT), preference optimization, and reinforcement learning (RL) with an SFS-aligned reward; SFT yields notable gains up to +0.56, and SFS-aligned RL provides more consistent improvements than preference optimization. Our results establish misconception faithfulness as a challenging yet trainable property, motivating a shift from static output matching toward interactive, belief-aware student modeling.

Related papers

• One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL [75.34038228759182]
  Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.<n>We show that this approach systematically fails to generalize, and trace the failure to simulator collapse.<n>We propose two complementary solutions, one at inference time and one at training time.
  arXiv  Detail & Related papers  (2026-08-12T16:55:50Z)
• A Classroom Study of LLM-Generated Feedback Intervention in Introductory Programming [1.0241350366020698]
  We present a large-scale classroom study in which AI-generated feedback was deployed in an introductory Python programming course.<n>Students received one of three feedback conditions on incorrect submissions: natural language hints, AI-generated failing test cases, or no AI feedback.<n>We find that natural language feedback is significantly associated with higher completion rates and faster convergence to correct solutions.
  arXiv  Detail & Related papers  (2026-06-07T19:55:51Z)
• Reinforcing Human Behavior Simulation via Verbal Feedback [105.68319269895653]
  We present DITTO, a model trained by treating verbal feedback as a first-class signal in reinforcement learning.<n>We also introduce SOUL, a unified benchmark and training data suite spanning 10 tasks across six categories: Theory of Mind, character role play, social skill, learner simulation, user simulation, and persona simulation.<n>DitTO achieves an average 36% improvement over the base model and exceeds GPT-5.4 on 6 of 10 SOUL benchmarks.
  arXiv  Detail & Related papers  (2026-05-19T21:23:24Z)
• Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization [65.37179698521766]
  Vision-Language-Action (VLA) models show strong promise for generalist robot control.<n>It remains unclear whether -- and under what conditions -- the standard "scale data" recipe translates to robotics.<n>We present a systematic, controlled study of VLA scaling that revisits core training choices for pretraining across diverse robots.
  arXiv  Detail & Related papers  (2026-02-10T12:25:43Z)
• Expanding the Capabilities of Reinforcement Learning via Text Feedback [49.561885700139676]
  We formalize a multi-turn RL setup, RL from Text Feedback (RLTF), where text feedback is available during training but not at inference.<n>To do this, we propose two methods: Self Distillation (RLTF-SD), which trains the single-turn policy to match its own feedback-conditioned second-turn generations; and Feedback Modeling (RLTF-FM), which predicts the feedback as an auxiliary objective.<n>Our results show that both methods consistently outperform strong baselines across benchmarks.
  arXiv  Detail & Related papers  (2026-02-02T18:56:56Z)
• SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors [58.87134689752605]
  We introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation.<n>We show that even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size.<n>We demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning.
  arXiv  Detail & Related papers  (2025-10-20T13:14:38Z)
• LANPO: Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMs [73.27182315028021]
  LANPO is a framework that cleanly separates the roles of feedback: language guides exploration, while numerical rewards drive optimization.<n>Our work provides a robust method for integrating historical experiences into the LLM RL loop, creating more effective and data-efficient learning agents.
  arXiv  Detail & Related papers  (2025-10-18T15:51:19Z)
• SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction [38.7828715471869]
  We present SMART (Simulated Students Aligned with IRT), a novel method for aligning simulated students with instructed ability.<n>We show that SMART outperforms other item difficulty prediction methods by leveraging its improved ability alignment.
  arXiv  Detail & Related papers  (2025-07-07T15:41:38Z)
• Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback [35.13591109493438]
  We show that solver models consistently show resistance to feedback, a limitation that we term Feedback Friction.<n>We analyze Feedback Friction and find that models' confidence on specific questions, measured by semantic entropy, predicts feedback resistance.
  arXiv  Detail & Related papers  (2025-06-13T16:31:51Z)
• Flow Matching for Posterior Inference with Simulator Feedback [20.315933488318986]
  Flow-based generative modeling is a powerful tool for solving inverse problems in physical sciences. We propose to refine flows with additional control signals based on a simulator. We demonstrate that including feedback from the simulator improves the accuracy by $53%$, making it competitive with traditional techniques.
  arXiv  Detail & Related papers  (2024-10-29T22:26:39Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.