When LLMs Learn to be Students: The SOEI Framework for Modeling and Evaluating Virtual Student Agents in Educational Interaction

• URL: http://arxiv.org/abs/2410.15701v2
• Date: Thu, 22 May 2025 10:19:15 GMT
• Title: When LLMs Learn to be Students: The SOEI Framework for Modeling and Evaluating Virtual Student Agents in Educational Interaction
• Authors: Yiping Ma, Shiyu Hu, Xuchen Li, Yipei Wang, Yuqing Chen, Shiqing Liu, Kang Hao Cheong,
• Abstract summary: We propose the SOEI framework for constructing and evaluating personality-aligned Virtual Student Agents (LVSAs) in classroom scenarios.<n>We generate five LVSAs based on Big Five traits through LoRA fine-tuning and expert-informed prompt design.<n>Our results provide: (1) an educationally and psychologically grounded generation pipeline for LLM-based student agents; (2) a hybrid, scalable evaluation framework for behavioral realism; and (3) empirical insights into the pedagogical utility of LVSAs in shaping instructional adaptation.
• Score: 12.070907646464537
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large language models (LLMs) have enabled intelligent tutoring systems, yet the development of LLM-based Virtual Student Agents (LVSAs) remains underexplored. Such agents are essential for teacher-facing applications, where simulating diverse learner traits can support adaptive instruction and pedagogical skill development. However, current methods lack principled personality modeling, scalable evaluation of behavioral consistency, and empirical validation in interactive teaching settings. We propose the SOEI framework, a structured pipeline comprising Scene, Object, Evaluation, and Interaction, for constructing and evaluating personality-aligned LVSAs in classroom scenarios. Leveraging Chinese language instruction as a cognitively and emotionally rich testbed, we generate five LVSAs based on Big Five traits through LoRA fine-tuning and expert-informed prompt design. Their behavioral realism and personality coherence are assessed using a hybrid human & GPT-4 evaluation and a multi-dimensional annotation protocol. Through controlled experiments with real pre-service teachers, we demonstrate that LVSAs can elicit adaptive teaching strategies and maintain trait-consistent behavior across multi-turn dialogues. Our results provide: (1) an educationally and psychologically grounded generation pipeline for LLM-based student agents; (2) a hybrid, scalable evaluation framework for behavioral realism; and (3) empirical insights into the pedagogical utility of LVSAs in shaping instructional adaptation. By embedding LVSAs into both generative modeling and human-in-the-loop teaching, SOEI bridges AI for Education (AI4Edu) and Education for AI (Edu4AI), positioning classroom interaction as a rigorous testbed for controllability, personality alignment, and human-likeness in large language models.

Related papers

• A Theory of Adaptive Scaffolding for LLM-Based Pedagogical Agents [3.6084561124905297]
  Large language models (LLMs) present new opportunities for creating pedagogical agents that engage in meaningful dialogue to support student learning.<n>We propose a framework that combines Evidence-Centered Design with Social Cognitive Theory for adaptive scaffolding in LLM-based agents focused on STEM+C learning.<n>Our findings show that Inquizzitor delivers high-quality assessment and interaction aligned with core learning theories, offering teachers effective guidance that students value.
  arXiv  Detail & Related papers  (2025-08-02T21:58:32Z)
• Automated Feedback on Student-Generated UML and ER Diagrams Using Large Language Models [39.58317527488534]
  We introduce DUET (Diamatic & ER Tutor), a prototype of an LLM-based tool.<n>It converts a reference diagram and a student-submitted diagram into a textual representation and provides structured feedback based on the differences.<n>It uses a multi-stage LLM pipeline to compare diagrams and generate reflective feedback.<n>It enables analytical insights for educators, aiming to foster self-directed learning and inform instructional strategies.
  arXiv  Detail & Related papers  (2025-07-31T11:49:01Z)
• A Large-Scale Real-World Evaluation of LLM-Based Virtual Teaching Assistant [19.026750427901423]
  Virtual Teaching Assistants (VTAs) have the potential to enhance student learning by providing instant feedback and facilitating multi-turn interactions.<n>This study develops an LLM-based VTA and deploys it in an AI programming course with 477 graduate students.<n>We assess the feasibility of deploying VTAs in real-world classrooms and identify key challenges for broader adoption.
  arXiv  Detail & Related papers  (2025-06-20T10:59:57Z)
• Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study [50.065744358362345]
  Large language models (LLMs) have shown impressive capabilities across tasks such as mathematics, coding, and reasoning.<n>Yet their learning ability, which is crucial for adapting to dynamic environments and acquiring new knowledge, remains underexplored.
  arXiv  Detail & Related papers  (2025-06-16T13:24:50Z)
• Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style [16.985943868964394]
  Effective teaching requires adapting instructional strategies to accommodate the diverse cognitive and behavioral profiles of students.<n>This paper introduces a novel simulation framework that integrates heterogeneous student agents with a self-optimizing teacher agent.<n>Our results highlight the potential of LLM-driven simulations to inform adaptive teaching practices and provide a testbed for training human educators in data-driven environments.
  arXiv  Detail & Related papers  (2025-05-25T14:45:35Z)
• From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning [76.09281171131941]
  Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy.<n>We propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors.
  arXiv  Detail & Related papers  (2025-05-21T15:00:07Z)
• Exploring LLM-based Student Simulation for Metacognitive Cultivation [33.346260553878984]
  We propose a pipeline for automatically generating and filtering high-quality simulated student agents. Our work paves the way for broader applications in personalized learning and educational assessment.
  arXiv  Detail & Related papers  (2025-02-17T11:12:47Z)
• Position: LLMs Can be Good Tutors in Foreign Language Education [87.88557755407815]
  We argue that large language models (LLMs) have the potential to serve as effective tutors in foreign language education (FLE) Specifically, LLMs can play three critical roles: (1) as data enhancers, improving the creation of learning materials or serving as student simulations; (2) as task predictors, serving as learner assessment or optimizing learning pathway; and (3) as agents, enabling personalized and inclusive education.
  arXiv  Detail & Related papers  (2025-02-08T06:48:49Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, integrating psychology-grounded principles of personality: social practice, consistency, and dynamic development. We incorporate personality traits directly into the model parameters, enhancing the model's resistance to induction, promoting consistency, and supporting the dynamic evolution of personality.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Simulating Classroom Education with LLM-Empowered Agents [52.62324491261461]
  SimClass is a multi-agent classroom simulation framework involving user participation. We recognize representative class roles and introduce a novel class control mechanism for automatic classroom teaching. We demonstrate that LLMs can simulate traditional classroom interaction patterns effectively while enhancing user's experience.
  arXiv  Detail & Related papers  (2024-06-27T14:51:07Z)
• Toward In-Context Teaching: Adapting Examples to Students' Misconceptions [54.82965010592045]
  We introduce a suite of models and evaluation methods we call AdapT. AToM is a new probabilistic model for adaptive teaching that jointly infers students' past beliefs and optimize for the correctness of future beliefs. Our results highlight both the difficulty of the adaptive teaching task and the potential of learned adaptive models for solving it.
  arXiv  Detail & Related papers  (2024-05-07T17:05:27Z)
• Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning [25.90420385230675]
  The pursuit of personalized education has led to the integration of Large Language Models (LLMs) in developing intelligent tutoring systems. Our research uncovers a fundamental challenge in this approach: the Student Data Paradox'' This paradox emerges when LLMs, trained on student data to understand learner behavior, inadvertently compromise their own factual knowledge and reasoning abilities.
  arXiv  Detail & Related papers  (2024-04-23T15:57:55Z)
• Personality-aware Student Simulation for Conversational Intelligent Tutoring Systems [34.760230622675365]
  Intelligent Tutoring Systems (ITSs) can provide personalized and self-paced learning experience. The emergence of large language models (LLMs) further enables better human-machine interaction. LLMs can produce diverse student responses according to the given language ability and personality traits.
  arXiv  Detail & Related papers  (2024-04-10T06:03:13Z)
• MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education [19.549398447035376]
  Large language models (LLMs) have recently demonstrated strong capability in both modeling mathematical problems and simulating characters. We present MATHVC, the very first LLM-powered virtual classroom containing multiple LLM-simulated student characters. We propose three innovations: integrating MM domain knowledge into the simulation, defining a symbolic schema as the ground for character simulation, and designing a meta planner at the platform level to drive the conversational procedure.
  arXiv  Detail & Related papers  (2024-04-10T03:35:51Z)
• EduAgent: Generative Student Agents in Learning [15.215078619481732]
  Student simulation in online education is important to address dynamic learning behaviors of students with diverse backgrounds. Existing simulation models based on deep learning usually need massive training data, lacking prior knowledge in educational contexts. This work proposes EduAgent, a novel generative agent framework incorporating cognitive prior knowledge.
  arXiv  Detail & Related papers  (2024-03-23T18:19:17Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs [13.262711792955377]
  This study explores the effectiveness of Large Language Models (LLMs) for automated essay scoring. We propose an open-source LLM-based AES system, inspired by the dual-process theory. We find that our system not only automates the grading process but also enhances the performance and efficiency of human graders.
  arXiv  Detail & Related papers  (2024-01-12T07:50:10Z)
• Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges [60.62904929065257]
  Large language models (LLMs) offer possibility for resolving this issue by comprehending individual requests. This paper reviews the recently emerged LLM research related to educational capabilities, including mathematics, writing, programming, reasoning, and knowledge-based question answering.
  arXiv  Detail & Related papers  (2023-12-27T14:37:32Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)
• Dual Policy Distillation [58.43610940026261]
  Policy distillation, which transfers a teacher policy to a student policy, has achieved great success in challenging tasks of deep reinforcement learning. In this work, we introduce dual policy distillation(DPD), a student-student framework in which two learners operate on the same environment to explore different perspectives of the environment. The key challenge in developing this dual learning framework is to identify the beneficial knowledge from the peer learner for contemporary learning-based reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-06-07T06:49:47Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.