LLM Pedagogical Behavior in AI Tutoring Interactions

• URL: http://arxiv.org/abs/2608.22993v1
• Date: Mon, 24 Aug 2026 08:57:40 GMT
• Title: LLM Pedagogical Behavior in AI Tutoring Interactions
• Abstract summary: Little is known about the level of assistance LLMs provide when students use them as tutors in authentic learning interactions.<n>We operationalize this dimension as scaffolding level and develop a five-level scale, validated against human annotations.<n>We apply the scale to 14,637 LLM responses from 203 students in a university AI course.
• Score: 2.7316402953904233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students use them as tutors in authentic learning interactions. This matters because tutoring responses can differ substantially in how directly they help students complete a task. We operationalize this dimension as scaffolding level and develop a five-level scale, validated against human annotations, that characterizes responses according to the degree of direct assistance they provide. We apply the scale to 14,637 LLM responses from 203 students in a university AI course. Responses are overwhelmingly concentrated at high levels of assistance, with more than 95% classified as either Explaining or Solving. Scaffolding level is systematically associated with students' subsequent conversational behavior, but provides little additional predictive information about performance on three subsequent exams beyond prior achievement and dialogue behavior. These findings provide an empirical baseline for LLM assistance in tutoring interactions and a measurement framework for evaluating how alternative tutoring designs change that assistance.

Related papers

• Self-Explanation Tutor for Active Study of CS1 Worked Examples [1.7504194797252353]
  We build a self-explanation tutor for introductory programming, ESSE, in which students explain lines of worked examples.<n>We ask whether the LLM judges student explanations well enough to serve as the engine of the tutor.<n>We find that its feedback leads students to persist and revise rather than abandon a line.
  arXiv  Detail & Related papers  (2026-08-25T21:54:47Z)
• AI Assistants Overassist [11.910820103556574]
  Large language models (LLMs) are increasingly used as tutors and thought partners, helping users reason through problems.<n>How AI systems navigate intervention decisions during problem-solving remains poorly understood.<n>Int-Bench simulates a "student" solving a problem while a "teacher" monitors the student's reasoning and decides whether, when, and how to intervene.
  arXiv  Detail & Related papers  (2026-07-23T13:30:01Z)
• Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming [0.0]
  Large Language Models (LLMs) can provide personalized support in learning.<n>This study examined how two types of LLM-based tutors shape students' prompting practices.
  arXiv  Detail & Related papers  (2026-07-03T13:15:36Z)
• PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors [66.56586559631516]
  Large language models (LLMs) have potential as educational tutors.<n>But different tutoring strategies benefit different student personalities.<n>Despite this, current LLM tutoring systems do not take into account student personality traits.
  arXiv  Detail & Related papers  (2026-01-13T10:17:26Z)
• Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Courses [4.761218834684297]
  Large language models (LLMs) are promising for bridging this gap, but interactions between students and LLMs are rarely overseen by educators.<n>We developed and studied an LLM-powered course assistant deployed across multiple computer science courses to characterize real-world use and understand pedagogical implications.
  arXiv  Detail & Related papers  (2025-09-10T02:21:11Z)
• Can Large Language Models Help Students Prove Software Correctness? An Experimental Study with Dafny [75.55915044740566]
  Students in computing education increasingly use large language models (LLMs) such as ChatGPT.<n>This paper investigates how students interact with an LLM when solving formal verification exercises in Dafny.
  arXiv  Detail & Related papers  (2025-06-27T16:34:13Z)
• From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning [82.50157695987558]
  Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy.<n>We propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors.
  arXiv  Detail & Related papers  (2025-05-21T15:00:07Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course [49.296957552006226]
  Using large language models (LLMs) for automatic evaluation has become an important evaluation method in NLP research. This report shares how we use GPT-4 as an automatic assignment evaluator in a university course with 1,028 students.
  arXiv  Detail & Related papers  (2024-07-07T00:17:24Z)
• MACAROON: Training Vision-Language Models To Be Your Engaged Partners [95.32771929749514]
  Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unlabeled. In this study, we aim to shift LVLMs from passive answer providers to proactive engaged partners. We introduce MACAROON, self-iMaginAtion for ContrAstive pReference OptimizatiON, which instructs LVLMs to autonomously generate contrastive response pairs for unlabeled questions.
  arXiv  Detail & Related papers  (2024-06-20T09:27:33Z)
• Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception [19.335003380399527]
  Large language models (LLMs) offer a promising avenue, with increasing research exploring their educational utility. Our work highlights the role that teachers can play in shaping LLM-supported learning environments.
  arXiv  Detail & Related papers  (2023-10-13T01:21:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.