Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice

• URL: http://arxiv.org/abs/2509.12107v1
• Date: Mon, 15 Sep 2025 16:33:37 GMT
• Title: Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice
• Authors: Si Chen, Isabel R. Molnar, Peiyu Li, Adam Acunin, Ting Hua, Alex Ambrose, Nitesh V. Chawla, Ronald Metoyer,
• Abstract summary: We evaluate TeaPT, a large language model that supports instructors' professional development through two conversational approaches.<n>A Socratic approach that uses guided questioning to foster reflection, and a Narrative approach that offers elaborated suggestions to extend externalized cognition.<n>Less-experienced, AI-optimistic instructors favored the Socratic version, whereas more-experienced, AI-cautious instructors preferred the Narrative version.
• Score: 24.54129847914925
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large language models (LLMs) typically generate direct answers, yet they are increasingly used as learning tools. Studying instructors' usage is critical, given their role in teaching and guiding AI adoption in education. We designed and evaluated TeaPT, an LLM for pedagogical purposes that supports instructors' professional development through two conversational approaches: a Socratic approach that uses guided questioning to foster reflection, and a Narrative approach that offers elaborated suggestions to extend externalized cognition. In a mixed-method study with 41 higher-education instructors, the Socratic version elicited greater engagement, while the Narrative version was preferred for actionable guidance. Subgroup analyses further revealed that less-experienced, AI-optimistic instructors favored the Socratic version, whereas more-experienced, AI-cautious instructors preferred the Narrative version. We contribute design implications for LLMs for pedagogical purposes, showing how adaptive conversational approaches can support instructors with varied profiles while highlighting how AI attitudes and experience shape interaction and learning.

Related papers

• Letting Tutor Personas "Speak Up" for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization [45.40380629269521]
  We show how tutor personas embedded in human tutor-student dialogues can be used to guide LLM behavior without relying on explicitly prompted instructions.<n>We find that a steering vector captures tutor-specific variation across dialogue contexts, improving semantic alignment with ground-truth tutor utterances and increasing preference-based evaluations.
  arXiv  Detail & Related papers  (2026-02-07T17:44:07Z)
• PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors [66.56586559631516]
  Large language models (LLMs) have potential as educational tutors.<n>But different tutoring strategies benefit different student personalities.<n>Despite this, current LLM tutoring systems do not take into account student personality traits.
  arXiv  Detail & Related papers  (2026-01-13T10:17:26Z)
• Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs [34.94756659609455]
  This study shifts focus from mere question generation to the broader instructional guidance capability.<n>We propose GuideEval, a benchmark grounded in authentic educational dialogues.<n> Empirical findings reveal that existing LLMs frequently fail to provide effective adaptive scaffolding.
  arXiv  Detail & Related papers  (2025-08-08T01:02:44Z)
• \textsc{SimInstruct}: A Responsible Tool for Collecting Scaffolding Dialogues Between Experts and LLM-Simulated Novices [21.67295740032255]
  SimInstruct is a scalable, expert-in-the-loop tool for collecting scaffolding dialogues.<n>Using teaching development coaching as an example domain, SimInstruct simulates novice instructors via LLMs.<n>Our results reveal that persona traits, such as extroversion and introversion, meaningfully influence how experts engage.
  arXiv  Detail & Related papers  (2025-08-06T13:16:10Z)
• Improving Student-AI Interaction Through Pedagogical Prompting: An Example in Computer Science Education [1.1517315048749441]
  Large language model (LLM) applications have sparked both excitement and concern.<n>Recent studies consistently highlight students' (mis)use of LLMs can hinder learning outcomes.<n>This work aims to teach students how to effectively prompt LLMs to improve their learning.
  arXiv  Detail & Related papers  (2025-06-23T20:39:17Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• SPL: A Socratic Playground for Learning Powered by Large Language Model [5.383689446227398]
  Socratic Playground for Learning (SPL) is a dialogue-based ITS powered by the GPT-4 model. SPL aims to enhance personalized and adaptive learning experiences tailored to individual needs.
  arXiv  Detail & Related papers  (2024-06-20T01:18:52Z)
• ProSwitch: Knowledge-Guided Instruction Tuning to Switch Between Professional and Non-Professional Responses [56.949741308866535]
  Large Language Models (LLMs) have demonstrated efficacy in various linguistic applications.<n>This study introduces a novel approach, named ProSwitch, which enables a language model to switch between professional and non-professional answers.
  arXiv  Detail & Related papers  (2024-03-14T06:49:16Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning [63.63840740526497]
  We investigate how instruction tuning adjusts pre-trained models with a focus on intrinsic changes. The impact of instruction tuning is then studied by comparing the explanations derived from the pre-trained and instruction-tuned models. Our findings reveal three significant impacts of instruction tuning.
  arXiv  Detail & Related papers  (2023-09-30T21:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.