Improving Student-AI Interaction Through Pedagogical Prompting: An Example in Computer Science Education

• URL: http://arxiv.org/abs/2506.19107v2
• Date: Sat, 28 Jun 2025 18:15:32 GMT
• Title: Improving Student-AI Interaction Through Pedagogical Prompting: An Example in Computer Science Education
• Authors: Ruiwei Xiao, Xinying Hou, Runlong Ye, Majeed Kazemitabaar, Nicholas Diana, Michael Liut, John Stamper,
• Abstract summary: Large language model (LLM) applications have sparked both excitement and concern.<n>Recent studies consistently highlight students' (mis)use of LLMs can hinder learning outcomes.<n>This work aims to teach students how to effectively prompt LLMs to improve their learning.
• Score: 1.1517315048749441
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the proliferation of large language model (LLM) applications since 2022, their use in education has sparked both excitement and concern. Recent studies consistently highlight students' (mis)use of LLMs can hinder learning outcomes. This work aims to teach students how to effectively prompt LLMs to improve their learning. We first proposed pedagogical prompting, a theoretically-grounded new concept to elicit learning-oriented responses from LLMs. To move from concept design to a proof-of-concept learning intervention in real educational settings, we selected early undergraduate CS education (CS1/CS2) as the example context. We began with a formative survey study with instructors (N=36) teaching early-stage undergraduate-level CS courses to inform the instructional design based on classroom needs. Based on their insights, we designed and developed a learning intervention through an interactive system with scenario-based instruction to train pedagogical prompting skills. Finally, we evaluated its instructional effectiveness through a user study with CS novice students (N=22) using pre/post-tests. Through mixed methods analyses, our results indicate significant improvements in learners' LLM-based pedagogical help-seeking skills, along with positive attitudes toward the system and increased willingness to use pedagogical prompts in the future. Our contributions include (1) a theoretical framework of pedagogical prompting; (2) empirical insights into current instructor attitudes toward pedagogical prompting; and (3) a learning intervention design with an interactive learning tool and scenario-based instruction leading to promising results on teaching LLM-based help-seeking. Our approach is scalable for broader implementation in classrooms and has the potential to be integrated into tools like ChatGPT as an on-boarding experience to encourage learning-oriented use of generative AI.

Related papers

• Partnering with AI: A Pedagogical Feedback System for LLM Integration into Programming Education [19.441958600393342]
  This paper introduces a novel framework for large language models (LLMs)-driven feedback generation.<n>Our findings suggest that teachers consider that, when aligned with the framework, LLMs can effectively support students.<n>However, we found several limitations, such as its inability to adapt feedback to dynamic classroom contexts.
  arXiv  Detail & Related papers  (2025-07-01T03:48:48Z)
• From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning [76.09281171131941]
  Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy.<n>We propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors.
  arXiv  Detail & Related papers  (2025-05-21T15:00:07Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• BIPED: Pedagogically Informed Tutoring System for ESL Education [11.209992106075788]
  Large Language Models (LLMs) have a great potential to serve as readily available and cost-efficient Conversational Intelligent Tutoring Systems (CITS) Existing CITS are designed to teach only simple concepts or lack the pedagogical depth necessary to address diverse learning strategies. We construct a BIlingual PEDagogically-informed Tutoring dataset of one-on-one, human-to-human English tutoring interactions.
  arXiv  Detail & Related papers  (2024-06-05T17:49:24Z)
• Scaffolding Language Learning via Multi-modal Tutoring Systems with Pedagogical Instructions [34.760230622675365]
  Intelligent tutoring systems (ITSs) imitate human tutors and aim to provide customized instructions or feedback to learners. With the emergence of generative artificial intelligence, large language models (LLMs) entitle the systems to complex and coherent conversational interactions. We investigate how pedagogical instructions facilitate the scaffolding in ITSs, by conducting a case study on guiding children to describe images for language learning.
  arXiv  Detail & Related papers  (2024-04-04T13:22:28Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• YODA: Teacher-Student Progressive Learning for Language Models [82.0172215948963]
  This paper introduces YODA, a teacher-student progressive learning framework. It emulates the teacher-student education process to improve the efficacy of model fine-tuning. Experiments show that training LLaMA2 with data from YODA improves SFT with significant performance gain.
  arXiv  Detail & Related papers  (2024-01-28T14:32:15Z)
• Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis [0.9187505256430948]
  The present study uses transmodal ordered network analysis to understand effective teacher practices in relationship to traditional metrics of in-system learning in a mathematics classroom working with AI tutors. Comparing teacher practices by student learning rates, we find that students with low learning rates exhibited more hint use after monitoring. Students with low learning rates showed learning behavior similar to their high learning rate peers, achieving repeated correct attempts in the tutor.
  arXiv  Detail & Related papers  (2023-12-17T21:50:02Z)
• Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception [19.335003380399527]
  Large language models (LLMs) offer a promising avenue, with increasing research exploring their educational utility. Our work highlights the role that teachers can play in shaping LLM-supported learning environments.
  arXiv  Detail & Related papers  (2023-10-13T01:21:52Z)
• A Machine Learning system to monitor student progress in educational institutes [0.0]
  We propose a data driven approach that makes use of Machine Learning techniques to generate a classifier called credit score. The proposal to use credit score as progress indicator is well suited to be used in a Learning Management System.
  arXiv  Detail & Related papers  (2022-11-02T08:24:08Z)
• Iterative Teacher-Aware Learning [136.05341445369265]
  In human pedagogy, teachers and students can interact adaptively to maximize communication efficiency. We propose a gradient optimization based teacher-aware learner who can incorporate teacher's cooperative intention into the likelihood function.
  arXiv  Detail & Related papers  (2021-10-01T00:27:47Z)
• Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems [58.724629408229205]
  We demonstrate how traditional supervised learning and a simulator-free adversarial learning method can be used to achieve performance comparable to state-of-the-art RL-based methods. Our main goal is not to beat reinforcement learning with supervised learning, but to demonstrate the value of rethinking the role of reinforcement learning and supervised learning in optimizing task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2020-09-21T12:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.