Designing and Evaluating Hint Generation Systems for Science Education

• URL: http://arxiv.org/abs/2510.21087v1
• Date: Fri, 24 Oct 2025 02:00:16 GMT
• Title: Designing and Evaluating Hint Generation Systems for Science Education
• Authors: Anubhav Jangra, Smaranda Muresan,
• Abstract summary: We study the role of automatic hint generation as a pedagogical strategy to promote active engagement with the learning content.<n>We compare two distinct hinting strategies: static hints, pre-generated for each problem, and dynamic hints, adapted to learners' progress.
• Score: 21.96985881818661
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are influencing the education landscape, with students relying on them in their learning process. Often implemented using general-purpose models, these systems are likely to give away the answers, which could hinder conceptual understanding and critical thinking. We study the role of automatic hint generation as a pedagogical strategy to promote active engagement with the learning content, while guiding learners toward the answers. Focusing on scientific topics at the secondary education level, we explore the potential of large language models to generate chains of hints that scaffold learners without revealing answers. We compare two distinct hinting strategies: static hints, pre-generated for each problem, and dynamic hints, adapted to learners' progress. Through a quantitative study with 41 participants, we uncover different preferences among learners with respect to hinting strategies, and identify the limitations of automatic evaluation metrics to capture them. Our findings highlight key design considerations for future research on hint generation and intelligent tutoring systems that seek to develop learner-centered educational technologies.

Related papers

• AgentTutor: Empowering Personalized Learning with Multi-Turn Interactive Teaching in Intelligent Education Systems [11.202091624300062]
  AgentTutor is a multi-turn interactive intelligent education system to empower personalized learning.<n>It features an LLM-powered generative multi-agent system and a learner-specific personalized learning profile environment.<n>It includes five key modules: curriculum decomposition, learner assessment, dynamic strategy, teaching reflection, and knowledge & experience memory.
  arXiv  Detail & Related papers  (2025-12-24T12:26:28Z)
• Understanding Student Interaction with AI-Powered Next-Step Hints: Strategies and Challenges [1.446446435461625]
  Next-step hint feedback provides students with actionable steps to progress towards solving programming tasks.<n>This study investigates how students interact with an AI-driven next-step hint system in an in-IDE learning environment.
  arXiv  Detail & Related papers  (2025-11-09T12:56:34Z)
• The Imperfect Learner: Incorporating Developmental Trajectories in Memory-based Student Simulation [55.722188569369656]
  This paper introduces a novel framework for memory-based student simulation.<n>It incorporates developmental trajectories through a hierarchical memory mechanism with structured knowledge representation.<n>In practice, we implement a curriculum-aligned simulator grounded on the Next Generation Science Standards.
  arXiv  Detail & Related papers  (2025-11-08T08:05:43Z)
• A Survey on Generative Recommendation: Data, Model, and Tasks [55.36322811257545]
  generative recommendation reconceptualizes recommendation as a generation task rather than discriminative scoring.<n>This survey provides a comprehensive examination through a unified tripartite framework spanning data, model, and task dimensions.<n>We identify five key advantages: world knowledge integration, natural language understanding, reasoning capabilities, scaling laws, and creative generation.
  arXiv  Detail & Related papers  (2025-10-31T04:02:58Z)
• Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study [50.065744358362345]
  Large language models (LLMs) have shown impressive capabilities across tasks such as mathematics, coding, and reasoning.<n>Yet their learning ability, which is crucial for adapting to dynamic environments and acquiring new knowledge, remains underexplored.
  arXiv  Detail & Related papers  (2025-06-16T13:24:50Z)
• Machine Learning: a Lecture Note [51.31735291774885]
  This lecture note is intended to prepare early-year master's and PhD students in data science or a related discipline with foundational ideas in machine learning.<n>It starts with basic ideas in modern machine learning with classification as a main target task.<n>Based on these basic ideas, the lecture note explores in depth the probablistic approach to unsupervised learning.
  arXiv  Detail & Related papers  (2025-05-06T16:03:41Z)
• YouLeQD: Decoding the Cognitive Complexity of Questions and Engagement in Online Educational Videos from Learners' Perspectives [1.2084539012992408]
  YouLeQD dataset contains learner-posed questions from YouTube lecture video comments.<n>We developed two RoBERTa-based classification models to detect questions and analyze their cognitive complexity.
  arXiv  Detail & Related papers  (2025-01-20T19:54:38Z)
• Navigating the Landscape of Hint Generation Research: From the Past to the Future [34.47999708205151]
  We present a review of prior research on hint generation, aiming to bridge the gap between research in education and cognitive science.<n>We propose a formal definition of the hint generation task, and discuss the roadmap of building an effective hint generation system.
  arXiv  Detail & Related papers  (2024-04-06T20:42:46Z)
• Scaffolding Language Learning via Multi-modal Tutoring Systems with Pedagogical Instructions [34.760230622675365]
  Intelligent tutoring systems (ITSs) imitate human tutors and aim to provide customized instructions or feedback to learners. With the emergence of generative artificial intelligence, large language models (LLMs) entitle the systems to complex and coherent conversational interactions. We investigate how pedagogical instructions facilitate the scaffolding in ITSs, by conducting a case study on guiding children to describe images for language learning.
  arXiv  Detail & Related papers  (2024-04-04T13:22:28Z)
• Enhancing Generative Class Incremental Learning Performance with Model Forgetting Approach [50.36650300087987]
  This study presents a novel approach to Generative Class Incremental Learning (GCIL) by introducing the forgetting mechanism. We have found that integrating the forgetting mechanisms significantly enhances the models' performance in acquiring new knowledge.
  arXiv  Detail & Related papers  (2024-03-27T05:10:38Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• Integrating AI and Learning Analytics for Data-Driven Pedagogical Decisions and Personalized Interventions in Education [0.2812395851874055]
  This research study explores the conceptualization, development, and deployment of an innovative learning analytics tool. By analyzing critical data points such as students' stress levels, curiosity, confusion, agitation, topic preferences, and study methods, the tool provides a comprehensive view of the learning environment. This research underscores AI's role in shaping personalized, data-driven education.
  arXiv  Detail & Related papers  (2023-12-15T06:00:26Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.