Avoiding Help Avoidance: Using Interface Design Changes to Promote Unsolicited Hint Usage in an Intelligent Tutor

• URL: http://arxiv.org/abs/2009.13371v2
• Date: Tue, 13 Oct 2020 16:28:55 GMT
• Title: Avoiding Help Avoidance: Using Interface Design Changes to Promote Unsolicited Hint Usage in an Intelligent Tutor
• Authors: Mehak Maniktala, Christa Cody, Tiffany Barnes, and Min Chi
• Abstract summary: We propose a new hint delivery mechanism called "Assertions" for providing unsolicited hints in a data-driven intelligent tutor. In systems that only provide hints upon request, hint avoidance results in students not receiving hints when they are needed. Our results show that Assertions significantly increase unsolicited hint usage compared to Messages.
• Score: 6.639504127104268
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Within intelligent tutoring systems, considerable research has investigated hints, including how to generate data-driven hints, what hint content to present, and when to provide hints for optimal learning outcomes. However, less attention has been paid to how hints are presented. In this paper, we propose a new hint delivery mechanism called "Assertions" for providing unsolicited hints in a data-driven intelligent tutor. Assertions are partially-worked example steps designed to appear within a student workspace, and in the same format as student-derived steps, to show students a possible subgoal leading to the solution. We hypothesized that Assertions can help address the well-known hint avoidance problem. In systems that only provide hints upon request, hint avoidance results in students not receiving hints when they are needed. Our unsolicited Assertions do not seek to improve student help-seeking, but rather seek to ensure students receive the help they need. We contrast Assertions with Messages, text-based, unsolicited hints that appear after student inactivity. Our results show that Assertions significantly increase unsolicited hint usage compared to Messages. Further, they show a significant aptitude-treatment interaction between Assertions and prior proficiency, with Assertions leading students with low prior proficiency to generate shorter (more efficient) posttest solutions faster. We also present a clustering analysis that shows patterns of productive persistence among students with low prior knowledge when the tutor provides unsolicited help in the form of Assertions. Overall, this work provides encouraging evidence that hint presentation can significantly impact how students use them and using Assertions can be an effective way to address help avoidance.

Related papers

• Likelihood as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that likelihoods serve as an effective gauge for language model performance. We propose two methods that use question likelihood as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• A Knowledge-Component-Based Methodology for Evaluating AI Assistants [9.412070852474313]
  We evaluate an automatic hint generator for CS1 programming assignments powered by GPT-4. This system provides natural language guidance about how students can improve their incorrect solutions to short programming exercises.
  arXiv  Detail & Related papers  (2024-06-09T00:58:39Z)
• Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices [0.0]
  We investigated whether different levels of hints can support students' problem-solving and learning. We conducted a think-aloud study with 12 novices using the LLM Hint Factory. We discovered that high-level natural language hints alone can be helpless or even misleading.
  arXiv  Detail & Related papers  (2024-04-02T18:05:26Z)
• Improved Compositional Generalization by Generating Demonstrations for Meta-Learning [53.818234285773165]
  We show substantially improved performance on a previously unsolved compositional behaviour split without a loss of performance on other splits. In this case, searching for relevant demonstrations even with an oracle function is not sufficient to attain good performance when using meta-learning.
  arXiv  Detail & Related papers  (2023-05-22T14:58:54Z)
• Semantic Prompt for Few-Shot Image Recognition [76.68959583129335]
  We propose a novel Semantic Prompt (SP) approach for few-shot learning. The proposed approach achieves promising results, improving the 1-shot learning accuracy by 3.67% on average.
  arXiv  Detail & Related papers  (2023-03-24T16:32:19Z)
• Evaluating the Robustness of Discrete Prompts [27.919548466481583]
  We conduct a systematic study of the robustness of discrete prompts. We measure their performance in two Natural Language Inference (NLI) datasets. Our results show that although the discrete prompt-based method remains relatively robust against perturbations to NLI inputs, they are highly sensitive to other types of perturbations such as shuffling and deletion of prompt tokens.
  arXiv  Detail & Related papers  (2023-02-11T07:01:53Z)
• Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
  Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
  arXiv  Detail & Related papers  (2023-02-09T18:02:34Z)
• Enhancing a Student Productivity Model for Adaptive Problem-Solving Assistance [7.253181280137071]
  We present a novel data-driven approach to incorporate students' hint usage in predicting their need for help. We show empirical evidence to support that such a policy can save students a significant amount of time in training. We conclude with suggestions on the domains that can benefit from this approach as well as the requirements for adoption.
  arXiv  Detail & Related papers  (2022-07-07T00:41:00Z)
• PROMPT WAYWARDNESS: The Curious Case of Discretized Interpretation of Continuous Prompts [99.03864962014431]
  Fine-tuning continuous prompts for target tasks has emerged as a compact alternative to full model fine-tuning. In practice, we observe a "wayward" behavior between the task solved by continuous prompts and their nearest neighbor.
  arXiv  Detail & Related papers  (2021-12-15T18:55:05Z)
• Extending the Hint Factory for the assistance dilemma: A novel, data-driven HelpNeed Predictor for proactive problem-solving help [6.188683567894372]
  We present a set of data-driven methods to classify, predict, and prevent unproductive problem-solving steps. We present a HelpNeed classification, that uses prior student data to determine when students are likely to be unproductive. We conclude with suggestions on how these HelpNeed methods could be applied in other well-structured open-ended domains.
  arXiv  Detail & Related papers  (2020-10-08T17:04:03Z)
• Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space [56.22272513894306]
  Active learning for sentence understanding aims at discovering informative unlabeled data for annotation. We argue that the typical uncertainty sampling method for active learning is time-consuming and can hardly work in real-time. We propose adversarial uncertainty sampling in discrete space (AUSDS) to retrieve informative unlabeled samples more efficiently.
  arXiv  Detail & Related papers  (2020-04-17T03:12:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.