That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai

• URL: http://arxiv.org/abs/2506.20433v2
• Date: Thu, 26 Jun 2025 12:36:56 GMT
• Title: That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai
• Authors: Sven Jacobs, Maurice Kempf, Natalie Kiesler,
• Abstract summary: We build a custom web application providing students with Python programming tasks, a code editor, GenAI feedback, and compiler feedback.<n>We investigate how much attention the generated feedback received from learners and to what extent the generated feedback is helpful (or not)<n>The findings indicate that GenAI feedback generally receives a lot of visual attention, with inexperienced students spending twice as much fixation time.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The potential of Generative AI (GenAI) for generating feedback in computing education has been the subject of numerous studies. However, there is still limited research on how computing students engage with this feedback and to what extent it supports their problem-solving. For this reason, we built a custom web application providing students with Python programming tasks, a code editor, GenAI feedback, and compiler feedback. Via a think-aloud protocol including eye-tracking and a post-interview with 11 undergraduate students, we investigate (1) how much attention the generated feedback received from learners and (2) to what extent the generated feedback is helpful (or not). In addition, students' attention to GenAI feedback is compared with that towards the compiler feedback. We further investigate differences between students with and without prior programming experience. The findings indicate that GenAI feedback generally receives a lot of visual attention, with inexperienced students spending twice as much fixation time. More experienced students requested GenAI less frequently, and could utilize it better to solve the given problem. It was more challenging for inexperienced students to do so, as they could not always comprehend the GenAI feedback. They often relied solely on the GenAI feedback, while compiler feedback was not read. Understanding students' attention and perception toward GenAI feedback is crucial for developing educational tools that support student learning.

Related papers

• Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For? [0.0]
  Building on prior research on Generative AI (GenAI) and related tools for programming education, we developed SCRIPT to support novice learners.<n>SCRIPT allows for open-ended interactions and structured guidance through predefined prompts.<n>We analyzed how students interacted with SCRIPT while solving programming tasks with a focus on their feedback preferences.
  arXiv  Detail & Related papers  (2025-07-23T06:56:26Z)
• Using Generative AI in Software Design Education: An Experience Report [0.6827423171182154]
  Students were required to use GenAI to help complete a team-based assignment.<n>Students identified numerous ways ChatGPT helped them in their design process.<n>We identified several key lessons for educators in how to deploy GenAI in a software design class effectively.
  arXiv  Detail & Related papers  (2025-06-26T18:40:16Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• How Do Programming Students Use Generative AI? [7.863638253070439]
  We studied how programming students actually use generative AI tools like ChatGPT.<n>We observed two prevalent usage strategies: to seek knowledge about general concepts and to directly generate solutions.<n>Our findings indicate that concerns about potential decrease in programmers' agency and productivity with Generative AI are justified.
  arXiv  Detail & Related papers  (2025-01-17T10:25:41Z)
• Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools [4.352985782794601]
  In 2024, new research started to emerge on the effects of GenAI usage in the computing classroom.<n>New data involve the use of GenAI to support classroom instruction at scale and to teach students how to code with GenAI.<n>New class of tools is emerging that can provide personalized feedback to students on their programming assignments or teach both programming and prompting skills at the same time.
  arXiv  Detail & Related papers  (2024-12-19T11:01:11Z)
• Misconceptions, Pragmatism, and Value Tensions: Evaluating Students' Understanding and Perception of Generative AI for Education [0.11704154007740832]
  Students are early adopters of the technology, utilizing it in atypical ways. Students were asked to describe 1) their understanding of GenAI; 2) their use of GenAI; 3) their opinions on the benefits, downsides, and ethical issues pertaining to its use in education.
  arXiv  Detail & Related papers  (2024-10-29T17:41:06Z)
• How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course [0.0]
  This research paper contributes to the computing education research community's understanding of Generative AI (GenAI) in the context of introductory programming. This study is guided by the following research questions:. What do students report on their use pattern of ChatGPT in the context of introductory programming exercises? How do students perceive ChatGPT in the context of introductory programming exercises?
  arXiv  Detail & Related papers  (2024-07-30T12:55:42Z)
• The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers [1.995977018536036]
  Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Many novices are now programming with generative AI (GenAI) Our findings show an unfortunate divide in the use of GenAI tools between students who accelerated and students who struggled.
  arXiv  Detail & Related papers  (2024-05-28T01:48:28Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learning [54.31495290436766]
  We extend BabyAI to automatically generate language feedback from environment dynamics and goal condition success. We modify the Decision Transformer architecture to take advantage of this additional signal. We find that training with language feedback either in place of or in addition to the return-to-go or goal descriptions improves agents' generalisation performance.
  arXiv  Detail & Related papers  (2023-12-07T22:33:34Z)
• Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
  Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. Standard approaches require instructors to manually grade student-implemented interactive programs. Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
  arXiv  Detail & Related papers  (2022-11-16T10:00:23Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.