Making AI-Generated Feedback Matter: A Large-Scale Study of Feedback Workflows and Student Enactment

• URL: http://arxiv.org/abs/2608.11625v2
• Date: Thu, 13 Aug 2026 22:03:11 GMT
• Title: Making AI-Generated Feedback Matter: A Large-Scale Study of Feedback Workflows and Student Enactment
• Abstract summary: The educational value of AI-generated feedback depends not only on the quality of feedback comments, but also on that actively structure students' enactment of feedback literacy processes.<n>Overall findings show that AI access alone is insufficient; purposeful workflow design is central to productive feedback use.
• Score: 1.978444537409772
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feedback processes strongly influence student learning, yet their educational value depends on addressing two distinct challenges: providing high-quality, timely, and individualised feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively. Generative AI offers a credible means of addressing the provision challenge, but students' uptake of AI-generated feedback remains limited. We conducted a large-scale quasi-experimental sequential cohort study comparing three AI-mediated feedback workflows across 13,037 students and 51,296 student-authored resources. In Directed Feedback (n = 3,723), students received AI-generated feedback comments without structured support. In Self-Directed Feedback (n = 3,951), students could initiate optional AI-supported dialogue. In Enacted Feedback (n = 5,363), students were prompted to select feedback suggestions, evaluate their relevance, and engage in targeted AI-supported dialogue anchored to those selections. Enacted Feedback was associated with significantly higher uptake of AI-generated feedback, with an estimated probability of 26.2%, compared with 14.1% for Directed Feedback and 0.1% for Self-Directed Feedback. It was also associated with significantly higher self-assessment confidence and submitted-work quality than both comparison conditions. These findings suggest that the educational value of AI-generated feedback depends not only on the quality of feedback comments, but also on workflows that actively structure students' enactment of feedback literacy processes. The results have implications for the design of AI feedback systems that position learners as active participants in judgement, dialogue, and improvement rather than passive recipients of comments. Overall findings show that AI access alone is insufficient; purposeful workflow design is central to productive feedback use.

Related papers

• A Classroom Study of LLM-Generated Feedback Intervention in Introductory Programming [1.0241350366020698]
  We present a large-scale classroom study in which AI-generated feedback was deployed in an introductory Python programming course.<n>Students received one of three feedback conditions on incorrect submissions: natural language hints, AI-generated failing test cases, or no AI feedback.<n>We find that natural language feedback is significantly associated with higher completion rates and faster convergence to correct solutions.
  arXiv  Detail & Related papers  (2026-06-07T19:55:51Z)
• AI-Mediated Feedback Improves Student Revisions: A Randomized Trial with FeedbackWriter in a Large Undergraduate Course [5.829384802817518]
  We introduce and deploy FeedbackWriter, a system that generates AI suggestions to teaching assistants (TAs) while they provide feedback on students' knowledge-intensive essays.<n>Students were randomly assigned to receive either handwritten feedback from TAs or AI-mediated feedback where TAs received suggestions from FeedbackWriter.<n>We found that students receiving AI-mediated feedback produced significantly higher-quality revisions, with gains increasing as TAs adopted more AI suggestions.
  arXiv  Detail & Related papers  (2026-02-18T19:33:56Z)
• LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback [4.225232488376583]
  This study introduces a real-time AI-facilitated multimodal feedback system that integrates structured textual explanations with dynamic multimedia resources.<n>In an online crowdsourcing experiment, we compared this system against fixed business-as-usual feedback by educators across three dimensions.<n>Results showed that AI multimodal feedback achieved learning gains equivalent to original educator feedback while significantly outperforming it on perceived clarity, specificity, conciseness, motivation, satisfaction, and reducing cognitive load.
  arXiv  Detail & Related papers  (2026-01-21T18:58:08Z)
• Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use [3.345149032274467]
  This project examines the prospect of using AI-generated feedback as suggestions to expedite and enhance human instructors' feedback provision.<n>We developed a feedback engine that generates feedback on students' essays based on grading rubrics used by the teaching assistants (TAs)<n>We performed think-aloud studies with 5 TAs over 20 1-hour sessions to have them evaluate the AI feedback, contrast the AI feedback with their handwritten feedback, and share how they envision using the AI feedback if they were offered as suggestions.
  arXiv  Detail & Related papers  (2025-05-21T14:50:30Z)
• Evaluating Trust in AI, Human, and Co-produced Feedback Among Undergraduate Students [2.935250567679577]
  This study compares undergraduate students' trust in large language models (LLMs), human, and human-AI co-produced feedback in their authentic HE context.<n>Findings revealed students preferred AI and co-produced feedback over human feedback regarding perceived usefulness and objectivity.<n>Educational AI experience improved students' ability to identify LLM-generated feedback and increased their trust in all types of feedback.
  arXiv  Detail & Related papers  (2025-04-15T08:06:36Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• HREF: Human Response-Guided Evaluation of Instruction Following in Language Models [61.273153125847166]
  We develop a new evaluation benchmark, Human Response-Guided Evaluation of Instruction Following (HREF)<n>In addition to providing reliable evaluation, HREF emphasizes individual task performance and is free from contamination.<n>We study the impact of key design choices in HREF, including the size of the evaluation set, the judge model, the baseline model, and the prompt template.
  arXiv  Detail & Related papers  (2024-12-20T03:26:47Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Constructive Large Language Models Alignment with Diverse Feedback [76.9578950893839]
  We introduce Constructive and Diverse Feedback (CDF) as a novel method to enhance large language models alignment. We exploit critique feedback for easy problems, refinement feedback for medium problems, and preference feedback for hard problems. By training our model with this diversified feedback, we achieve enhanced alignment performance while using less training data.
  arXiv  Detail & Related papers  (2023-10-10T09:20:14Z)
• Continually Improving Extractive QA via Human Feedback [59.49549491725224]
  We study continually improving an extractive question answering (QA) system via human user feedback. We conduct experiments involving thousands of user interactions under diverse setups to broaden the understanding of learning from feedback over time.
  arXiv  Detail & Related papers  (2023-05-21T14:35:32Z)
• Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework [51.237191651923666]
  We investigate the potential of agent learning from trainers' facial expressions via interpreting them as evaluative feedback. With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback. Our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible.
  arXiv  Detail & Related papers  (2020-01-23T17:50:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.