Understanding Teacher Revisions of Large Language Model-Generated Feedback

• URL: http://arxiv.org/abs/2603.27806v1
• Date: Sun, 29 Mar 2026 18:41:00 GMT
• Title: Understanding Teacher Revisions of Large Language Model-Generated Feedback
• Abstract summary: We analyze 1,349 instances of AI-generated feedback and corresponding teacher-edited explanations from 117 teachers.<n>We find that teachers accept AI feedback without modification in about 80% of cases, while edited feedback tends to be significantly longer and subsequently shortened by teachers.<n>Machine learning models trained only on the AI feedback text as input features achieve fair performance in identifying which feedback will be revised.
• Score: 0.8039067099377077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) increasingly generate formative feedback for students, yet little is known about how teachers revise this feedback before it reaches learners. Teachers' revisions shape what students receive, making revision practices central to evaluating AI classroom tools. We analyze a dataset of 1,349 instances of AI-generated feedback and corresponding teacher-edited explanations from 117 teachers. We examine (i) textual characteristics associated with teacher revisions, (ii) whether revision decisions can be predicted from the AI feedback text, and (iii) how revisions change the pedagogical type of feedback delivered. First, we find that teachers accept AI feedback without modification in about 80% of cases, while edited feedback tends to be significantly longer and subsequently shortened by teachers. Editing behavior varies substantially across teachers: about 50% never edit AI feedback, and only about 10% edit more than two-thirds of feedback instances. Second, machine learning models trained only on the AI feedback text as input features, using sentence embeddings, achieve fair performance in identifying which feedback will be revised (AUC=0.75). Third, qualitative coding shows that when revisions occur, teachers often simplify AI-generated feedback, shifting it away from high-information explanations toward more concise, corrective forms. Together, these findings characterize how teachers engage with AI-generated feedback in practice and highlight opportunities to design feedback systems that better align with teacher priorities while reducing unnecessary editing effort.

Related papers

• What Drives Interactive Improvement from Feedback? [0.7330217643497284]
  We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone.<n>In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation.<n>We introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1.
  arXiv  Detail & Related papers  (2026-06-29T18:06:36Z)
• A Classroom Study of LLM-Generated Feedback Intervention in Introductory Programming [1.0241350366020698]
  We present a large-scale classroom study in which AI-generated feedback was deployed in an introductory Python programming course.<n>Students received one of three feedback conditions on incorrect submissions: natural language hints, AI-generated failing test cases, or no AI feedback.<n>We find that natural language feedback is significantly associated with higher completion rates and faster convergence to correct solutions.
  arXiv  Detail & Related papers  (2026-06-07T19:55:51Z)
• AI-Mediated Feedback Improves Student Revisions: A Randomized Trial with FeedbackWriter in a Large Undergraduate Course [5.829384802817518]
  We introduce and deploy FeedbackWriter, a system that generates AI suggestions to teaching assistants (TAs) while they provide feedback on students' knowledge-intensive essays.<n>Students were randomly assigned to receive either handwritten feedback from TAs or AI-mediated feedback where TAs received suggestions from FeedbackWriter.<n>We found that students receiving AI-mediated feedback produced significantly higher-quality revisions, with gains increasing as TAs adopted more AI suggestions.
  arXiv  Detail & Related papers  (2026-02-18T19:33:56Z)
• Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback [21.114005575615586]
  We explore learnersourcing as a means to fine-tune language models for generating feedback that is more similar to that written by humans.<n>We collected approximately 1,900 instances of student-written feedback on multiple programming problems and buggy programs.<n>Our findings indicate that fine-tuning models on learnersourced data not only produces feedback that better matches the style of feedback written by students, but also improves accuracy compared to feedback generated through prompt engineering alone.
  arXiv  Detail & Related papers  (2025-09-12T19:23:05Z)
• Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use [3.345149032274467]
  This project examines the prospect of using AI-generated feedback as suggestions to expedite and enhance human instructors' feedback provision.<n>We developed a feedback engine that generates feedback on students' essays based on grading rubrics used by the teaching assistants (TAs)<n>We performed think-aloud studies with 5 TAs over 20 1-hour sessions to have them evaluate the AI feedback, contrast the AI feedback with their handwritten feedback, and share how they envision using the AI feedback if they were offered as suggestions.
  arXiv  Detail & Related papers  (2025-05-21T14:50:30Z)
• Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors [78.53699244846285]
  Large language models (LLMs) present an opportunity to scale high-quality personalized education to all. LLMs struggle to precisely detect student's errors and tailor their feedback to these errors. Inspired by real-world teaching practice where teachers identify student errors and customize their response based on them, we focus on verifying student solutions.
  arXiv  Detail & Related papers  (2024-07-12T10:11:40Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Constructive Large Language Models Alignment with Diverse Feedback [76.9578950893839]
  We introduce Constructive and Diverse Feedback (CDF) as a novel method to enhance large language models alignment. We exploit critique feedback for easy problems, refinement feedback for medium problems, and preference feedback for hard problems. By training our model with this diversified feedback, we achieve enhanced alignment performance while using less training data.
  arXiv  Detail & Related papers  (2023-10-10T09:20:14Z)
• PapagAI:Automated Feedback for Reflective Essays [48.4434976446053]
  We present the first open-source automated feedback tool based on didactic theory and implemented as a hybrid AI system. The main objective of our work is to enable better learning outcomes for students and to complement the teaching activities of lecturers.
  arXiv  Detail & Related papers  (2023-07-10T11:05:51Z)
• HIVE: Harnessing Human Feedback for Instructional Visual Editing [127.29436858998064]
  We present a novel framework to harness human feedback for instructional visual editing (HIVE) Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward.
  arXiv  Detail & Related papers  (2023-03-16T19:47:41Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Effects of Human vs. Automatic Feedback on Students' Understanding of AI Concepts and Programming Style [0.0]
  The use of automatic grading tools has become nearly ubiquitous in large undergraduate programming courses. There is a relative lack of data directly comparing student outcomes when receiving computer-generated feedback and human-written feedback. This paper addresses this gap by splitting one 90-student class into two feedback groups and analyzing differences in the two cohorts' performance.
  arXiv  Detail & Related papers  (2020-11-20T21:40:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.