Automated Formative Feedback for Short-form Writing: An LLM-Driven Approach and Adoption Analysis

• URL: http://arxiv.org/abs/2509.22734v1
• Date: Thu, 25 Sep 2025 15:17:51 GMT
• Title: Automated Formative Feedback for Short-form Writing: An LLM-Driven Approach and Adoption Analysis
• Authors: Tiago Fernandes Tavares, Luciano Pereira Soares,
• Abstract summary: This paper explores the development and adoption of AI-based formative feedback in an engineering program.<n>A tool was developed to provide students with personalized feedback on their draft reports, guiding them toward improved completeness and quality.
• Score: 1.2234742322758418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper explores the development and adoption of AI-based formative feedback in the context of biweekly reports in an engineering Capstone program. Each student is required to write a short report detailing their individual accomplishments over the past two weeks, which is then assessed by their advising professor. An LLM-powered tool was developed to provide students with personalized feedback on their draft reports, guiding them toward improved completeness and quality. Usage data across two rounds revealed an initial barrier to adoption, with low engagement rates. However, students who engaged in the AI feedback system demonstrated the ability to use it effectively, leading to improvements in the completeness and quality of their reports. Furthermore, the tool's task-parsing capabilities provided a novel approach to identify potential student organizational tasks and deliverables. The findings suggest initial skepticism toward the tool with a limited adoption within the studied context, however, they also highlight the potential for AI-driven tools to provide students and professors valuable insights and formative support.

Related papers

• Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review [53.99984738447279]
  Recent work frames this task as automatic text generation, underusing author expertise and intent.<n>We introduce REspGen, a generation framework that integrates explicit author input, multi-attribute control, and evaluation-guided refinement.<n>To support this formulation, we construct Re$3$Align, the first large-scale dataset of aligned review-response--revision triplets.
  arXiv  Detail & Related papers  (2026-01-19T14:07:10Z)
• Exposía: Academic Writing Assessment of Exposés and Peer Feedback [56.428320613219306]
  We present Exposa, the first public dataset that connects writing and feedback assessment in higher education.<n>We use Exposa to benchmark state-of-the-art open-source large language models (LLMs) for two tasks: automated scoring of (1) the proposals and (2) the student reviews.
  arXiv  Detail & Related papers  (2026-01-10T11:33:26Z)
• A Survey on Feedback Types in Automated Programming Assessment Systems [3.9845307287664973]
  This study investigates how different feedback mechanisms in APASs are perceived by students, and how effective they are in supporting problem-solving.<n>Results indicate that while students rate unit test feedback as the most helpful, AI-generated feedback leads to significantly better performances.
  arXiv  Detail & Related papers  (2025-10-21T09:08:22Z)
• Teaching at Scale: Leveraging AI to Evaluate and Elevate Engineering Education [3.557803321422781]
  This article presents a scalable, AI-supported framework for qualitative student feedback using large language models.<n>The system employs hierarchical summarization, anonymization, and exception handling to extract actionable themes from open-ended comments.<n>We report on its successful deployment across a large college of engineering.
  arXiv  Detail & Related papers  (2025-08-01T20:27:40Z)
• LLM Contribution Summarization in Software Projects [0.0]
  This paper addresses the need for an automated and objective approach to evaluate individual contributions within team projects.<n>We present a tool that leverages a large language model (LLM) to automatically summarize code contributions extracted from version control repositories.<n>The tool was assessed over two semesters during a three-week, full-time software development sprint involving 65 students.
  arXiv  Detail & Related papers  (2025-05-23T10:26:43Z)
• Advancing AI Research Assistants with Expert-Involved Learning [84.30323604785646]
  Large language models (LLMs) and large multimodal models (LMMs) promise to accelerate biomedical discovery, yet their reliability remains unclear.<n>We introduce ARIEL (AI Research Assistant for Expert-in-the-Loop Learning), an open-source evaluation and optimization framework.<n>We find that state-of-the-art models generate fluent but incomplete summaries, whereas LMMs struggle with detailed visual reasoning.
  arXiv  Detail & Related papers  (2025-05-03T14:21:48Z)
• PyEvalAI: AI-assisted evaluation of Jupyter Notebooks for immediate personalized feedback [43.56788158589046]
  PyEvalAI scores Jupyter notebooks using a combination of unit tests and a locally hosted language model to preserve privacy.<n>A case study demonstrates its effectiveness in improving feedback speed and grading efficiency for exercises in a university-level course on numerics.
  arXiv  Detail & Related papers  (2025-02-25T18:20:20Z)
• Transforming Student Evaluation with Adaptive Intelligence and Performance Analytics [0.0]
  This paper creates a system for the evaluation of students performance using Artificial intelligence.<n>There are formats of questions in the system which comprises multiple choice, short answers and descriptive questions.<n>The platform has intelligent learning progressions where the user will be able to monitor his/her performances to be recommended a certain level of performance.
  arXiv  Detail & Related papers  (2025-02-07T18:57:51Z)
• AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment [15.969280805269976]
  AERA Chat is an interactive visualization platform designed for automated explainable student answer assessment.<n>AERA Chat leverages multiple language models (LLMs) to concurrently score student answers and generate explanatory rationales.<n>We demonstrate the effectiveness of our platform through evaluations of multiple rationale-generation methods on several datasets.
  arXiv  Detail & Related papers  (2024-10-12T11:57:53Z)
• "I understand why I got this grade": Automatic Short Answer Grading with Feedback [33.63970664152288]
  We introduce Engineering Short Answer Feedback (EngSAF), a dataset designed for automatic short-answer grading with feedback.<n>We incorporate feedback into our dataset by leveraging the generative capabilities of state-of-the-art large language models (LLMs) using our Label-Aware Synthetic Feedback Generation (LASFG) strategy.<n>The best-performing model (Mistral-7B) achieves an overall accuracy of 75.4% and 58.7% on unseen answers and unseen question test sets, respectively.
  arXiv  Detail & Related papers  (2024-06-30T15:42:18Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Little Giants: Exploring the Potential of Small LLMs as Evaluation Metrics in Summarization in the Eval4NLP 2023 Shared Task [53.163534619649866]
  This paper focuses on assessing the effectiveness of prompt-based techniques to empower Large Language Models to handle the task of quality estimation. We conducted systematic experiments with various prompting techniques, including standard prompting, prompts informed by annotator instructions, and innovative chain-of-thought prompting. Our work reveals that combining these approaches using a "small", open source model (orca_mini_v3_7B) yields competitive results.
  arXiv  Detail & Related papers  (2023-11-01T17:44:35Z)
• Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture [43.85335847262138]
  Real-world domain experts (e.g., doctors) rarely annotate only a decision label in their day-to-day workflow without providing explanations. This work proposes a novel Active Learning architecture to support experts' real-world need for label and explanation annotations.
  arXiv  Detail & Related papers  (2023-05-22T04:38:10Z)
• Post Hoc Explanations of Language Models Can Improve Language Models [43.2109029463221]
  We present a novel framework, Amplifying Model Performance by Leveraging In-Context Learning with Post Hoc Explanations (AMPLIFY) We leverage post hoc explanation methods which output attribution scores (explanations) capturing the influence of each of the input features on model predictions. Our framework, AMPLIFY, leads to prediction accuracy improvements of about 10-25% over a wide range of tasks.
  arXiv  Detail & Related papers  (2023-05-19T04:46:04Z)
• Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU [88.8401599172922]
  We develop a framework based on self-training language models with limited task-specific labels and rationales. We show that the neural model performance can be significantly improved by making it aware of its rationalized predictions.
  arXiv  Detail & Related papers  (2021-09-17T00:36:46Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.