SEFORA: Student Essays with Feedback Corpus and LLM Feedback Evaluation Framework

• URL: http://arxiv.org/abs/2607.00274v1
• Date: Tue, 30 Jun 2026 23:48:15 GMT
• Title: SEFORA: Student Essays with Feedback Corpus and LLM Feedback Evaluation Framework
• Abstract summary: SEFORA is a public corpus pairing instructor inline feedback with assignment prompts, rubrics, scores, and multi-draft revisions.<n>UniMatch is a reference-based evaluation framework for open-ended generation.<n>It segments feedback into feedback units, scores semantic correspondence under instructor-derived criteria, and aligns them via optimal matching to yield interpretable precision, recall, and F1.
• Score: 2.7531308685466183
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective writing feedback is among the strongest drivers of student learning, yet producing it at scale is labor-intensive. LLMs offer a natural path to scaling writing support, but two gaps stand in the way: few public corpora capture how instructors actually deliver feedback in real classrooms, and no reliable method measures whether generated feedback aligns with what an instructor would write. We address both. SEFORA is a public corpus pairing instructor inline feedback with assignment prompts, rubrics, scores, and multi-draft revisions across various college writing genres, comprising 564 drafts and 8,240 instructor annotations. UniMatch is a reference-based evaluation framework for open-ended generation: it segments feedback into feedback units, scores their semantic correspondence under instructor-derived criteria, and aligns them via optimal matching to yield interpretable precision, recall, and F1. Across 74 experimental configurations spanning multiple LLMs, no setting exceeds 0.4 F1. UniMatch reveals that models struggle to identify the feedback instructors would prioritize, and performance degrades as models generate more.

Related papers

• Generating Constructive Feedback on Stories via Reinforcement Learning [17.493071928687993]
  Constructive feedback is crucial for creative writers to refine their storytelling abilities.<n>Since receiving feedback from human experts is often costly and time-intensive, large language models (LLMs) offer a scalable and efficient alternative.<n>We present a reinforcement learning approach that steers LLMs to generate constructive feedback without the need for ground-truth feedback.
  arXiv  Detail & Related papers  (2026-09-04T07:26:23Z)
• Identifying Quality Indicators in Student Self-Reflections in Software Engineering [4.435094091999926]
  This study builds on reflective writing frameworks to produce an eight-indicator scheme for assessing student reflections in software engineering.<n> Furthermore, this study validates an automated classifier for assessing reflections against the framework.
  arXiv  Detail & Related papers  (2026-06-21T21:52:00Z)
• FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays [21.90557945161922]
  We introduce FOXGLOVE, a dataset of 696 feedback comments written by trained writing instructors on 69 twelfth-grade argumentative essays.<n>We find that instructors and LLMs distribute feedback similarly across goals and essay positions, yet instructors and models diverge on the specific sentences on which to provide feedback.
  arXiv  Detail & Related papers  (2026-06-04T15:13:25Z)
• Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators [55.617099475539305]
  Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators.<n>Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction.<n>We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception.
  arXiv  Detail & Related papers  (2026-05-12T20:55:23Z)
• A Systematic Study of Pseudo-Relevance Feedback with LLMs [49.53124785319461]
  Pseudo-relevance feedback (PRF) methods built on large language models (LLMs) can be organized along two key design dimensions.<n>We study how the choice of feedback source and feedback model impact PRF effectiveness through controlled experimentation.
  arXiv  Detail & Related papers  (2026-03-11T17:31:50Z)
• Expanding the Capabilities of Reinforcement Learning via Text Feedback [49.561885700139676]
  We formalize a multi-turn RL setup, RL from Text Feedback (RLTF), where text feedback is available during training but not at inference.<n>To do this, we propose two methods: Self Distillation (RLTF-SD), which trains the single-turn policy to match its own feedback-conditioned second-turn generations; and Feedback Modeling (RLTF-FM), which predicts the feedback as an auxiliary objective.<n>Our results show that both methods consistently outperform strong baselines across benchmarks.
  arXiv  Detail & Related papers  (2026-02-02T18:56:56Z)
• Exposía: Academic Writing Assessment of Exposés and Peer Feedback [56.428320613219306]
  We present Exposa, the first public dataset that connects writing and feedback assessment in higher education.<n>We use Exposa to benchmark state-of-the-art open-source large language models (LLMs) for two tasks: automated scoring of (1) the proposals and (2) the student reviews.
  arXiv  Detail & Related papers  (2026-01-10T11:33:26Z)
• Scaling Equitable Reflection Assessment in Education via Large Language Models and Role-Based Feedback Agents [2.825140278227664]
  Formative feedback is one of the most effective drivers of student learning.<n>In large or low-resource courses, instructors often lack the time, staffing, and bandwidth required to review and respond to every student reflection.<n>This paper presents a theory-grounded system that uses five coordinated role-based LLM agents to score learner reflections.
  arXiv  Detail & Related papers  (2025-11-14T09:46:21Z)
• Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback [57.200668979963694]
  We present a novel test set of 1,300 stories that we corrupted to intentionally introduce writing issues.<n>We study the performance of commonly used LLMs in this task with both automatic and human evaluation metrics.
  arXiv  Detail & Related papers  (2025-07-21T18:56:50Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction [40.76665188171691]
  In the context of English as a Foreign Language (EFL) writing education, LLM-as-a-tutor can assist students by providing real-time feedback on their essays. To bridge this gap, we integrate pedagogical principles to assess student-LLM interaction.
  arXiv  Detail & Related papers  (2023-10-08T15:00:04Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.