eRevise+RF: A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedback

• URL: http://arxiv.org/abs/2501.00715v1
• Date: Wed, 01 Jan 2025 03:49:48 GMT
• Title: eRevise+RF: A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedback
• Authors: Zhexiong Liu, Diane Litman, Elaine Wang, Tianwen Li, Mason Gobat, Lindsay Clare Matsumura, Richard Correnti,
• Abstract summary: eRevise+RF is an enhanced AWE system for assessing student essay revisions and providing revision feedback. We deployed the system with 6 teachers and 406 students across 3 schools in Pennsylvania and Louisiana. The results confirmed its effectiveness in (1) assessing student essays in terms of evidence usage, (2) extracting evidence and reasoning revisions across essays, and (3) determining revision success in responding to feedback.
• Score: 1.5367711550341163
• License:
• Abstract: The ability to revise essays in response to feedback is important for students' writing success. An automated writing evaluation (AWE) system that supports students in revising their essays is thus essential. We present eRevise+RF, an enhanced AWE system for assessing student essay revisions (e.g., changes made to an essay to improve its quality in response to essay feedback) and providing revision feedback. We deployed the system with 6 teachers and 406 students across 3 schools in Pennsylvania and Louisiana. The results confirmed its effectiveness in (1) assessing student essays in terms of evidence usage, (2) extracting evidence and reasoning revisions across essays, and (3) determining revision success in responding to feedback. The evaluation also suggested eRevise+RF is a helpful system for young students to improve their argumentative writing skills through revision and formative feedback.

Related papers

• Automatic Feedback Generation for Short Answer Questions using Answer Diagnostic Graphs [21.965223446869064]
  Short-reading comprehension questions help students understand text structure but lack effective feedback. Students struggle to identify and correct errors, while manual feedback creation is labor-intensive. We propose a system that generates feedback for student responses.
  arXiv  Detail & Related papers  (2025-01-27T04:49:10Z)
• Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation [13.854903594424876]
  Large language models (LLMs) have demonstrated strong performance in generating coherent and contextually relevant text. This work explores several prompting strategies for LLM-based zero-shot and few-shot generation of essay feedback. Inspired by Chain-of-Thought prompting, we study how and to what extent automated essay scoring (AES) can benefit the quality of generated feedback.
  arXiv  Detail & Related papers  (2024-04-24T12:48:06Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL) Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Evaluation of ChatGPT Feedback on ELL Writers' Coherence and Cohesion [0.7028778922533686]
  ChatGPT has had a transformative effect on education where students are using it to help with homework assignments and teachers are actively employing it in their teaching practices. This study evaluated the quality of the feedback generated by ChatGPT regarding the coherence and cohesion of the essays written by English Language learners (ELLs) students.
  arXiv  Detail & Related papers  (2023-10-10T10:25:56Z)
• PapagAI:Automated Feedback for Reflective Essays [48.4434976446053]
  We present the first open-source automated feedback tool based on didactic theory and implemented as a hybrid AI system. The main objective of our work is to enable better learning outcomes for students and to complement the teaching activities of lecturers.
  arXiv  Detail & Related papers  (2023-07-10T11:05:51Z)
• Review of feedback in Automated Essay Scoring [6.445605125467574]
  The first automated essay scoring system was developed 50 years ago. This paper reviews research on feedback including different feedback types and essay traits on automated essay scoring.
  arXiv  Detail & Related papers  (2023-07-09T11:04:13Z)
• SETSum: Summarization and Visualization of Student Evaluations of Teaching [74.76373136325032]
  Student Evaluations of Teaching (SETs) are widely used in colleges and universities. SETSum provides organized illustrations of SET findings to instructors and other reviewers.
  arXiv  Detail & Related papers  (2022-07-08T01:40:11Z)
• Ranking Scientific Papers Using Preference Learning [48.78161994501516]
  We cast it as a paper ranking problem based on peer review texts and reviewer scores. We introduce a novel, multi-faceted generic evaluation framework for making final decisions based on peer reviews.
  arXiv  Detail & Related papers  (2021-09-02T19:41:47Z)
• Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing [0.9449650062296824]
  We introduce an annotation scheme to capture the nature of sentence-level revisions of evidence use and reasoning. We show that reliable manual annotation can be achieved and that revision annotations correlate with a holistic assessment of essay improvement.
  arXiv  Detail & Related papers  (2021-07-14T20:58:26Z)
• ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis [62.76038841302741]
  We build a novel ReviewRobot to automatically assign a review score and write comments for multiple categories such as novelty and meaningful comparison. Experimental results show that our review score predictor reaches 71.4%-100% accuracy. Human assessment by domain experts shows that 41.7%-70.5% of the comments generated by ReviewRobot are valid and constructive, and better than human-written ones for 20% of the time.
  arXiv  Detail & Related papers  (2020-10-13T02:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.