Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar

• URL: http://arxiv.org/abs/2605.17055v1
• Date: Sat, 16 May 2026 15:55:39 GMT
• Title: Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar
• Abstract summary: The study involved 143 students and five teachers in grades 7 through 11 across five U.S. middle and high schools.<n>Students valued CyberScholar's immediate, rubric-based feedback and noticed improvements in their writing as they revised.<n>However, participants noted inconsistencies in the automated rating system and occasional misalignment with assignment expectations.
• Score: 0.11184789007828978
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This multiple-case study examined the potential of a Generative AI (GenAI) tool, CyberScholar, to support K-12 students' writing across disciplines. This tool integrates teacher-provided rubrics, materials, and exemplars through Retrieval-Augmented Generation (RAG), producing criterion-specific formative feedback and ratings. The study involved 143 students and five teachers in grades 7 through 11 across five U.S. middle and high schools. Data sources included classroom observations, student post-surveys (n = 79), student focus group interviews (n = 18), and teacher surveys (n = 5). Qualitative analysis followed two cycles of coding to identify patterns within and across cases. Findings indicate that students valued CyberScholar's immediate, rubric-based feedback and noticed improvements in their writing as they revised, using it to refine organization, elaboration, and style. They also highlighted the tool's interactive, iterative qualities, which fostered revision and reduced reliance on teacher feedback. However, participants noted inconsistencies in the automated rating system and occasional misalignment with assignment expectations. Teachers reported that CyberScholar saved time on feedback and supported more targeted, higher-order instructional practices. The study underscores the promise of rubric-grounded GenAI formative feedback for developing writing skills, while emphasizing the need for human oversight, calibration of automated ratings, and attention to contextual factors shaping adoption.

Related papers

• Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education [0.0]
  The integration of GenAI tools into higher education assessment raises important questions about how students understand, interpret, and respond to AI-mediated evaluation.<n>This study reports findings from a qualitative pedagogical inquiry conducted in an undergraduate technical communication course for computing students at a Saudi public university.<n>Students were explicitly informed that ChatGPT had generated the score and feedback and were invited to reflect on the evaluation in writing.
  arXiv  Detail & Related papers  (2026-09-04T16:49:33Z)
• PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers on Overleaf [51.79108552674703]
  We present PaperMentor, a human-centered writing assistant system that delivers actionable suggestions as Overleaf-native inline comments.<n>PaperMentor integrates an expert skill library carefully curated from established researchers' writing advice with 12 specialized agents covering different aspects of paper writing.<n>In a user study, 90.6% of the generated comments were rated actionable and 67.5% were rated valid, significantly outperforming a GPT-5.2 baseline uswithout the skill library.
  arXiv  Detail & Related papers  (2026-06-07T22:17:15Z)
• MAIC-UI: Making Interactive Courseware with Generative UI [55.66338307665182]
  We present MAIC-UI, a zero-code authoring system that enables educators to create and rapidly edit interactive courseware.<n>MAIC-UI employs structured knowledge analysis with multi-modal understanding to ensure pedagogical rigor.
  arXiv  Detail & Related papers  (2026-04-28T16:15:04Z)
• Understanding Teacher Revisions of Large Language Model-Generated Feedback [0.8039067099377077]
  We analyze 1,349 instances of AI-generated feedback and corresponding teacher-edited explanations from 117 teachers.<n>We find that teachers accept AI feedback without modification in about 80% of cases, while edited feedback tends to be significantly longer and subsequently shortened by teachers.<n>Machine learning models trained only on the AI feedback text as input features achieve fair performance in identifying which feedback will be revised.
  arXiv  Detail & Related papers  (2026-03-29T18:41:00Z)
• ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review [48.60540055009675]
  ScholarPeer is a search-enabled multi-agent framework designed to emulate the cognitive processes of a senior researcher.<n>We evaluate ScholarPeer on DeepReview-13K and the results demonstrate that ScholarPeer achieves significant win-rates against state-of-the-art approaches in side-by-side evaluations.
  arXiv  Detail & Related papers  (2026-01-30T06:54:55Z)
• Exposía: Academic Writing Assessment of Exposés and Peer Feedback [56.428320613219306]
  We present Exposa, the first public dataset that connects writing and feedback assessment in higher education.<n>We use Exposa to benchmark state-of-the-art open-source large language models (LLMs) for two tasks: automated scoring of (1) the proposals and (2) the student reviews.
  arXiv  Detail & Related papers  (2026-01-10T11:33:26Z)
• Examining Student Interactions with a Pedagogical AI-Assistant for Essay Writing and their Impact on Students Writing Quality [4.112932467662682]
  The dynamic nature of interactions between students and GenAI, as well as their relationship to writing quality, remains underexplored.<n>We evaluated a GenAI-driven essay-writing assistant (EWA) designed to support higher education students in argumentative writing.
  arXiv  Detail & Related papers  (2025-12-09T13:34:33Z)
• From Coders to Critics: Empowering Students through Peer Assessment in the Age of AI Copilots [3.3094795918443634]
  This paper presents an empirical study of a rubric based, anonymized peer review process implemented in a large programming course.<n>Students evaluated each other's final projects (2D game) and their assessments were compared to instructor grades using correlation, mean absolute error, and root mean square error (RMSE)<n>Results show that peer review can approximate instructor evaluation with moderate accuracy and foster student engagement, evaluative thinking, and interest in providing good feedback to their peers.
  arXiv  Detail & Related papers  (2025-05-28T08:17:05Z)
• Generative AI in K-12 Education: The CyberScholar Initiative [1.3461796810226618]
  This paper focuses on the piloting of CyberScholar, a Generative AI assistant tool that aims to provide formative feedback on writing in K-12 contexts.<n>This study explores how students worked with CyberScholar in diverse subject areas, including English Language Arts, Social Studies, and Modern World History classes.
  arXiv  Detail & Related papers  (2025-01-28T18:23:03Z)
• Improving the Validity of Automatically Generated Feedback via Reinforcement Learning [46.667783153759636]
  We propose a framework for feedback generation that optimize both correctness and alignment using reinforcement learning (RL)<n>Specifically, we use GPT-4's annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-02T20:25:50Z)
• Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning [43.83422798569986]
  Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and reliable form of assessment. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers. We propose a simple, in-context learning-based solution for automated distractor and corresponding feedback message generation.
  arXiv  Detail & Related papers  (2023-08-07T01:03:04Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.