Using Learning Progressions to Guide AI Feedback for Science Learning

• URL: http://arxiv.org/abs/2603.03249v1
• Date: Tue, 03 Mar 2026 18:39:58 GMT
• Title: Using Learning Progressions to Guide AI Feedback for Science Learning
• Abstract summary: Generative artificial intelligence (AI) offers scalable support for formative feedback.<n>While effective, rubric authoring is time-consuming and limits scalability across instructional contexts.<n>This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics.
• Score: 9.90590245204881
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing Clarity, Accuracy, Relevance, Engagement and Motivation, and Reflectiveness (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's kappa values for estimable dimensions (kappa = .66 to .88). Paired t-tests revealed no statistically significant differences between the two pipelines for Clarity (t1 = 0.00, p1 = 1.000; t2 = 0.84, p2 = .399), Relevance (t1 = 0.28, p1 = .782; t2 = -0.58, p2 = .565), Engagement and Motivation (t1 = 0.50, p1 = .618; t2 = -0.58, p2 = .565), or Reflectiveness (t = -0.45, p = .656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution.

Related papers

• Simulating Disengaged Students to Evaluate LLM-based Tutors [0.40105987447353786]
  Disengagement-Aware Student Simulators (DAS2) models five learner-engagement states: engaged, gaming, wheel-spinning, off-task, and mixed.<n>DAS2 provides a pre-deployment framework for evaluating how AI tutors respond to diverse learner-engagement states before deployment.
  arXiv  Detail & Related papers  (2026-09-11T01:28:48Z)
• TutorTrace: A Dataset and Taxonomy for Classifying Learner Behavioral States during AI-Assisted Programming Education [10.562166959716771]
  TutorTrace is a dataset and behavioral abstraction pipeline that makes learners' behavioral context visible and computable in real time.<n>We derive a taxonomy of learner activity before the first AI query, between consecutive queries, and across the full session.<n>In a preliminary classroom evaluation, behavior-aware prompts were associated with a decrease in intervals between queries with no independent work from 50.0% to 20.7%.
  arXiv  Detail & Related papers  (2026-08-22T11:40:23Z)
• AI-Assisted Code Review as a Scaffold for Code Quality and Self-Regulated Learning: An Experience Report [4.541490370053892]
  We investigate an LLM-as-reviewer integrated directly into GitHub pull requests (human-in-the-loop)<n>We examine engagement and responsiveness as behavioral indicators of self-regulated learning processes.<n>We contribute: (i) an in-workflow design for an AI reviewer that scaffolds learning while mitigating cognitive offloading; (ii) a repeated cross sectional comparison across two cohorts in authentic settings; and (iv) evidence-based pedagogical recommendations for responsible, student-led AI-assisted review.
  arXiv  Detail & Related papers  (2026-04-25T11:10:57Z)
• PeerPrism: Peer Evaluation Expertise vs Review-writing AI [12.533035088439975]
  We introduce PeerPrism, a benchmark of 20,690 peer reviews explicitly designed to disentangle idea provenance from text provenance.<n>We benchmark state-of-the-art LLM text detection methods on PeerPrism.<n>Our results show that current detection methods conflate surface realization with intellectual contribution.
  arXiv  Detail & Related papers  (2026-04-16T00:59:55Z)
• Evaluating Generative AI for CS1 Code Grading: Direct vs Reverse Methods [0.0]
  This paper compares two AI-based grading techniques: textitDirect, where the AI model applies a rubric directly to student code, and textitReverse (a newly proposed approach), where the AI first fixes errors, then deduces a grade based on the nature and number of fixes.<n>We discuss the strengths and limitations of each approach, practical considerations for prompt design, and future directions for hybrid human-AI grading systems.
  arXiv  Detail & Related papers  (2025-11-17T01:38:06Z)
• Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning [137.33138614095435]
  Retrieval-augmented generation (RAG) has proven to be effective in mitigating hallucinations in large language models.<n>Recent efforts have incorporated search-based interactions into RAG, enabling iterative reasoning with real-time retrieval.<n>We propose Bi-RAR, a novel retrieval-augmented reasoning framework that evaluates each intermediate step jointly in both forward and backward directions.
  arXiv  Detail & Related papers  (2025-11-12T08:29:39Z)
• Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation [5.555479009357263]
  Asynchronous patient-clinician messaging via EHR portals is a growing source of clinician workload.<n>Our contributions are threefold: (1) we introduce a clinically grounded error ontology comprising 5 domains and 59 granular error codes; (2) we develop a retrieval-augmented evaluation pipeline; and (3) we provide a two-stage prompting architecture using DSPy to enable scalable, interpretable, and hierarchical error detection.
  arXiv  Detail & Related papers  (2025-09-26T16:42:43Z)
• Automated Classification of Tutors' Dialogue Acts Using Generative AI: A Case Study Using the CIMA Corpus [10.325932865188514]
  This case study uses the open-source CIMA corpus, in which tutors' responses are pre-annotated into four DA categories.<n>Results show that GPT-4 achieved 80% accuracy, a weighted F1-score of 0.81, and a Cohen's Kappa of 0.74.
  arXiv  Detail & Related papers  (2025-09-11T03:36:03Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation [60.620408007636016]
  We propose T2I-Eval-R1, a novel reinforcement learning framework that trains open-source MLLMs using only coarse-grained quality scores.<n>Our approach integrates Group Relative Policy Optimization into the instruction-tuning process, enabling models to generate both scalar scores and interpretable reasoning chains.
  arXiv  Detail & Related papers  (2025-05-23T13:44:59Z)
• VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use [49.574651930395305]
  VisIT-Bench is a benchmark for evaluation of instruction-following vision-language models. Our dataset comprises 592 test queries, each with a human-authored instruction-conditioned caption. We quantify quality gaps between models and references using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2023-08-12T15:27:51Z)
• Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education [10.606131520965604]
  This study investigates AI content detection in a rarely explored yet realistic setting. We first formalized the detection task as identifying the transition points between human-written content and AI-generated content. We then proposed a two-step approach where we separated AI-generated content from human-written content during the encoder training process.
  arXiv  Detail & Related papers  (2023-07-23T08:47:51Z)
• INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback [80.57617091714448]
  We present InstructScore, an explainable evaluation metric for text generation. We fine-tune a text evaluation metric based on LLaMA, producing a score for generated text and a human readable diagnostic report.
  arXiv  Detail & Related papers  (2023-05-23T17:27:22Z)
• Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values [73.82043713141142]
  Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. We introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command.
  arXiv  Detail & Related papers  (2022-10-14T09:10:49Z)
• Interactive Code Generation via Test-Driven User-Intent Formalization [60.90035204567797]
  Large language models (LLMs) produce code from informal natural language (NL) intent. It is hard to define a notion of correctness since natural language can be ambiguous and lacks a formal semantics. We describe a language-agnostic abstract algorithm and a concrete implementation TiCoder.
  arXiv  Detail & Related papers  (2022-08-11T17:41:08Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.