A Practical Guide for Supporting Formative Assessment and Feedback Using Generative AI

• URL: http://arxiv.org/abs/2505.23405v2
• Date: Mon, 02 Jun 2025 08:21:31 GMT
• Title: A Practical Guide for Supporting Formative Assessment and Feedback Using Generative AI
• Authors: Sapolnach Prompiengchai, Charith Narreddy, Steve Joordens,
• Abstract summary: Large-language models (LLMs) can help students, teachers, and peers understand "where learners are going," "where learners currently are," and "how to move learners forward"<n>This review provides a comprehensive foundation for integrating LLMs into formative assessment in a pedagogically informed manner.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Formative assessment is a cornerstone of effective teaching and learning, providing students with feedback to guide their learning. While there has been an exponential growth in the application of generative AI in scaling various aspects of formative assessment, ranging from automatic question generation to intelligent tutoring systems and personalized feedback, few have directly addressed the core pedagogical principles of formative assessment. Here, we critically examined how generative AI, especially large-language models (LLMs) such as ChatGPT, can support key components of formative assessment: helping students, teachers, and peers understand "where learners are going," "where learners currently are," and "how to move learners forward" in the learning process. With the rapid emergence of new prompting techniques and LLM capabilities, we also provide guiding principles for educators to effectively leverage cost-free LLMs in formative assessments while remaining grounded in pedagogical best practices. Furthermore, we reviewed the role of LLMs in generating feedback, highlighting limitations in current evaluation metrics that inadequately capture the nuances of formative feedback, such as distinguishing feedback at the task, process, and self-regulatory levels. Finally, we offer practical guidelines for educators and researchers, including concrete classroom strategies and future directions such as developing robust metrics to assess LLM-generated feedback, leveraging LLMs to overcome systemic and cultural barriers to formative assessment, and designing AI-aware assessment strategies that promote transferable skills while mitigating overreliance on LLM-generated responses. By structuring the discussion within an established formative assessment framework, this review provides a comprehensive foundation for integrating LLMs into formative assessment in a pedagogically informed manner.

Related papers

• Partnering with AI: A Pedagogical Feedback System for LLM Integration into Programming Education [19.441958600393342]
  This paper introduces a novel framework for large language models (LLMs)-driven feedback generation.<n>Our findings suggest that teachers consider that, when aligned with the framework, LLMs can effectively support students.<n>However, we found several limitations, such as its inability to adapt feedback to dynamic classroom contexts.
  arXiv  Detail & Related papers  (2025-07-01T03:48:48Z)
• OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics [101.78963920333342]
  We introduce OpenUnlearning, a standardized framework for benchmarking large language models (LLMs) unlearning methods and metrics.<n>OpenUnlearning integrates 9 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks.<n>We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite.
  arXiv  Detail & Related papers  (2025-06-14T20:16:37Z)
• Training an LLM-as-a-Judge Model: Pipeline, Insights, and Practical Lessons [9.954960702259918]
  This paper introduces Themis, a fine-tuned large language model (LLMs) judge that delivers context-aware evaluations.<n>We provide a comprehensive overview of the development pipeline for Themis, highlighting its scenario-dependent evaluation prompts.<n>We introduce two human-labeled benchmarks for meta-evaluation, demonstrating that Themis can achieve high alignment with human preferences in an economical manner.
  arXiv  Detail & Related papers  (2025-02-05T08:35:55Z)
• Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors [7.834688858839734]
  We investigate whether current state-of-the-art large language models (LLMs) are effective as AI tutors.<n>We propose a unified evaluation taxonomy with eight pedagogical dimensions based on key learning sciences principles.<n>We release MRBench - a new evaluation benchmark containing 192 conversations and 1,596 responses from seven state-of-the-art LLM-based and human tutors.
  arXiv  Detail & Related papers  (2024-12-12T16:24:35Z)
• An Exploration of Higher Education Course Evaluation by Large Language Models [4.943165921136573]
  Large language models (LLMs) within artificial intelligence (AI) present promising new avenues for enhancing course evaluation processes. This study explores the application of LLMs in automated course evaluation from multiple perspectives and conducts rigorous experiments across 100 courses at a major university in China.
  arXiv  Detail & Related papers  (2024-11-03T20:43:52Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course [49.296957552006226]
  Using large language models (LLMs) for automatic evaluation has become an important evaluation method in NLP research. This report shares how we use GPT-4 as an automatic assignment evaluator in a university course with 1,028 students.
  arXiv  Detail & Related papers  (2024-07-07T00:17:24Z)
• DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation [75.81096662788254]
  Large Language Models (LLMs) are scalable and economical evaluators.<n>The question of how reliable these evaluators are has emerged as a crucial research question.<n>We propose Decompose and Aggregate, which breaks down the evaluation process into different stages based on pedagogical practices.
  arXiv  Detail & Related papers  (2024-05-24T08:12:30Z)
• Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences [0.0]
  This work advocates careful and caring AIED research by going through previous research on feedback generation in ITS. The main contributions of this paper include: an avocation of applying more cautious, theoretically grounded methods in feedback generation in the era of generative AI.
  arXiv  Detail & Related papers  (2024-05-07T20:09:18Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation [60.65820977963331]
  We introduce a novel evaluation paradigm for Large Language Models (LLMs) This paradigm shifts the emphasis from result-oriented assessments, which often neglect the reasoning process, to a more comprehensive evaluation. By applying this paradigm in the GSM8K dataset, we have developed the MR-GSM8K benchmark.
  arXiv  Detail & Related papers  (2023-12-28T15:49:43Z)
• A Survey on Evaluation of Large Language Models [87.60417393701331]
  Large language models (LLMs) are gaining increasing popularity in both academia and industry. This paper focuses on three key dimensions: what to evaluate, where to evaluate, and how to evaluate.
  arXiv  Detail & Related papers  (2023-07-06T16:28:35Z)
• Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic Approach [40.06500618820166]
  This paper presents an approach to deriving a learner model directly from an assessment rubric. We illustrate how the approach can be applied to automatize the human assessment of an activity developed for testing computational thinking skills.
  arXiv  Detail & Related papers  (2022-09-07T10:09:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.