AI-Assisted Code Review as a Scaffold for Code Quality and Self-Regulated Learning: An Experience Report

• URL: http://arxiv.org/abs/2604.23251v1
• Date: Sat, 25 Apr 2026 11:10:57 GMT
• Title: AI-Assisted Code Review as a Scaffold for Code Quality and Self-Regulated Learning: An Experience Report
• Abstract summary: We investigate an LLM-as-reviewer integrated directly into GitHub pull requests (human-in-the-loop)<n>We examine engagement and responsiveness as behavioral indicators of self-regulated learning processes.<n>We contribute: (i) an in-workflow design for an AI reviewer that scaffolds learning while mitigating cognitive offloading; (ii) a repeated cross sectional comparison across two cohorts in authentic settings; and (iv) evidence-based pedagogical recommendations for responsible, student-led AI-assisted review.
• Score: 4.541490370053892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code review is central to software engineering education but hard to scale in capstone projects due to tight deadlines, uneven peer feedback, and limited prior experience. We investigate an LLM-as-reviewer integrated directly into GitHub pull requests (human-in-the-loop) across two cohorts (more than 100 students, 2023--2024). Using a mixed-methods design -- GitHub data, reflective reports, and a targeted survey -- we examine engagement and responsiveness as behavioral indicators of self-regulated learning processes. Quantitatively, the 2024 cohort produced more iterative activity (1176 vs. 581 PRs), while technical issues observed in 2023 (227 failed AI attempts) dropped to zero after tool and instructional refinements. Despite different adoption levels (93\% vs. 50\% of teams using the tool), responsiveness was stable: 32\% (2023) and 33\% (2024) of successfully AI-reviewed PRs were followed by subsequent commits on the same PR. Qualitatively, students used the LLM's structured comments to focus reviews and discuss code quality, while guidance reduced over-reliance. We contribute: (i) an in-workflow design for an AI reviewer that scaffolds learning while mitigating cognitive offloading; (ii) a repeated cross sectional comparison across two cohorts in authentic settings; (iii) a mixed-methods analysis combining objective GitHub metrics with student self-reports; and (iv) evidence-based pedagogical recommendations for responsible, student-led AI-assisted review.

Related papers

• AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective [11.163797770670683]
  This study analyzed AI-assisted help-seeking trajectories in university-level programming.<n>Results indicate that many students primarily used AI for reactive troubleshooting rather than for planned, self-regulated problem-solving.
  arXiv  Detail & Related papers  (2026-06-22T03:37:58Z)
• Empirical Study on the Characteristics and Evolution of AI-usage in GitHub Repositories: Evidence from Code Comments [7.213400161126317]
  We analyze 35,361 GitHub code comments that explicitly reference AI use and their associated code blocks.<n>We first open-code 500 unique comments and code blocks to derive a taxonomy of AI-assisted development activities.<n>We then annotate the full dataset using two LLM-based classifiers and aggregate predictions with Dawid-Skene expectation-maximization.
  arXiv  Detail & Related papers  (2026-06-05T02:37:44Z)
• AcademiClaw: When Students Set Challenges for AI Agents [69.70303995121154]
  AcademiClaw is a benchmark of 80 complex, long-horizon tasks sourced directly from university students' real academic disconnects.<n>We show that even the best of models achieves only a 55% pass rate.<n>We hope that AcademiClaw and its open-sourced data and code can serve as a useful resource for the OpenClaw community.
  arXiv  Detail & Related papers  (2026-05-04T14:40:42Z)
• An AI Teaching Assistant for Motion Picture Engineering [0.0]
  This article describes implementation of an AI Teaching Assistant (AI-TA) at Trinity College Dublin's Master's Motion Picture Engineering (MPE) course.<n>We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics.<n>Student feedback revealed that the AI-TA was beneficial, while students had mixed feelings about preferring it over human tutoring.
  arXiv  Detail & Related papers  (2026-04-06T13:30:11Z)
• Using Learning Progressions to Guide AI Feedback for Science Learning [9.90590245204881]
  Generative artificial intelligence (AI) offers scalable support for formative feedback.<n>While effective, rubric authoring is time-consuming and limits scalability across instructional contexts.<n>This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics.
  arXiv  Detail & Related papers  (2026-03-03T18:39:58Z)
• Early-Stage Prediction of Review Effort in AI-Generated Pull Requests [0.0]
  We analyze 33,707 agent-authored PRs from the AIDev dataset across 2,807 repositories.<n>We propose a Circuit Breaker triage model that predicts high-review-effort PRs at creation time.
  arXiv  Detail & Related papers  (2026-01-02T17:18:01Z)
• Enhancing Python Programming Education with an AI-Powered Code Helper: Design, Implementation, and Impact [0.0]
  This study presents an AI-Python-based chatbots that helps students learn programming.<n>The system demonstrated an 85% error resolution success rate, outperforming standalone tools like pylint.<n>The research provides a blueprint for AI tools that prioritize educational equity and long-term skill retention over mere code completion.
  arXiv  Detail & Related papers  (2025-09-24T19:43:46Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• From Coders to Critics: Empowering Students through Peer Assessment in the Age of AI Copilots [3.3094795918443634]
  This paper presents an empirical study of a rubric based, anonymized peer review process implemented in a large programming course.<n>Students evaluated each other's final projects (2D game) and their assessments were compared to instructor grades using correlation, mean absolute error, and root mean square error (RMSE)<n>Results show that peer review can approximate instructor evaluation with moderate accuracy and foster student engagement, evaluative thinking, and interest in providing good feedback to their peers.
  arXiv  Detail & Related papers  (2025-05-28T08:17:05Z)
• A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look [52.114284476700874]
  This paper reports on the results of a large-scale evaluation (the TREC 2024 RAG Track) where four different relevance assessment approaches were deployed. We find that automatically generated UMBRELA judgments can replace fully manual judgments to accurately capture run-level effectiveness. Surprisingly, we find that LLM assistance does not appear to increase correlation with fully manual assessments, suggesting that costs associated with human-in-the-loop processes do not bring obvious tangible benefits.
  arXiv  Detail & Related papers  (2024-11-13T01:12:35Z)
• Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews [51.453135368388686]
  We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM) Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level.
  arXiv  Detail & Related papers  (2024-03-11T21:51:39Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.