A Large-Scale Observational Study on Obtaining Lightweight, Randomized Weekly Student Feedback

• URL: http://arxiv.org/abs/2605.02281v1
• Date: Mon, 04 May 2026 07:12:34 GMT
• Title: A Large-Scale Observational Study on Obtaining Lightweight, Randomized Weekly Student Feedback
• Abstract summary: High-Resolution Course Feedback (HRCF) elicits feedback that instructors find helpful without imposing excessive burden on students.<n>We study HRCF use across 103 course offerings, totaling 24,216 student enrollments, over four years from Fall 2021 through Fall 2025 at an R1 institution.<n>We find that continued HRCF use is associated with average rating increases of 0.045 to 0.048 points per additional term of use for learning-related items.
• Score: 3.2485598296072156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional methods of obtaining student feedback on course experience face a fundamental tradeoff between feedback frequency and quality: as feedback requests become more frequent, participation often declines, and responses become less thoughtful over time. To obtain both timely and thoughtful feedback from students, Kim and Piech (Learning at Scale, 2023) recently proposed a simple, lightweight course feedback mechanism: surveying each student a small number of times per term during randomly selected weeks. Named High-Resolution Course Feedback (HRCF), this method has been shown to elicit feedback that instructors find helpful without imposing excessive burden on students. An important question, however, remains unanswered: is the use of this simple method associated with measurable improvements in students' actual course experiences? We study HRCF use across 103 course offerings, totaling 24,216 student enrollments, over four years from Fall 2021 through Fall 2025, spanning 42 unique computer science courses at an R1 institution. Through a regression analysis of four end-of-term student evaluation items for these courses, we find that first-time use of HRCF is not associated with a measurable change in average student ratings. However, among small- and medium-enrollment (<250 students) course offerings, continued HRCF use is associated with average rating increases of 0.045 to 0.048 points per additional term of use for learning-related items. We observe no statistically significant associations for large-enrollment (250 or more students) course offerings, nor for items measuring instructional quality and course organization. Together, these findings suggest that sustained HRCF use may support improvements in students' learning experiences, but that further design enhancements may be needed to produce measurable improvements in instructional quality and course organization.

Related papers

• The "Astonishing Regularity'' Revisited: Sensitivity of Learning-Rate Estimates to Practice-Sequence Length [0.7966335996690441]
  We probe unexamined assumption that observation length in student log data is ignorable for mixed-effects estimation.<n>We show that capping at the first ten opportunities per student-skill pair inflates the median estimated IQR of student learning rates by 75%.<n>We argue that practice sequence length distributions are an unexamined property of mixed-effects estimation on observational learning data.
  arXiv  Detail & Related papers  (2026-05-03T03:07:36Z)
• Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment [82.00769536768509]
  Rank-Surprisal Ratio is a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.<n>We demonstrate its practical utility in both trajectory selection and teacher selection.
  arXiv  Detail & Related papers  (2026-01-20T18:58:10Z)
• Re-Rankers as Relevance Judges [65.37611299805856]
  We reproduce re-rankers in a re-ranker-as-relevance-judge setup.<n>We perform experiments on TREC-DL 2019 to 2023 with 8 re-rankers from 3 families, ranging from 220M to 32B, and analyse the evaluation bias exhibited by re-ranker-based judges.
  arXiv  Detail & Related papers  (2026-01-08T00:02:59Z)
• CS-Guide: Leveraging LLMs and Student Reflections to Provide Frequent, Scalable Academic Monitoring Feedback to Computer Science Students [2.2012643583422347]
  CS departments often serve large student populations, making timely academic monitoring and personalized feedback difficult.<n>We present CS-Guide, which leverages Large Language Models (LLMs) to deliver scalable, frequent academic feedback.<n>We evaluated CS-Guide on a four-year, 20K-entry longitudinal dataset, and it achieved up to a 97% F1 score in recommending interventions for first-year students.
  arXiv  Detail & Related papers  (2025-12-22T20:43:59Z)
• Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains [97.5573252172065]
  We train a family of Automatic Reasoning Evaluators (FARE) with a simple iterative rejection-sampling supervised finetuning approach.<n>FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators.<n>As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH.
  arXiv  Detail & Related papers  (2025-10-20T17:52:06Z)
• How Adding Metacognitive Requirements in Support of AI Feedback in Practice Exams Transforms Student Learning Behaviors [1.3285222309805063]
  We present an empirically evaluated practice exam system that integrates AI generated feedback with targeted textbook references.<n>Our system encourages metacognitive behavior by asking students to explain their answers and declare their confidence.<n>It uses OpenAI's GPT-4o to generate personalized feedback based on this information, while directing them to relevant textbook sections.
  arXiv  Detail & Related papers  (2025-05-19T17:25:07Z)
• A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look [52.114284476700874]
  This paper reports on the results of a large-scale evaluation (the TREC 2024 RAG Track) where four different relevance assessment approaches were deployed. We find that automatically generated UMBRELA judgments can replace fully manual judgments to accurately capture run-level effectiveness. Surprisingly, we find that LLM assistance does not appear to increase correlation with fully manual assessments, suggesting that costs associated with human-in-the-loop processes do not bring obvious tangible benefits.
  arXiv  Detail & Related papers  (2024-11-13T01:12:35Z)
• Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy [0.0]
  This paper introduces AMMORE, a new dataset of 53,000 math open-response question-answer pairs from Rori. We conduct two experiments to evaluate the use of large language models (LLM) for grading challenging student answers.
  arXiv  Detail & Related papers  (2024-09-26T14:51:40Z)
• Efficient course recommendations with T5-based ranking and summarization [2.6968321526169503]
  We develop a two-step retrieval pipeline with RankT5 finetuned on MSMARCO as re-ranker. We evaluate our rankers on two newly labelled datasets, with an A/B test, and with a user questionnaire. We conclude that T5-based re-ranking and summarization for online course recommendation can obtain much better effectiveness than single-step lexical retrieval.
  arXiv  Detail & Related papers  (2024-06-27T09:07:32Z)
• SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model [73.80068155830708]
  We present an extensive analysis for continual learning on a pre-trained model (CLPM) We propose a simple but extremely effective approach named Slow Learner with Alignment (SLCA) Across a variety of scenarios, our proposal provides substantial improvements for CLPM.
  arXiv  Detail & Related papers  (2023-03-09T08:57:01Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering [49.024513062811685]
  We develop a simple and effective re-ranking approach (RECONSIDER) for span-extraction tasks. RECONSIDER is trained on positive and negative examples extracted from high confidence predictions of MRC models. It uses in-passage span annotations to perform span-focused re-ranking over a smaller candidate set.
  arXiv  Detail & Related papers  (2020-10-21T04:28:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.