Measuring Changes in Instructor Class Design and Student Learning After the Release of Large Language Models (LLMs)

• URL: http://arxiv.org/abs/2605.16284v1
• Date: Sun, 12 Apr 2026 14:55:04 GMT
• Title: Measuring Changes in Instructor Class Design and Student Learning After the Release of Large Language Models (LLMs)
• Abstract summary: Student use of Generative AI (GenAI) products in completing their classwork has resulted in substantial shifts in higher education.<n>While GenAI use is widespread, its impact on student study methods, faculty course development, grade reporting, and overall learning is not well documented.<n>This research seeks to identify and document patterns in student and faculty perceptions of, and experiences in, the use of LLMs as a learning tool inside and outside of the university classroom.
• Score: 1.526156659531584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Student use of Generative AI (GenAI) products in completing their classwork, with or without their professors' knowledge and/or approval, has resulted in substantial shifts in higher education. While GenAI use is widespread, its impact on student study methods, faculty course development, grade reporting, and overall learning is not well documented. This is a mixed-methods, multi-course study using retrospective quantitative analysis, instructor surveys, and anonymous student surveys at a university in the New England region of the United States. This research seeks to identify and document patterns in student and faculty perceptions of, and experiences in, the use of LLMs as a learning tool inside and outside of the university classroom. Alongside quantitative and thematic analysis of both faculty and student survey responses, historical grade data as reported to the university registrar is used to triangulate the phenomenon of learning achievement in pre- and post-LLM eras. It is hoped that this research can serve as a pilot study for a broader set of institutions. Results from this study can inform GenAI policy for professors, universities, and other educational institutions that are trying to maximize student learning in the age of AI.

Related papers

• Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course [0.0]
  This research paper examines how different forms of learner-AI interaction relate to learning outcomes in object-oriented programming (OOP) courses.<n>Survey data were collected from 210 undergraduate students enrolled in a first-year OOP course.
  arXiv  Detail & Related papers  (2026-05-26T07:48:00Z)
• Exploring Student Perception on Gen AI Adoption in Higher Education: A Descriptive Study [1.978884131103313]
  This study examines how students perceive, use, and evaluate Generative Artificial Intelligence (GenAI)<n>Data were collected through a questionnaire administered to 436 postgraduate Computer Science students at the University of Hertfordshire.
  arXiv  Detail & Related papers  (2026-03-29T17:19:39Z)
• Integrating Generative AI in BIM Education: Insights from Classroom Implementation [0.4805964026801514]
  This study evaluates the implementation of a Generative AI-powered rule checking workflow within a graduate-level Building Information Modeling course.<n>Over two semesters, 55 students participated in a classroom-based pilot exploring the use of GenAI for BIM compliance tasks.
  arXiv  Detail & Related papers  (2025-07-06T03:41:04Z)
• Is ChatGPT Massively Used by Students Nowadays? A Survey on the Use of Large Language Models such as ChatGPT in Educational Settings [0.25782420501870296]
  This study investigates how 395 students aged 13 to 25 years old in France and Italy integrate Large Language Models (LLMs) into their educational routines.<n>Key findings include the widespread use of these tools across all age groups and disciplines.<n>Results also show gender disparities, raising concerns about an emerging AI literacy and technological gender gap.
  arXiv  Detail & Related papers  (2024-12-23T11:29:44Z)
• Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students [53.20318273452059]
  Large language models (LLMs) like OpenAI's ChatGPT have opened up new avenues in education.<n>Despite school restrictions, our survey of over 300 middle and high school students revealed that a remarkable 70% of students have utilized LLMs.<n>We propose a few ideas to address such issues, including subject-specific models, personalized learning, and AI classrooms.
  arXiv  Detail & Related papers  (2024-11-27T19:19:34Z)
• Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI [41.96102438774773]
  This work presents the findings from a university-level competition, which challenged participants to design prompts for eliciting biased outputs from GenAI tools. We quantitatively and qualitatively analyze the competition submissions and identify a diverse set of biases in GenAI and strategies employed by participants to induce bias in GenAI.
  arXiv  Detail & Related papers  (2024-10-20T18:44:45Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• A machine learning approach to predict university enrolment choices through students' high school background in Italy [42.57210316104905]
  This paper explores the influence of Italian high school students' proficiency in mathematics and the Italian language on their university enrolment choices. We investigate potential gender differences in response to similar previous educational choices and achievements. The findings shed light on the complex interplay of academic proficiency, gender, and high school background in shaping students' choices regarding university education.
  arXiv  Detail & Related papers  (2024-02-29T10:05:37Z)
• Intelligent System for Assessing University Student Personality Development and Career Readiness [0.0]
  This research paper explores the impact of various factors on university students' readiness for change and transition. The collected data from a KBTU student survey was processed through machine learning models. An intelligent system was built using these models and fuzzy sets.
  arXiv  Detail & Related papers  (2023-08-29T20:32:58Z)
• A Survey of Knowledge Tracing: Models, Variants, and Applications [70.69281873057619]
  Knowledge Tracing is one of the fundamental tasks for student behavioral data analysis. We present three types of fundamental KT models with distinct technical routes. We discuss potential directions for future research in this rapidly growing field.
  arXiv  Detail & Related papers  (2021-05-06T13:05:55Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.