Large Language Models in Teaching and Learning: Reflections on Implementing an AI Chatbot in Higher Education

• URL: http://arxiv.org/abs/2603.17773v1
• Date: Wed, 18 Mar 2026 14:34:12 GMT
• Title: Large Language Models in Teaching and Learning: Reflections on Implementing an AI Chatbot in Higher Education
• Abstract summary: Large language models (LLMs) present substantial opportunities especially in Higher Education.<n>Their propensity to generate hallucinations and their limited specialized knowledge may introduce significant risks.<n>This study examines the practical implementation of an LLM-enhanced assistant in a university level course.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The landscape of education is changing rapidly, shaped by emerging pedagogical approaches, technological innovations such as artificial intelligence (AI), and evolving societal expectations, all of which demand thorough evaluation of new educational tools. Although large language models (LLMs) present substantial opportunities especially in Higher Education, their propensity to generate hallucinations and their limited specialized knowledge may introduce significant risks. This study aims to address these risks by examining the practical implementation of an LLM-enhanced assistant in a university level course. We implemented a generative AI assistant grounded in a retrieval-augmented generation (RAG) model to replicate a previously teacher-led, time-intensive exercise. To assess the effectiveness of the LLM, we conducted three separate experiments through iterative mixed-methods approaches, including a crossover design. The resulting data address central research questions related to student motivation, perceived differences between engaging with the LLM versus a human teacher, the quality of AI-generated responses, and the impact of the LLM on students' academic performance. The results offer direct insights into students' views and the pedagogical feasibility of embedding LLMs into specialized courses. Finally, we discuss the main challenges, opportunities and future directions of LLMs in teaching and learning in Higher Education.

Related papers

• Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI [0.8703455323398351]
  Review of studies using large language models (LLMs) to simulate student behaviour across educational environments.<n>Wee current evidence on the capacity of LLM-based agents to emulate learner archetypes, respond to instructional inputs, and interact within multi-agent classroom scenarios.<n>We examine the implications of such systems for curriculum development, instructional evaluation, and teacher training.
  arXiv  Detail & Related papers  (2025-11-08T17:23:13Z)
• From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education [14.320271477960192]
  Large Language Models (LLMs) are transforming education by enabling personalization, feedback, and knowledge access.<n>Yet empirical evidence on these risks remains fragmented.<n>This paper presents a systematic review of 70 empirical studies across computer science, education, and psychology.
  arXiv  Detail & Related papers  (2025-09-26T06:59:36Z)
• Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities [62.05713042908654]
  This paper provides a review of advances in Large Language Models (LLMs) alignment through the lens of inverse reinforcement learning (IRL)<n>We highlight the necessity of constructing neural reward models from human data and discuss the formal and practical implications of this paradigm shift.
  arXiv  Detail & Related papers  (2025-07-17T14:22:24Z)
• LLM Agents for Education: Advances and Applications [49.3663528354802]
  Large Language Model (LLM) agents have demonstrated remarkable capabilities in automating tasks and driving innovation across diverse educational applications.<n>This survey aims to provide a comprehensive technological overview of LLM agents for education, fostering further research and collaboration to enhance their impact for the greater good of learners and educators alike.
  arXiv  Detail & Related papers  (2025-03-14T11:53:44Z)
• Position: LLMs Can be Good Tutors in English Education [120.25980336297444]
  Large language models (LLMs) have the potential to serve as effective tutors in English Education.<n>LLMs can play three critical roles: as data enhancers, improving the creation of learning materials or serving as student simulations.
  arXiv  Detail & Related papers  (2025-02-08T06:48:49Z)
• When LLMs Learn to be Students: The SOEI Framework for Modeling and Evaluating Virtual Student Agents in Educational Interaction [12.070907646464537]
  We propose the SOEI framework for constructing and evaluating personality-aligned Virtual Student Agents (LVSAs) in classroom scenarios.<n>We generate five LVSAs based on Big Five traits through LoRA fine-tuning and expert-informed prompt design.<n>Our results provide: (1) an educationally and psychologically grounded generation pipeline for LLM-based student agents; (2) a hybrid, scalable evaluation framework for behavioral realism; and (3) empirical insights into the pedagogical utility of LVSAs in shaping instructional adaptation.
  arXiv  Detail & Related papers  (2024-10-21T07:18:24Z)
• Large Language Models for Education: A Survey and Outlook [69.02214694865229]
  We systematically review the technological advancements in each perspective, organize related datasets and benchmarks, and identify the risks and challenges associated with deploying LLMs in education. Our survey aims to provide a comprehensive technological picture for educators, researchers, and policymakers to harness the power of LLMs to revolutionize educational practices and foster a more effective personalized learning environment.
  arXiv  Detail & Related papers  (2024-03-26T21:04:29Z)
• Evaluating and Optimizing Educational Content with Large Language Model Judgments [52.33701672559594]
  We use Language Models (LMs) as educational experts to assess the impact of various instructions on learning outcomes. We introduce an instruction optimization approach in which one LM generates instructional materials using the judgments of another LM as a reward function. Human teachers' evaluations of these LM-generated worksheets show a significant alignment between the LM judgments and human teacher preferences.
  arXiv  Detail & Related papers  (2024-03-05T09:09:15Z)
• Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education [13.87944568193996]
  Multimodal Large Language Models (MLLMs) are capable of processing multimodal data including text, sound, and visual inputs. This paper explores the transformative role of MLLMs in central aspects of science education by presenting exemplary innovative learning scenarios.
  arXiv  Detail & Related papers  (2024-01-01T18:11:43Z)
• Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges [60.62904929065257]
  Large language models (LLMs) offer possibility for resolving this issue by comprehending individual requests. This paper reviews the recently emerged LLM research related to educational capabilities, including mathematics, writing, programming, reasoning, and knowledge-based question answering.
  arXiv  Detail & Related papers  (2023-12-27T14:37:32Z)
• Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception [19.335003380399527]
  Large language models (LLMs) offer a promising avenue, with increasing research exploring their educational utility. Our work highlights the role that teachers can play in shaping LLM-supported learning environments.
  arXiv  Detail & Related papers  (2023-10-13T01:21:52Z)
• Prototyping the use of Large Language Models (LLMs) for adult learning content creation at scale [0.6628807224384127]
  This paper presents an investigation into the use of Large Language Models (LLMs) in asynchronous course creation. We developed a course prototype leveraging an LLM, implementing a robust human-in-the-loop process. Initial findings indicate that taking this approach can indeed facilitate faster content creation without compromising on accuracy or clarity.
  arXiv  Detail & Related papers  (2023-06-02T10:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.