Experiences from Integrating Large Language Model Chatbots into the Classroom

• URL: http://arxiv.org/abs/2406.04817v1
• Date: Fri, 7 Jun 2024 10:37:14 GMT
• Title: Experiences from Integrating Large Language Model Chatbots into the Classroom
• Authors: Arto Hellas, Juho Leinonen, Leo Leppänen,
• Abstract summary: We provide students an unfiltered access to a state-of-the-art large language model (LLM) chatbots. In all courses, the majority of the LLM usage came from a few superusers. We discuss potential reasons for the low usage, suggesting the need for more tailored and scaffolded LLM experiences.
• Score: 4.449125623758632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the present study, we provided students an unfiltered access to a state-of-the-art large language model (LLM) chatbot. The chatbot was intentionally designed to mimic proprietary commercial chatbots such as ChatGPT where the chatbot has not been tailored for the educational context; the underlying engine was OpenAI GPT-4. The chatbot was integrated into online learning materials of three courses. One of the courses focused on software engineering with LLMs, while the two other courses were not directly related to LLMs. Our results suggest that only a minority of students engage with the chatbot in the courses that do not relate to LLMs. At the same time, unsurprisingly, nearly all students in the LLM-focused course leveraged the chatbot. In all courses, the majority of the LLM usage came from a few superusers, whereas the majority of the students did not heavily use the chatbot even though it was readily available and effectively provided a free access to the OpenAI GPT-4 model. We also observe that in addition to students using the chatbot for course-specific purposes, many use the chatbot for their own purposes. These results suggest that the worst fears of educators -- all students overrelying on LLMs -- did not materialize even when the chatbot access was unfiltered. We finally discuss potential reasons for the low usage, suggesting the need for more tailored and scaffolded LLM experiences targeted for specific types of student use cases.

Related papers

• LLM Roleplay: Simulating Human-Chatbot Interaction [52.03241266241294]
  We propose a goal-oriented, persona-based method to automatically generate diverse multi-turn dialogues simulating human-chatbot interaction. Our method can simulate human-chatbot dialogues with a high indistinguishability rate.
  arXiv  Detail & Related papers  (2024-07-04T14:49:46Z)
• A Complete Survey on LLM-based AI Chatbots [46.18523139094807]
  The past few decades have witnessed an upsurge in data, forming the foundation for data-hungry, learning-based AI technology. Conversational agents, often referred to as AI chatbots, rely heavily on such data to train large language models (LLMs) and generate new content (knowledge) in response to user prompts. This paper presents a complete survey of the evolution and deployment of LLM-based chatbots in various sectors.
  arXiv  Detail & Related papers  (2024-06-17T09:39:34Z)
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference [48.99117537559644]
  We introduce Arena, an open platform for evaluating Large Language Models (LLMs) based on human preferences. Our methodology employs a pairwise comparison approach and leverages input from a diverse user base through crowdsourcing. This paper describes the platform, analyzes the data we have collected so far, and explains the tried-and-true statistical methods we are using.
  arXiv  Detail & Related papers  (2024-03-07T01:22:38Z)
• Supporting Student Decisions on Learning Recommendations: An LLM-Based Chatbot with Knowledge Graph Contextualization for Conversational Explainability and Mentoring [0.0]
  We propose an approach to utilize chatbots as mediators of the conversation and sources of limited and controlled generation of explanations. A group chat approach is developed to connect students with human mentors, either on demand or in cases that exceed the chatbots's pre-defined tasks.
  arXiv  Detail & Related papers  (2024-01-16T17:31:35Z)
• Bias and Fairness in Chatbots: An Overview [38.21995125571103]
  Modern chatbots are more powerful and have been used in real-world applications. Due to the huge amounts of training data, extremely large model sizes, and lack of interpretability, bias mitigation and fairness preservation are challenging.
  arXiv  Detail & Related papers  (2023-09-16T02:01:18Z)
• EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education [44.2456523088426]
  EduChat is a large-scale language model (LLM)-based chat system in the education domain. Currently, EduChat is available online as an open-source project, with its code, data, and model parameters available on platforms.
  arXiv  Detail & Related papers  (2023-08-05T02:55:35Z)
• Implementing a Chatbot Solution for Learning Management System [0.0]
  One of the main problem that chatbots face today is to mimic human language. Extreme programming methodology was chosen to use integrate ChatterBot, Pyside2, web scraping and Tampermonkey into Blackboard. We showed the plausibility of integrating an AI bot in an educational setting.
  arXiv  Detail & Related papers  (2022-06-27T11:04:42Z)
• Few-Shot Bot: Prompt-Based Learning for Dialogue Systems [58.27337673451943]
  Learning to converse using only a few examples is a great challenge in conversational AI. The current best conversational models are either good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL) We propose prompt-based few-shot learning which does not require gradient-based fine-tuning but instead uses a few examples as the only source of learning.
  arXiv  Detail & Related papers  (2021-10-15T14:36:45Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)
• Conversational agents for learning foreign languages -- a survey [0.0]
  Conversational practice, while crucial for all language learners, can be challenging to get enough of and very expensive. This paper provides an overview of the chatbots for learning languages, critically analyze existing approaches, and discuss the major challenges for future work.
  arXiv  Detail & Related papers  (2020-11-16T12:27:02Z)
• Developing FB Chatbot Based on Deep Learning Using RASA Framework for University Enquiries [0.0]
  This research is a first stage development within fairly sufficient simulate data. The concept is not something new in today society which is developing with recent technology. This uses the FB platform because of the FB users have already reached up to 60.8% of its entire population in Indonesia.
  arXiv  Detail & Related papers  (2020-09-25T17:01:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.