Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems

• URL: http://arxiv.org/abs/2310.01420v2
• Date: Tue, 14 Nov 2023 15:41:45 GMT
• Title: Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems
• Authors: Robin Schmucker, Meng Xia, Amos Azaria, Tom Mitchell
• Abstract summary: Conversational tutoring systems (CTSs) offer learning experiences driven by natural language interaction. We introduce a novel type of CTS that leverages the recent advances in large language models (LLMs) in two ways. Ruffle&Riley allows a free-form conversation that follows the ITS-typical inner and outer loop structure.
• Score: 23.093767743306973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational tutoring systems (CTSs) offer learning experiences driven by natural language interaction. They are known to promote high levels of cognitive engagement and benefit learning outcomes, particularly in reasoning tasks. Nonetheless, the time and cost required to author CTS content is a major obstacle to widespread adoption. In this paper, we introduce a novel type of CTS that leverages the recent advances in large language models (LLMs) in two ways: First, the system induces a tutoring script automatically from a lesson text. Second, the system automates the script orchestration via two LLM-based agents (Ruffle&Riley) with the roles of a student and a professor in a learning-by-teaching format. The system allows a free-form conversation that follows the ITS-typical inner and outer loop structure. In an initial between-subject online user study (N = 100) comparing Ruffle&Riley to simpler QA chatbots and reading activity, we found no significant differences in post-test scores. Nonetheless, in the learning experience survey, Ruffle&Riley users expressed higher ratings of understanding and remembering and further perceived the offered support as more helpful and the conversation as coherent. Our study provides insights for a new generation of scalable CTS technologies.

Related papers

• Exploring Knowledge Tracing in Tutor-Student Dialogues [53.52699766206808]
  We present a first attempt at performing knowledge tracing (KT) in tutor-student dialogues. We propose methods to identify the knowledge components/skills involved in each dialogue turn. We then apply a range of KT methods on the resulting labeled data to track student knowledge levels over an entire dialogue.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Pronunciation Assessment with Multi-modal Large Language Models [10.35401596425946]
  We propose a scoring system based on large language models (LLMs) The speech encoder first maps the learner's speech into contextual features. The adapter layer then transforms these features to align with the text embedding in latent space.
  arXiv  Detail & Related papers  (2024-07-12T12:16:14Z)
• Ruffle&Riley: Insights from Designing and Evaluating a Large Language Model-Based Conversational Tutoring System [21.139850269835858]
  Conversational tutoring systems (CTSs) offer learning experiences through interactions based on natural language. We discuss and evaluate a novel type of CTS that leverages recent advances in large language models (LLMs) in two ways. The system enables AI-assisted content authoring by inducing an easily editable tutoring script automatically from a lesson text.
  arXiv  Detail & Related papers  (2024-04-26T14:57:55Z)
• ChatPRCS: A Personalized Support System for English Reading Comprehension based on ChatGPT [3.847982502219679]
  This paper presents a novel personalized support system for reading comprehension, referred to as ChatPRCS. ChatPRCS employs methods including reading comprehension proficiency prediction, question generation, and automatic evaluation.
  arXiv  Detail & Related papers  (2023-09-22T11:46:44Z)
• Prototype of a robotic system to assist the learning process of English language with text-generation through DNN [0.0]
  We present a working prototype of a humanoid robotic system to assist English language self-learners. The learners interact with the system using a Graphic User Interface that generates text according to the English level of the user.
  arXiv  Detail & Related papers  (2023-09-20T08:39:51Z)
• AI Chatbots as Multi-Role Pedagogical Agents: Transforming Engagement in CS Education [8.898863361318817]
  We develop, implement, and evaluate a novel learning environment enriched with four distinct chatbots. These roles cater to the three innate psychological needs of learners - competence, autonomy, and relatedness. The system embraces an inquiry-based learning paradigm, encouraging students to ask questions, seek solutions, and explore their curiosities.
  arXiv  Detail & Related papers  (2023-08-08T02:13:44Z)
• Few-Shot Bot: Prompt-Based Learning for Dialogue Systems [58.27337673451943]
  Learning to converse using only a few examples is a great challenge in conversational AI. The current best conversational models are either good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL) We propose prompt-based few-shot learning which does not require gradient-based fine-tuning but instead uses a few examples as the only source of learning.
  arXiv  Detail & Related papers  (2021-10-15T14:36:45Z)
• A study on the efficacy of model pre-training in developing neural text-to-speech system [55.947807261757056]
  This study aims to understand better why and how model pre-training can positively contribute to TTS system performance. It is found that the TTS system could achieve comparable performance when the pre-training data is reduced to 1/8 of its original size.
  arXiv  Detail & Related papers  (2021-10-08T02:09:28Z)
• An Adversarial Learning based Multi-Step Spoken Language Understanding System through Human-Computer Interaction [70.25183730482915]
  We introduce a novel multi-step spoken language understanding system based on adversarial learning. We demonstrate that the new system can improve parsing performance by at least $2.5%$ in terms of F1.
  arXiv  Detail & Related papers  (2021-06-06T03:46:53Z)
• Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
  We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
  arXiv  Detail & Related papers  (2020-10-11T08:27:07Z)
• MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems [75.43457658815943]
  We propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ.
  arXiv  Detail & Related papers  (2020-09-25T02:19:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.