Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination

• URL: http://arxiv.org/abs/2409.07372v1
• Date: Wed, 11 Sep 2024 16:03:09 GMT
• Title: Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination
• Authors: Daniel Zhang-Li, Zheyuan Zhang, Jifan Yu, Joy Lim Jia Yin, Shangqing Tu, Linlu Gong, Haohua Wang, Zhiyuan Liu, Huiqin Liu, Lei Hou, Juanzi Li,
• Abstract summary: We develop Slide2Lecture, a tuning-free and knowledge-regulated intelligent tutoring system. It can effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions. For teachers and developers, Slide2Lecture enables customization to cater to personalized demands.
• Score: 52.20542825755132
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The vast pre-existing slides serve as rich and important materials to carry lecture knowledge. However, effectively leveraging lecture slides to serve students is difficult due to the multi-modal nature of slide content and the heterogeneous teaching actions. We study the problem of discovering effective designs that convert a slide into an interactive lecture. We develop Slide2Lecture, a tuning-free and knowledge-regulated intelligent tutoring system that can (1) effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions; (2) create and manage an interactive lecture that generates responsive interactions catering to student learning demands while regulating the interactions to follow teaching actions. Slide2Lecture contains a complete pipeline for learners to obtain an interactive classroom experience to learn the slide. For teachers and developers, Slide2Lecture enables customization to cater to personalized demands. The evaluation rated by annotators and students shows that Slide2Lecture is effective in outperforming the remaining implementation. Slide2Lecture's online deployment has made more than 200K interaction with students in the 3K lecture sessions. We open source Slide2Lecture's implementation in https://anonymous.4open.science/r/slide2lecture-4210/.

Related papers

• Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration [64.6107798750142]
  Vocal Sandbox is a framework for enabling seamless human-robot collaboration in situated environments. We design lightweight and interpretable learning algorithms that allow users to build an understanding and co-adapt to a robot's capabilities in real-time. We evaluate Vocal Sandbox in two settings: collaborative gift bag assembly and LEGO stop-motion animation.
  arXiv  Detail & Related papers  (2024-11-04T20:44:40Z)
• Multimodal Classification of Teaching Activities from University Lecture Recordings [0.9790236766474201]
  We present a multimodal classification algorithm that identifies the type of activity that is being carried out at any time of the lesson. Some academic activities are more easily identifiable with the audio signal while resorting to the text transcription is needed to identify others.
  arXiv  Detail & Related papers  (2023-12-24T08:33:30Z)
• Large Language Model-Driven Classroom Flipping: Empowering Student-Centric Peer Questioning with Flipped Interaction [3.1473798197405953]
  This paper investigates a pedagogical approach of classroom flipping based on flipped interaction in large language models. Flipped interaction involves using language models to prioritize generating questions instead of answers to prompts. We propose a workflow to integrate prompt engineering with clicker and JiTT quizzes by a poll-prompt-quiz routine and a quiz-prompt-discuss routine.
  arXiv  Detail & Related papers  (2023-11-14T15:48:19Z)
• Cross Architecture Distillation for Face Recognition [49.55061794917994]
  We develop an Adaptable Prompting Teacher network (APT) that integrates prompts into the teacher, enabling it to manage distillation-specific knowledge. Experiments on popular face benchmarks and two large-scale verification sets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2023-06-26T12:54:28Z)
• TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition [68.53072549422775]
  We propose a student-teacher semi-supervised learning framework, TimeBalance. We distill the knowledge from a temporally-invariant and a temporally-distinctive teacher. Our method achieves state-of-the-art performance on three action recognition benchmarks.
  arXiv  Detail & Related papers  (2023-03-28T19:28:54Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• SliTraNet: Automatic Detection of Slide Transitions in Lecture Videos using Convolutional Neural Networks [10.097888451225234]
  We propose a deep learning method to detect slide transitions in lectures videos. We first predict each frame of the video by a process-based approach using a 2-D convolutional neural network. We increase the complexity by employing two 3-D convolutional neural networks to refine the transition candidates.
  arXiv  Detail & Related papers  (2022-02-07T22:03:27Z)
• Interaction-limited Inverse Reinforcement Learning [50.201765937436654]
  We present two different training strategies: Curriculum Inverse Reinforcement Learning (CIRL) covering the teacher's perspective, and Self-Paced Inverse Reinforcement Learning (SPIRL) focusing on the learner's perspective. Using experiments in simulations and experiments with a real robot learning a task from a human demonstrator, we show that our training strategies can allow a faster training than a random teacher for CIRL and than a batch learner for SPIRL.
  arXiv  Detail & Related papers  (2020-07-01T12:31:52Z)
• Interactive Summarizing -- Automatic Slide Localization Technology as Generative Learning Tool [10.81386784858998]
  Video summarization is an effective technology applied to enhance learners' summarizing experience in a video lecture. An interactive summarizing model is designed to explain how learners are engaged in the video lecture learning process supported by convolutional neural network.
  arXiv  Detail & Related papers  (2020-02-25T22:22:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.