Multimodality in Online Education: A Comparative Study

• URL: http://arxiv.org/abs/2312.05797v2
• Date: Sun, 17 Dec 2023 06:57:36 GMT
• Title: Multimodality in Online Education: A Comparative Study
• Authors: Praneeta Immadisetty, Pooja Rajesh, Akshita Gupta, Anala M R, Soumya A, K. N. Subramanya
• Abstract summary: Current systems consider only a single cue with a lack of focus in the educational domain. This paper highlights the need for a multimodal approach to affect recognition and its deployment in the online classroom. It compares the various machine learning models available for each cue and provides the most suitable approach.
• Score: 2.0472158451829827
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The commencement of the decade brought along with it a grave pandemic and in response the movement of education forums predominantly into the online world. With a surge in the usage of online video conferencing platforms and tools to better gauge student understanding, there needs to be a mechanism to assess whether instructors can grasp the extent to which students understand the subject and their response to the educational stimuli. The current systems consider only a single cue with a lack of focus in the educational domain. Thus, there is a necessity for the measurement of an all-encompassing holistic overview of the students' reaction to the subject matter. This paper highlights the need for a multimodal approach to affect recognition and its deployment in the online classroom while considering four cues, posture and gesture, facial, eye tracking and verbal recognition. It compares the various machine learning models available for each cue and provides the most suitable approach given the available dataset and parameters of classroom footage. A multimodal approach derived from weighted majority voting is proposed by combining the most fitting models from this analysis of individual cues based on accuracy, ease of procuring data corpus, sensitivity and any major drawbacks.

Related papers

• A multi-purpose automatic editing system based on lecture semantics for remote education [6.6826236187037305]
  This paper proposes an automatic multi-camera directing/editing system based on the lecture semantics. Our system directs the views by semantically analyzing the class events while following the professional directing rules.
  arXiv  Detail & Related papers  (2024-11-07T16:49:25Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• Empowering Private Tutoring by Chaining Large Language Models [87.76985829144834]
  This work explores the development of a full-fledged intelligent tutoring system powered by state-of-the-art large language models (LLMs) The system is into three inter-connected core processes-interaction, reflection, and reaction. Each process is implemented by chaining LLM-powered tools along with dynamically updated memory modules.
  arXiv  Detail & Related papers  (2023-09-15T02:42:03Z)
• Vision+X: A Survey on Multimodal Learning in the Light of Data [64.03266872103835]
  multimodal machine learning that incorporates data from various sources has become an increasingly popular research area. We analyze the commonness and uniqueness of each data format mainly ranging from vision, audio, text, and motions. We investigate the existing literature on multimodal learning from both the representation learning and downstream application levels.
  arXiv  Detail & Related papers  (2022-10-05T13:14:57Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• Real-time Attention Span Tracking in Online Education [0.0]
  This paper intends to provide a mechanism that uses the camera feed and microphone input to monitor the real-time attention level of students during online classes. We propose a system that uses five distinct non-verbal features to calculate the attention score of the student during computer based tasks and generate real-time feedback for both students and the organization.
  arXiv  Detail & Related papers  (2021-11-29T17:05:59Z)
• Improving Teacher-Student Interactions in Online Educational Forums using a Markov Chain based Stackelberg Game Model [5.004814662623874]
  We propose an analytical model based on continuous time Markov chains (CTMCs) to capture instructor-student interactions in an online forum (OEF) We observe that students exhibit varied degree of non-monotonicity in their participation with increasing instructor involvement. Our model exhibits the empirically observed super-poster phenomenon under certain parameter configurations and recommends an optimal plan to the instructor for maximizing student participation in OEFs.
  arXiv  Detail & Related papers  (2021-11-25T09:48:20Z)
• Multimodal Engagement Analysis from Facial Videos in the Classroom [5.202558003704116]
  The aim of this work is to provide the technical means to facilitate the manual data analysis of classroom videos in research on teaching quality and in the context of teacher training.
  arXiv  Detail & Related papers  (2021-01-11T22:15:04Z)
• Personalized Multimodal Feedback Generation in Education [50.95346877192268]
  The automatic evaluation for school assignments is an important application of AI in the education field. We propose a novel Personalized Multimodal Feedback Generation Network (PMFGN) armed with a modality gate mechanism and a personalized bias mechanism. Our model significantly outperforms several baselines by generating more accurate and diverse feedback.
  arXiv  Detail & Related papers  (2020-10-31T05:26:49Z)
• Embedded Deep Bilinear Interactive Information and Selective Fusion for Multi-view Learning [70.67092105994598]
  We propose a novel multi-view learning framework to make the multi-view classification better aimed at the above-mentioned two aspects. In particular, we train different deep neural networks to learn various intra-view representations. Experiments on six publicly available datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2020-07-13T01:13:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.