edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms

• URL: http://arxiv.org/abs/2211.09210v1
• Date: Wed, 16 Nov 2022 20:53:56 GMT
• Title: edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms
• Authors: Roberto Daza, Aythami Morales, Ruben Tolosana, Luis F. Gomez, Julian Fierrez, Javier Ortega-Garcia
• Abstract summary: The edBB platform aims to study the challenges associated to user recognition and behavior understanding in digital platforms. The information captured from the sensors during the student sessions is modelled in a multimodal learning framework.
• Score: 17.38605546335716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present edBB-Demo, a demonstrator of an AI-powered research platform for student monitoring in remote education. The edBB platform aims to study the challenges associated to user recognition and behavior understanding in digital platforms. This platform has been developed for data collection, acquiring signals from a variety of sensors including keyboard, mouse, webcam, microphone, smartwatch, and an Electroencephalography band. The information captured from the sensors during the student sessions is modelled in a multimodal learning framework. The demonstrator includes: i) Biometric user authentication in an unsupervised environment; ii) Human action recognition based on remote video analysis; iii) Heart rate estimation from webcam video; and iv) Attention level estimation from facial expression analysis.

Related papers

• Robust Audiovisual Speech Recognition Models with Mixture-of-Experts [67.75334989582709]
  We introduce EVA, leveraging the mixture-of-Experts for audioVisual ASR to perform robust speech recognition for in-the-wild'' videos. We first encode visual information into visual tokens sequence and map them into speech space by a lightweight projection. Experiments show our model achieves state-of-the-art results on three benchmarks.
  arXiv  Detail & Related papers  (2024-09-19T00:08:28Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• Any-point Trajectory Modeling for Policy Learning [64.23861308947852]
  We introduce Any-point Trajectory Modeling (ATM) to predict future trajectories of arbitrary points within a video frame. ATM outperforms strong video pre-training baselines by 80% on average. We show effective transfer learning of manipulation skills from human videos and videos from a different robot morphology.
  arXiv  Detail & Related papers  (2023-12-28T23:34:43Z)
• Data Center Audio/Video Intelligence on Device (DAVID) -- An Edge-AI Platform for Smart-Toys [2.740631793745274]
  The DAVID Smart-Toy platform is one of the first Edge AI platform designs. It incorporates advanced low-power data processing by neural inference models co-located with the relevant image or audio sensors. There is also on-board capability for in-device text-to-speech generation.
  arXiv  Detail & Related papers  (2023-11-18T10:38:35Z)
• M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education [15.30924350440346]
  M2LADS supports the integration and visualization of multimodal data recorded in MOOCs in the form of Web-based Dashboards. Based on the edBB platform, the multimodal data gathered contains biometric and behavioral signals. M2LADS provides opportunities to capture learners' holistic experience during their interactions with the MOOC.
  arXiv  Detail & Related papers  (2023-05-21T20:22:38Z)
• MATT: Multimodal Attention Level Estimation for e-learning Platforms [16.407885871027887]
  This work presents a new multimodal system for remote attention level estimation based on multimodal face analysis. Our multimodal approach uses different parameters and signals obtained from the behavior and physiological processes that have been related to modeling cognitive load. The mEBAL database is used in the experimental framework, a public multi-modal database for attention level estimation obtained in an e-learning environment.
  arXiv  Detail & Related papers  (2023-01-22T18:18:20Z)
• MDEAW: A Multimodal Dataset for Emotion Analysis through EDA and PPG signals from wireless wearable low-cost off-the-shelf Devices [0.0]
  We present MDEAW, a multimodal database consisting of Electrodermal Activity (EDA) and Photoplethys ( PPG) signals recorded during the exams for the course taught by the teacher at Eurecat Academy, Sabadell, Barcelona. Signals were captured using portable, wearable, wireless, low-cost, and off-the-shelf equipment that has the potential to allow the use of affective computing methods in everyday applications.
  arXiv  Detail & Related papers  (2022-07-14T07:04:29Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)
• Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings [56.93025161787725]
  Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing local data. We propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters. We show that the attribute inference attack is achievable for SER systems trained using FL.
  arXiv  Detail & Related papers  (2021-12-26T16:50:42Z)
• Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB [24.666654058140836]
  This study focuses on the RGB and near-infrared video sequences for performing heart rate estimation applying remote photoplethysmography techniques. Experiments include behavioral and physiological data from 25 different students completing a collection of tasks related to e-learning. Our proposed face heart rate estimation approach is compared with the heart rate provided by the smartwatch, achieving very promising results for its future deployment in e-learning applications.
  arXiv  Detail & Related papers  (2020-06-01T10:04:36Z)
• Multi-Modal Video Forensic Platform for Investigating Post-Terrorist Attack Scenarios [55.82693757287532]
  Large scale Video Analytic Platforms (VAP) assist law enforcement agencies (LEA) in identifying suspects and securing evidence. We present a video analytic platform that integrates visual and audio analytic modules and fuses information from surveillance cameras and video uploads from eyewitnesses.
  arXiv  Detail & Related papers  (2020-04-02T14:29:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.