Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features

• URL: http://arxiv.org/abs/2404.04394v2
• Date: Tue, 14 May 2024 13:20:53 GMT
• Title: Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features
• Authors: Alexander Vedernikov, Zhaodong Sun, Virpi-Liisa Kykyri, Mikko Pohjola, Miriam Nokia, Xiaobai Li,
• Abstract summary: Engagement measurement finds application in healthcare, education, services. Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
• Score: 50.82725748981231
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Engagement measurement finds application in healthcare, education, services. The use of physiological and behavioral features is viable, but the impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of unsupervised remote photoplethysmography (rPPG) as an alternative for contact sensors in deriving heart rate variability (HRV) features, then fusing these with behavioral features to measure engagement in online group meetings. Firstly, a unique Engagement Dataset of online interactions among social workers is collected with granular engagement labels, offering insight into virtual meeting dynamics. Secondly, a pre-trained rPPG model is customized to reconstruct rPPG signals from video meetings in an unsupervised manner, enabling the calculation of HRV features. Thirdly, the feasibility of estimating engagement from HRV features using short observation windows, with a notable enhancement when using longer observation windows of two to four minutes, is demonstrated. Fourthly, the effectiveness of behavioral cues is evaluated when fused with physiological data, which further enhances engagement estimation performance. An accuracy of 94% is achieved when only HRV features are used, eliminating the need for contact sensors or ground truth signals; use of behavioral cues raises the accuracy to 96%. Facial analysis offers precise engagement measurement, beneficial for future applications.

Related papers

• SympCam: Remote Optical Measurement of Sympathetic Arousal [41.97525156940624]
  We present a new convolutional architecture tailored to the task of remote sympathetic arousal prediction. We show that the sympathetic arousal predicted by our method allows detecting physical stress with a balanced accuracy of 90%. We contribute a dataset designed explicitly for the task of remote sympathetic arousal prediction.
  arXiv  Detail & Related papers  (2024-10-27T18:46:55Z)
• Integrative Deep Learning Framework for Parkinson's Disease Early Detection using Gait Cycle Data Measured by Wearable Sensors: A CNN-GRU-GNN Approach [0.3222802562733786]
  We present a pioneering deep learning architecture tailored for the binary classification of subjects. Our model harnesses the power of 1D-Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), and Graph Neural Network (GNN) layers. Our proposed model achieves exceptional performance metrics, boasting accuracy, precision, recall, and F1 score values of 99.51%, 99.57%, 99.71%, and 99.64%, respectively.
  arXiv  Detail & Related papers  (2024-04-09T15:19:13Z)
• ShaRPy: Shape Reconstruction and Hand Pose Estimation from RGB-D with Uncertainty [6.559796851992517]
  We propose ShaRPy, the first RGB-D Shape Reconstruction and hand Pose tracking system. ShaRPy approximates a personalized hand shape, promoting a more realistic and intuitive understanding of its digital twin. We evaluate ShaRPy on a keypoint detection benchmark and show qualitative results of hand function assessments for activity monitoring of musculoskeletal diseases.
  arXiv  Detail & Related papers  (2023-03-17T15:12:25Z)
• KIDS: kinematics-based (in)activity detection and segmentation in a sleep case study [5.707737640557724]
  Sleep behaviour and in-bed movements contain rich information on the neurophysiological health of people. This paper proposes an online Bayesian probabilistic framework for objective (in)activity detection and segmentation based on clinically meaningful joint kinematics.
  arXiv  Detail & Related papers  (2023-01-04T16:24:01Z)
• Measuring hand use in the home after cervical spinal cord injury using egocentric video [2.1064122195521926]
  Egocentric video has emerged as a potential solution for monitoring hand function in individuals living with tetraplegia in the community. We develop and validate a wearable vision-based system for measuring hand use in the home among individuals living with tetraplegia.
  arXiv  Detail & Related papers  (2022-03-31T12:43:23Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• Designing A Clinically Applicable Deep Recurrent Model to Identify Neuropsychiatric Symptoms in People Living with Dementia Using In-Home Monitoring Data [52.40058724040671]
  Agitation is one of the neuropsychiatric symptoms with high prevalence in dementia. Detecting agitation episodes can assist in providing People Living with Dementia (PLWD) with early and timely interventions. This preliminary study presents a supervised learning model to analyse the risk of agitation in PLWD using in-home monitoring data.
  arXiv  Detail & Related papers  (2021-10-19T11:45:01Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.