Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion

• URL: http://arxiv.org/abs/2107.04566v1
• Date: Fri, 9 Jul 2021 17:34:42 GMT
• Title: Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion
• Abstract summary: ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. Existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. We propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window.
• Score: 4.664989082015336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a single experience (e.g. watching a VR video) to a single stress level, which again prevents design of dynamic experiences where real-time in-game stress assessment can be utilized. In this paper, we report our findings on a new study on VR stress assessment, where three stress levels are assessed. ECG data was collected from 9 users experiencing a VR roller coaster. The VR experience was then manually labeled in 10-seconds segments to three stress levels by three raters. We then propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window. Experimental results demonstrate that the proposed model outperforms the classical HRV-based ML models (9% increase in accuracy) and baseline deep learning models (2.5% increase in accuracy). We also report results on the benchmark WESAD dataset to show the supremacy of the model.

Related papers

• Toward Postural State Classification in Immersive VR with Multimodal Data and Explainability Analysis [9.386262755183862]
  This study compares machine learning (ML) and deep learning (DL) models for classifying postural states in VR under visual perturbations.<n>We used a multimodal dataset containing kinematic, electromyographic (EMG), and electrodermal activity (EDA) signals.<n>Among the models, the Mamba-inspired CNN (MI-CNN) achieved the highest accuracy of 96.76%.
  arXiv  Detail & Related papers  (2026-08-28T20:31:00Z)
• On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR [51.935533482549545]
  We show that RLVR may exhibit implicit reward overfitting to the training dataset.<n>The model can achieve satisfactory performance on the test set even when its rewards remain relatively low during the training process.
  arXiv  Detail & Related papers  (2026-05-07T16:30:28Z)
• Introducing WARM-VR: Benchmark Dataset for Multimodal Wearable Affect Recognition in Virtual Reality [12.32040756213776]
  WARM-VR is a novel dataset designed to support affect recognition in immersive, multisensory environments.<n>Data was collected from 31 participants aged 19-37 using wearable sensors.
  arXiv  Detail & Related papers  (2026-04-30T20:00:56Z)
• Gesture Generation (Still) Needs Improved Human Evaluation Practices: Insights from a Community-Driven State-of-the-Art Benchmark [55.41250396114216]
  We review human evaluation practices in automated, speech-driven 3D gesture generation.<n>We introduce a detailed human evaluation protocol for the widely-used BEAT2 motion-capture dataset.
  arXiv  Detail & Related papers  (2025-11-03T05:17:28Z)
• ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models [52.819394001139955]
  Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR) are typical post-training paradigms for Large Vision-and-Language Models (LVLMs)<n>We propose ViSurf, a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage.<n>The core of ViSurf involves injecting ground-truth labels into the RLVR rollouts, thereby providing simultaneous external supervision and internal reinforcement.
  arXiv  Detail & Related papers  (2025-10-12T13:42:55Z)
• SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training [82.68200031146299]
  We propose a one-step diffusion-based VR model, termed as SeedVR2, which performs adversarial VR training against real data.<n>To handle the challenging high-resolution VR within a single step, we introduce several enhancements to both model architecture and training procedures.
  arXiv  Detail & Related papers  (2025-06-05T17:51:05Z)
• Extending Stress Detection Reproducibility to Consumer Wearable Sensors [0.9591571123059931]
  We compared validated research-grade devices, to consumer wearables, to assess device-specific stress detection performance.<n>Biopac MP160 performed the best, being consistent with our expectations of it as the gold standard.<n>Garmin Forerunner 55s demonstrated strong potential for real-world stress monitoring.
  arXiv  Detail & Related papers  (2025-05-09T00:06:06Z)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example [129.11571295790807]
  We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large language models (LLMs) We identify some interesting phenomena during 1-shot RLVR, including cross-domain generalization, increased frequency of self-reflection, and sustained test performance improvement even after the training accuracy has saturated.
  arXiv  Detail & Related papers  (2025-04-29T09:24:30Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms. We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction. Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study [61.64685376882383]
  Counterfactual learning to rank (CLTR) has attracted extensive attention in the IR community for its ability to leverage massive logged user interaction data to train ranking models. This paper investigates the robustness of existing CLTR models in complex and diverse situations. We find that the DLA models and IPS-DCM show better robustness under various simulation settings than IPS-PBM and PRS with offline propensity estimation.
  arXiv  Detail & Related papers  (2024-04-04T10:54:38Z)
• Stress Management Using Virtual Reality-Based Attention Training [3.686808512438363]
  We show that VR-based attention training consistently resulted in reducing the number of recognized stress instances in the recorded EEG signals. This research gives preliminary insights on adopting VR-based attention training for managing stress.
  arXiv  Detail & Related papers  (2023-12-10T22:42:00Z)
• Virtual-Reality based Vestibular Ocular Motor Screening for Concussion Detection using Machine-Learning [0.0]
  Sport-related concussion (SRC) depends on sensory information from visual, vestibular, and somatosensory systems. Current clinical administration of Vestibular/Ocular Motor Screening (VOMS) is subjective and deviates among administrators. With the advancement of technology, virtual reality (VR) can be utilized to advance the standardization of the VOMS.
  arXiv  Detail & Related papers  (2022-10-13T02:09:21Z)
• On the Generalizability of ECG-based Stress Detection Models [4.592950581802712]
  We explore the generalization capabilities of Electrocardiogram (ECG)-based deep learning models and models based on Heart Rate Variability (HRV) features. To the best of our knowledge, this is the first work to compare the cross-dataset generalizability between ECG-based deep learning models and HRV models.
  arXiv  Detail & Related papers  (2022-10-12T14:12:04Z)
• Dynamic Contrastive Distillation for Image-Text Retrieval [90.05345397400144]
  We present a novel plug-in dynamic contrastive distillation (DCD) framework to compress image-text retrieval models. We successfully apply our proposed DCD strategy to two state-of-the-art vision-language pretrained models, i.e. ViLT and METER. Experiments on MS-COCO and Flickr30K benchmarks show the effectiveness and efficiency of our DCD framework.
  arXiv  Detail & Related papers  (2022-07-04T14:08:59Z)
• Predicting Driver Self-Reported Stress by Analyzing the Road Scene [0.8155575318208631]
  We develop methods to test if the visual driving scene can be used to estimate a drivers' subjective stress levels. We use the AffectiveROAD video recordings and their corresponding stress labels. We design and evaluate three computer vision modeling approaches to classify the driver's stress levels.
  arXiv  Detail & Related papers  (2021-09-27T17:55:35Z)
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments [70.18430114842094]
  We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation. These activities are designed to be realistic, diverse, and complex. We include 500 human demonstrations in virtual reality (VR) to serve as the human ground truth.
  arXiv  Detail & Related papers  (2021-08-06T23:36:23Z)
• Personalized Stress Monitoring using Wearable Sensors in Everyday Settings [9.621481727547215]
  We explore objective prediction of stress levels in everyday settings based on heart rate (HR) and heart rate variability (HRV) We present a layered system architecture for personalized stress monitoring that supports a tunable collection of data samples for labeling, and present a method for selecting informative samples from the stream of real-time data for labeling.
  arXiv  Detail & Related papers  (2021-07-31T04:15:15Z)
• Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation [51.72616167073565]
  We propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance.
  arXiv  Detail & Related papers  (2020-06-28T21:03:19Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.