"Reading Between the Heat": Co-Teaching Body Thermal Signatures for
Non-intrusive Stress Detection
- URL: http://arxiv.org/abs/2310.09932v2
- Date: Tue, 28 Nov 2023 19:36:11 GMT
- Title: "Reading Between the Heat": Co-Teaching Body Thermal Signatures for
Non-intrusive Stress Detection
- Authors: Yi Xiao, Harshit Sharma, Zhongyang Zhang, Dessa Bergen-Cico, Tauhidur
Rahman, Asif Salekin
- Abstract summary: ThermaStrain is a novel co-teaching framework that achieves high-stress prediction performance by transferring knowledge from the wearable modality to the contactless thermal modality.
During training, ThermaStrain incorporates a wearable electrodermal activity (EDA) sensor to generate stress-indicative representations from thermal videos.
During testing, only thermal sensing is used, and stress-indicative patterns from thermal data and emulated EDA representations are extracted to improve stress assessment.
- Score: 7.302042464450543
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Stress impacts our physical and mental health as well as our social life. A
passive and contactless indoor stress monitoring system can unlock numerous
important applications such as workplace productivity assessment, smart homes,
and personalized mental health monitoring. While the thermal signatures from a
user's body captured by a thermal camera can provide important information
about the "fight-flight" response of the sympathetic and parasympathetic
nervous system, relying solely on thermal imaging for training a stress
prediction model often lead to overfitting and consequently a suboptimal
performance. This paper addresses this challenge by introducing ThermaStrain, a
novel co-teaching framework that achieves high-stress prediction performance by
transferring knowledge from the wearable modality to the contactless thermal
modality. During training, ThermaStrain incorporates a wearable electrodermal
activity (EDA) sensor to generate stress-indicative representations from
thermal videos, emulating stress-indicative representations from a wearable EDA
sensor. During testing, only thermal sensing is used, and stress-indicative
patterns from thermal data and emulated EDA representations are extracted to
improve stress assessment. The study collected a comprehensive dataset with
thermal video and EDA data under various stress conditions and distances.
ThermaStrain achieves an F1 score of 0.8293 in binary stress classification,
outperforming the thermal-only baseline approach by over 9%. Extensive
evaluations highlight ThermaStrain's effectiveness in recognizing
stress-indicative attributes, its adaptability across distances and stress
scenarios, real-time executability on edge platforms, its applicability to
multi-individual sensing, ability to function on limited visibility and
unfamiliar conditions, and the advantages of its co-teaching approach.
Related papers
- Stressor Type Matters! -- Exploring Factors Influencing Cross-Dataset Generalizability of Physiological Stress Detection [5.304745246313982]
This study explores the generalizability of machine learning models trained on HRV features for binary stress detection.
Our findings reveal a crucial factor affecting model generalizability: stressor type.
We recommend matching the stressor type when deploying HRV-based stress models in new environments.
arXiv Detail & Related papers (2024-05-06T14:47:48Z) - Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
Engagement measurement finds application in healthcare, education, services.
Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors.
We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
arXiv Detail & Related papers (2024-04-05T20:39:16Z) - Extreme Miscalibration and the Illusion of Adversarial Robustness [66.29268991629085]
Adversarial Training is often used to increase model robustness.
We show that this observed gain in robustness is an illusion of robustness (IOR)
We urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations.
arXiv Detail & Related papers (2024-02-27T13:49:12Z) - Investigating the Generalizability of Physiological Characteristics of Anxiety [3.4036712573981607]
We evaluate the generalizability of physiological features that have been shown to be correlated with anxiety and stress to high-arousal emotions.
This work is the first cross-corpus evaluation across stress and arousal from ECG and EDA signals, contributing new findings about the generalizability of stress detection.
arXiv Detail & Related papers (2024-01-23T16:49:54Z) - Analysing the Performance of Stress Detection Models on Consumer-Grade
Wearable Devices [9.580380455705397]
Stress levels can provide valuable data for mental health analytics as well as labels for annotation systems.
There is a lack of research on the potential of using low-resolution Electrodermal Activity (EDA) signals from consumer-grade wearable devices to identify stress patterns.
arXiv Detail & Related papers (2022-03-18T00:36:27Z) - Maximizing Self-supervision from Thermal Image for Effective
Self-supervised Learning of Depth and Ego-motion [78.19156040783061]
Self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios.
The inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images.
We propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
arXiv Detail & Related papers (2022-01-12T09:49:24Z) - Personalized Stress Monitoring using Wearable Sensors in Everyday
Settings [9.621481727547215]
We explore objective prediction of stress levels in everyday settings based on heart rate (HR) and heart rate variability (HRV)
We present a layered system architecture for personalized stress monitoring that supports a tunable collection of data samples for labeling, and present a method for selecting informative samples from the stream of real-time data for labeling.
arXiv Detail & Related papers (2021-07-31T04:15:15Z) - Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias.
A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
arXiv Detail & Related papers (2021-07-26T10:40:59Z) - StressNet: Detecting Stress in Thermal Videos [10.453959171422147]
This paper presents a novel approach to obtaining physiological signals and classifying stress states from thermal video.
"StressNet" reconstructs the ISTI ( Initial Systolic Time Interval: a measure of change in cardiac sympathetic activity that is considered to be a quantitative index of stress humans.
A detailed evaluation demonstrates that StressNet estimated the ISTI signal with 95% accuracy and detect stress with average precision of 0.842.
arXiv Detail & Related papers (2020-11-18T20:47:23Z) - Detecting Parkinsonian Tremor from IMU Data Collected In-The-Wild using
Deep Multiple-Instance Learning [59.74684475991192]
Parkinson's Disease (PD) is a slowly evolving neuro-logical disease that affects about 1% of the population above 60 years old.
PD symptoms include tremor, rigidity and braykinesia.
We present a method for automatically identifying tremorous episodes related to PD, based on IMU signals captured via a smartphone device.
arXiv Detail & Related papers (2020-05-06T09:02:30Z) - AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement
with Neural Searching [76.4844593082362]
We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS)
Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
arXiv Detail & Related papers (2020-04-26T05:43:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.