In Shift and In Variance: Assessing the Robustness of HAR Deep Learning Models against Variability

• URL: http://arxiv.org/abs/2503.11466v1
• Date: Fri, 14 Mar 2025 14:53:56 GMT
• Title: In Shift and In Variance: Assessing the Robustness of HAR Deep Learning Models against Variability
• Authors: Azhar Ali Khaked, Nobuyuki Oishi, Daniel Roggen, Paula Lago,
• Abstract summary: Human Activity Recognition (HAR) using wearable inertial measurement unit (IMU) sensors can revolutionize healthcare by enabling continual health monitoring, disease prediction, and routine recognition.<n>Despite the high accuracy of Deep Learning (DL) HAR models, their robustness to real-world variabilities remains untested.<n>We isolate subject, device, position, and orientation variability to determine their effect on DL HAR models and assess the robustness of these models in real-world conditions.
• Score: 4.330123738563178
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Human Activity Recognition (HAR) using wearable inertial measurement unit (IMU) sensors can revolutionize healthcare by enabling continual health monitoring, disease prediction, and routine recognition. Despite the high accuracy of Deep Learning (DL) HAR models, their robustness to real-world variabilities remains untested, as they have primarily been trained and tested on limited lab-confined data. In this study, we isolate subject, device, position, and orientation variability to determine their effect on DL HAR models and assess the robustness of these models in real-world conditions. We evaluated the DL HAR models using the HARVAR and REALDISP datasets, providing a comprehensive discussion on the impact of variability on data distribution shifts and changes in model performance. Our experiments measured shifts in data distribution using Maximum Mean Discrepancy (MMD) and observed DL model performance drops due to variability. We concur that studied variabilities affect DL HAR models differently, and there is an inverse relationship between data distribution shifts and model performance. The compounding effect of variability was analyzed, and the implications of variabilities in real-world scenarios were highlighted. MMD proved an effective metric for calculating data distribution shifts and explained the drop in performance due to variabilities in HARVAR and REALDISP datasets. Combining our understanding of variability with evaluating its effects will facilitate the development of more robust DL HAR models and optimal training techniques. Allowing Future models to not only be assessed based on their maximum F1 score but also on their ability to generalize effectively

Related papers

• A Semi-supervised CART Model for Covariate Shift [0.0]
  This paper introduces a semi-supervised classification and regression tree (CART) that uses importance weighting to address distribution discrepancies.<n>Our method improves the predictive performance of the CART model by assigning greater weights to training samples.<n>Through simulation studies and applications to real-world medical data, we demonstrate significant improvements in predictive accuracy.
  arXiv  Detail & Related papers  (2024-10-28T12:53:23Z)
• Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration [74.09687562334682]
  We introduce a novel training data attribution method called Debias and Denoise Attribution (DDA) Our method significantly outperforms existing approaches, achieving an averaged AUC of 91.64%. DDA exhibits strong generality and scalability across various sources and different-scale models like LLaMA2, QWEN2, and Mistral.
  arXiv  Detail & Related papers  (2024-10-02T07:14:26Z)
• DLFormer: Enhancing Explainability in Multivariate Time Series Forecasting using Distributed Lag Embedding [4.995397953581609]
  This study introduces DLFormer, an attention-based architecture integrated with distributed lag embedding. It showcases superior performance improvements compared to existing attention-based high-performance models.
  arXiv  Detail & Related papers  (2024-08-29T20:39:54Z)
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• MPRE: Multi-perspective Patient Representation Extractor for Disease Prediction [3.914545513460964]
  We propose the Multi-perspective Patient Representation Extractor (MPRE) for disease prediction. Specifically, we propose Frequency Transformation Module (FTM) to extract the trend and variation information of dynamic features. In the 2D Multi-Extraction Network (2D MEN), we form the 2D temporal tensor based on trend and variation. We also propose the First-Order Difference Attention Mechanism (FODAM) to calculate the contributions of differences in adjacent variations to the disease diagnosis.
  arXiv  Detail & Related papers  (2024-01-01T13:52:05Z)
• Causal Dynamic Variational Autoencoder for Counterfactual Regression in Longitudinal Data [3.662229789022107]
  Estimating treatment effects over time is relevant in many real-world applications, such as precision medicine, epidemiology, economy, and marketing. We take a different perspective by assuming unobserved risk factors, i.e., adjustment variables that affect only the sequence of outcomes. We address the challenges posed by time-varying effects and unobserved adjustment variables.
  arXiv  Detail & Related papers  (2023-10-16T16:32:35Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• How robust are pre-trained models to distribution shift? [82.08946007821184]
  We show how spurious correlations affect the performance of popular self-supervised learning (SSL) and auto-encoder based models (AE) We develop a novel evaluation scheme with the linear head trained on out-of-distribution (OOD) data, to isolate the performance of the pre-trained models from a potential bias of the linear head used for evaluation.
  arXiv  Detail & Related papers  (2022-06-17T16:18:28Z)
• BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark [21.040754460129854]
  We release BEDS-Bench, a benchmark for quantifying the behavior of ML models over EHR data under OOD settings. We evaluate several learning algorithms under BEDS-Bench and find that all of them show poor generalization performance under distributional shift in general.
  arXiv  Detail & Related papers  (2021-07-17T05:53:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.