Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition

• URL: http://arxiv.org/abs/2501.10917v1
• Date: Sun, 19 Jan 2025 01:52:28 GMT
• Title: Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition
• Authors: Haoyu Xie, Haoxuan Li, Chunyuan Zheng, Haonan Yuan, Guorui Liao, Jun Liao, Li Liu,
• Abstract summary: We propose the DecomposeWHAR model to better model the relationships between modality variables. The decomposition creates high-dimensional representations of each intra-sensor variable. The fusion phase begins by capturing relationships between intra-sensor variables and fusing their features at both the channel and variable levels.
• Score: 12.359681612030682
• License:
• Abstract: Wearable Human Activity Recognition (WHAR) is a prominent research area within ubiquitous computing. Multi-sensor synchronous measurement has proven to be more effective for WHAR than using a single sensor. However, existing WHAR methods use shared convolutional kernels for indiscriminate temporal feature extraction across each sensor variable, which fails to effectively capture spatio-temporal relationships of intra-sensor and inter-sensor variables. We propose the DecomposeWHAR model consisting of a decomposition phase and a fusion phase to better model the relationships between modality variables. The decomposition creates high-dimensional representations of each intra-sensor variable through the improved Depth Separable Convolution to capture local temporal features while preserving their unique characteristics. The fusion phase begins by capturing relationships between intra-sensor variables and fusing their features at both the channel and variable levels. Long-range temporal dependencies are modeled using the State Space Model (SSM), and later cross-sensor interactions are dynamically captured through a self-attention mechanism, highlighting inter-sensor spatial correlations. Our model demonstrates superior performance on three widely used WHAR datasets, significantly outperforming state-of-the-art models while maintaining acceptable computational efficiency. Our codes and supplementary materials are available at https://github.com/Anakin2555/DecomposeWHAR.

Related papers

• STeInFormer: Spatial-Temporal Interaction Transformer Architecture for Remote Sensing Change Detection [5.4610555622532475]
  We present STeInFormer, a spatial-temporal interaction Transformer architecture for multi-temporal feature extraction. We also propose a parameter-free multi-frequency token mixer to integrate frequency-domain features that provide spectral information for RSCD.
  arXiv  Detail & Related papers  (2024-12-23T03:40:04Z)
• Temporal and Heterogeneous Graph Neural Network for Remaining Useful Life Prediction [27.521188262343596]
  We introduce a novel model named Temporal and Heterogeneous Graph Neural Networks (THGNN) THGNN aggregates historical data from neighboring nodes to accurately capture the temporal dynamics and spatial correlations within the stream of sensor data. We have validated the effectiveness of our approach through comprehensive experiments.
  arXiv  Detail & Related papers  (2024-05-07T14:08:57Z)
• Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving [69.20604395205248]
  We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving. We introduce an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector. We conduct extensive experiments on the KITTI, runtime, and nuScenes datasets, and the results demonstrate the state-of-the-art performance of Shift-SSD.
  arXiv  Detail & Related papers  (2024-03-10T10:36:32Z)
• Graph-Aware Contrasting for Multivariate Time-Series Classification [50.84488941336865]
  Existing contrastive learning methods mainly focus on achieving temporal consistency with temporal augmentation and contrasting techniques. We propose Graph-Aware Contrasting for spatial consistency across MTS data. Our proposed method achieves state-of-the-art performance on various MTS classification tasks.
  arXiv  Detail & Related papers  (2023-09-11T02:35:22Z)
• Data-Induced Interactions of Sparse Sensors [3.050919759387984]
  We take a thermodynamic view to compute the full landscape of sensor interactions induced by the training data. Mapping out these data-induced sensor interactions allows combining them with external selection criteria and anticipating sensor replacement impacts.
  arXiv  Detail & Related papers  (2023-07-21T18:13:37Z)
• Multi-scale Spatial-temporal Interaction Network for Video Anomaly Detection [3.113134714967787]
  Video Anomaly Detection (VAD) is an essential yet challenging task in signal processing. We propose a Multi-scale Spatial-Temporal Interaction Network (MSTI-Net) for VAD.
  arXiv  Detail & Related papers  (2023-06-17T02:40:29Z)
• DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness [78.98998551326812]
  We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis. We show that the method can exploit the multi-modality features from related sensors and also learn from history time-series dynamics to reconstruct the data under extreme missingness.
  arXiv  Detail & Related papers  (2022-09-26T21:59:14Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• DecAug: Augmenting HOI Detection via Decomposition [54.65572599920679]
  Current algorithms suffer from insufficient training samples and category imbalance within datasets. We propose an efficient and effective data augmentation method called DecAug for HOI detection. Experiments show that our method brings up to 3.3 mAP and 1.6 mAP improvements on V-COCO and HICODET dataset.
  arXiv  Detail & Related papers  (2020-10-02T13:59:05Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• Human Activity Recognition from Wearable Sensor Data Using Self-Attention [2.9023633922848586]
  We present a self-attention based neural network model for activity recognition from body-worn sensor data. We performed experiments on four popular publicly available HAR datasets: PAMAP2, Opportunity, Skoda and USC-HAD. Our model achieve significant performance improvement over recent state-of-the-art models in both benchmark test subjects and Leave-one-out-subject evaluation.
  arXiv  Detail & Related papers  (2020-03-17T14:16:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.