UMSNet: An Universal Multi-sensor Network for Human Activity Recognition

• URL: http://arxiv.org/abs/2205.11756v1
• Date: Tue, 24 May 2022 03:29:54 GMT
• Title: UMSNet: An Universal Multi-sensor Network for Human Activity Recognition
• Authors: Jialiang Wang, Haotian Wei, Yi Wang, Shu Yang, Chi Li
• Abstract summary: This paper proposes a universal multi-sensor network (UMSNet) for human activity recognition. In particular, we propose a new lightweight sensor residual block (called LSR block), which improves the performance. Our framework has a clear structure and can be directly applied to various types of multi-modal Time Series Classification tasks.
• Score: 10.952666953066542
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human activity recognition (HAR) based on multimodal sensors has become a rapidly growing branch of biometric recognition and artificial intelligence. However, how to fully mine multimodal time series data and effectively learn accurate behavioral features has always been a hot topic in this field. Practical applications also require a well-generalized framework that can quickly process a variety of raw sensor data and learn better feature representations. This paper proposes a universal multi-sensor network (UMSNet) for human activity recognition. In particular, we propose a new lightweight sensor residual block (called LSR block), which improves the performance by reducing the number of activation function and normalization layers, and adding inverted bottleneck structure and grouping convolution. Then, the Transformer is used to extract the relationship of series features to realize the classification and recognition of human activities. Our framework has a clear structure and can be directly applied to various types of multi-modal Time Series Classification (TSC) tasks after simple specialization. Extensive experiments show that the proposed UMSNet outperforms other state-of-the-art methods on two popular multi-sensor human activity recognition datasets (i.e. HHAR dataset and MHEALTH dataset).

Related papers

• Scaling Wearable Foundation Models [54.93979158708164]
  We investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM. Our results establish the scaling laws of LSM for tasks such as imputation, extrapolation, both across time and sensor modalities.
  arXiv  Detail & Related papers  (2024-10-17T15:08:21Z)
• Feature Decoupling-Recycling Network for Fast Interactive Segmentation [79.22497777645806]
  Recent interactive segmentation methods iteratively take source image, user guidance and previously predicted mask as the input. We propose the Feature Decoupling-Recycling Network (FDRN), which decouples the modeling components based on their intrinsic discrepancies.
  arXiv  Detail & Related papers  (2023-08-07T12:26:34Z)
• Mobile Behavioral Biometrics for Passive Authentication [65.94403066225384]
  This work carries out a comparative analysis of unimodal and multimodal behavioral biometric traits. Experiments are performed over HuMIdb, one of the largest and most comprehensive freely available mobile user interaction databases. In our experiments, the most discriminative background sensor is the magnetometer, whereas among touch tasks the best results are achieved with keystroke.
  arXiv  Detail & Related papers  (2022-03-14T17:05:59Z)
• Attention-Based Sensor Fusion for Human Activity Recognition Using IMU Signals [4.558966602878624]
  We propose a novel attention-based approach to human activity recognition using multiple IMU sensors worn at different body locations. An attention-based fusion mechanism is developed to learn the importance of sensors at different body locations and to generate an attentive feature representation. The proposed approach is evaluated using five public datasets and it outperforms state-of-the-art methods on a wide variety of activity categories.
  arXiv  Detail & Related papers  (2021-12-20T17:00:27Z)
• Human Activity Recognition using Attribute-Based Neural Networks and Context Information [61.67246055629366]
  We consider human activity recognition (HAR) from wearable sensor data in manual-work processes. We show how context information can be integrated systematically into a deep neural network-based HAR system. We empirically show that our proposed architecture increases HAR performance, compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-28T06:08:25Z)
• A Novel Multi-Stage Training Approach for Human Activity Recognition from Multimodal Wearable Sensor Data Using Deep Neural Network [11.946078871080836]
  Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. In this paper, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process.
  arXiv  Detail & Related papers  (2021-01-03T20:48:56Z)
• A Tree-structure Convolutional Neural Network for Temporal Features Exaction on Sensor-based Multi-resident Activity Recognition [4.619245607612873]
  We propose an end-to-end Tree-Structure Convolutional neural network based framework for Multi-Resident Activity Recognition (TSC-MRAR) First, we treat each sample as an event and obtain the current event embedding through the previous sensor readings in the sliding window. Then, in order to automatically generate the temporal features, a tree-structure network is designed to derive the temporal dependence of nearby readings.
  arXiv  Detail & Related papers  (2020-11-05T14:31:00Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition [89.0152015268929]
  We propose the first neural architecture search (NAS)-based method for RGB-D gesture recognition. The proposed method includes two key components: 1) enhanced temporal representation via the 3D Central Difference Convolution (3D-CDC) family, and optimized backbones for multi-modal-rate branches and lateral connections. The resultant multi-rate network provides a new perspective to understand the relationship between RGB and depth modalities and their temporal dynamics.
  arXiv  Detail & Related papers  (2020-08-21T10:45:09Z)
• Deep ConvLSTM with self-attention for human activity decoding using wearables [0.0]
  We propose a deep neural network architecture that captures features of multiple sensor time-series data but also selects important time points. We show the validity of the proposed approach across different data sampling strategies and demonstrate that the self-attention mechanism gave a significant improvement. The proposed methods open avenues for better decoding of human activity from multiple body sensors over extended periods time.
  arXiv  Detail & Related papers  (2020-05-02T04:30:31Z)
• Sequential Weakly Labeled Multi-Activity Localization and Recognition on Wearable Sensors using Recurrent Attention Networks [13.64024154785943]
  We propose a recurrent attention network (RAN) to handle sequential weakly labeled multi-activity recognition and location tasks. Our RAN model can simultaneously infer multi-activity types from the coarse-grained sequential weak labels. It will greatly reduce the burden of manual labeling.
  arXiv  Detail & Related papers  (2020-04-13T04:57:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.