Temporal Action Localization for Inertial-based Human Activity Recognition

• URL: http://arxiv.org/abs/2311.15831v2
• Date: Mon, 14 Oct 2024 12:33:40 GMT
• Title: Temporal Action Localization for Inertial-based Human Activity Recognition
• Authors: Marius Bock, Michael Moeller, Kristof Van Laerhoven,
• Abstract summary: Video-based Human Activity Recognition (TAL) has followed a segment-based prediction approach, localizing activity segments in a timeline of arbitrary length. This paper is the first to systematically demonstrate the applicability of state-of-the-art TAL models for both offline and near-online Human Activity Recognition (HAR) We show that by analyzing timelines as a whole, TAL models can produce more coherent segments and achieve higher NULL-class accuracy across all datasets.
• Score: 9.948823510429902
• License:
• Abstract: As of today, state-of-the-art activity recognition from wearable sensors relies on algorithms being trained to classify fixed windows of data. In contrast, video-based Human Activity Recognition, known as Temporal Action Localization (TAL), has followed a segment-based prediction approach, localizing activity segments in a timeline of arbitrary length. This paper is the first to systematically demonstrate the applicability of state-of-the-art TAL models for both offline and near-online Human Activity Recognition (HAR) using raw inertial data as well as pre-extracted latent features as input. Offline prediction results show that TAL models are able to outperform popular inertial models on a multitude of HAR benchmark datasets, with improvements reaching as much as 26% in F1-score. We show that by analyzing timelines as a whole, TAL models can produce more coherent segments and achieve higher NULL-class accuracy across all datasets. We demonstrate that TAL is less suited for the immediate classification of small-sized windows of data, yet offers an interesting perspective on inertial-based HAR -- alleviating the need for fixed-size windows and enabling algorithms to recognize activities of arbitrary length. With design choices and training concepts yet to be explored, we argue that TAL architectures could be of significant value to the inertial-based HAR community. The code and data download to reproduce experiments is publicly available via github.com/mariusbock/tal_for_har.

Related papers

• DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs [59.434893231950205]
  Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
  arXiv  Detail & Related papers  (2024-08-13T15:21:46Z)
• Generative Active Learning for Long-tailed Instance Segmentation [55.66158205855948]
  We propose BSGAL, a new algorithm that estimates the contribution of generated data based on cache gradient. Experiments show that BSGAL outperforms the baseline approach and effectually improves the performance of long-tailed segmentation.
  arXiv  Detail & Related papers  (2024-06-04T15:57:43Z)
• HARMamba: Efficient and Lightweight Wearable Sensor Human Activity Recognition Based on Bidirectional Mamba [7.412537185607976]
  Wearable sensor-based human activity recognition (HAR) is a critical research domain in activity perception. This study introduces HARMamba, an innovative light-weight and versatile HAR architecture that combines selective bidirectional State Spaces Model and hardware-aware design. HarMamba outperforms contemporary state-of-the-art frameworks, delivering comparable or better accuracy with significantly reducing computational and memory demands.
  arXiv  Detail & Related papers  (2024-03-29T13:57:46Z)
• Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition [7.086647707011785]
  Human activity recognition (HAR) in wearable computing is typically based on direct processing of sensor data. Recent advancements in Vector Quantization (VQ) to wearables applications enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors. This work presents a proof-of-concept for demonstrating how effective discrete representations can be derived.
  arXiv  Detail & Related papers  (2023-06-01T19:49:43Z)
• Human Activity Recognition Using Self-Supervised Representations of Wearable Data [0.0]
  Development of accurate algorithms for human activity recognition (HAR) is hindered by the lack of large real-world labeled datasets. Here we develop a 6-class HAR model with strong performance when evaluated on real-world datasets not seen during training.
  arXiv  Detail & Related papers  (2023-04-26T07:33:54Z)
• Reinforcement Learning from Passive Data via Latent Intentions [86.4969514480008]
  We show that passive data can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions. Our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.
  arXiv  Detail & Related papers  (2023-04-10T17:59:05Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Beyond the Gates of Euclidean Space: Temporal-Discrimination-Fusions and Attention-based Graph Neural Network for Human Activity Recognition [5.600003119721707]
  Human activity recognition (HAR) through wearable devices has received much interest due to its numerous applications in fitness tracking, wellness screening, and supported living. Traditional deep learning (DL) has set a state of the art performance for HAR domain. We propose an approach based on Graph Neural Networks (GNNs) for structuring the input representation and exploiting the relations among the samples.
  arXiv  Detail & Related papers  (2022-06-10T03:04:23Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.