Video-based Pose-Estimation Data as Source for Transfer Learning in Human Activity Recognition

• URL: http://arxiv.org/abs/2212.01353v1
• Date: Fri, 2 Dec 2022 18:19:36 GMT
• Title: Video-based Pose-Estimation Data as Source for Transfer Learning in Human Activity Recognition
• Authors: Shrutarv Awasthi, Fernando Moya Rueda, Gernot A. Fink
• Abstract summary: Human Activity Recognition (HAR) using on-body devices identifies specific human actions in unconstrained environments. Previous works demonstrated that transfer learning is a good strategy for addressing scenarios with scarce data. This paper proposes using datasets intended for human-pose estimation as a source for transfer learning.
• Score: 71.91734471596433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human Activity Recognition (HAR) using on-body devices identifies specific human actions in unconstrained environments. HAR is challenging due to the inter and intra-variance of human movements; moreover, annotated datasets from on-body devices are scarce. This problem is mainly due to the difficulty of data creation, i.e., recording, expensive annotation, and lack of standard definitions of human activities. Previous works demonstrated that transfer learning is a good strategy for addressing scenarios with scarce data. However, the scarcity of annotated on-body device datasets remains. This paper proposes using datasets intended for human-pose estimation as a source for transfer learning; specifically, it deploys sequences of annotated pixel coordinates of human joints from video datasets for HAR and human pose estimation. We pre-train a deep architecture on four benchmark video-based source datasets. Finally, an evaluation is carried out on three on-body device datasets improving HAR performance.

Related papers

• A Unified Framework for Human-centric Point Cloud Video Understanding [23.91555808792291]
  Human-centric Point Cloud Video Understanding (PVU) is an emerging field focused on extracting and interpreting human-related features from sequences of human point clouds. We propose a unified framework to make full use of the prior knowledge and explore the inherent features in the data itself for generalized human-centric point cloud video understanding. Our method achieves state-of-the-art performance on various human-related tasks, including action recognition and 3D pose estimation.
  arXiv  Detail & Related papers  (2024-03-29T07:53:06Z)
• Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review [0.8142555609235358]
  Sensor-based Human Activity Recognition (HAR) is crucial in ubiquitous computing. HAR confronts challenges, particularly in data distribution assumptions. This review investigates how machine learning addresses data heterogeneity in HAR.
  arXiv  Detail & Related papers  (2024-03-12T22:22:14Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining [75.1086193340286]
  It is desirable to have a general pretrain model for versatile human-centric downstream tasks. We propose a textbfHumanBench based on existing datasets to evaluate on the common ground the generalization abilities of different pretraining methods. Our PATH achieves new state-of-the-art results on 17 downstream datasets and on-par results on the other 2 datasets.
  arXiv  Detail & Related papers  (2023-03-10T02:57:07Z)
• Dataset Bias in Human Activity Recognition [57.91018542715725]
  This contribution statistically curates the training data to assess to what degree the physical characteristics of humans influence HAR performance. We evaluate the performance of a state-of-the-art convolutional neural network on two HAR datasets that vary in the sensors, activities, and recording for time-series HAR.
  arXiv  Detail & Related papers  (2023-01-19T12:33:50Z)
• HighlightMe: Detecting Highlights from Human-Centric Videos [52.84233165201391]
  We present a domain- and user-preference-agnostic approach to detect highlightable excerpts from human-centric videos. We use an autoencoder network equipped with spatial-temporal graph convolutions to detect human activities and interactions. We observe a 4-12% improvement in the mean average precision of matching the human-annotated highlights over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-05T01:18:15Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition [12.91206329972949]
  We introduce IMUTube, an automated processing pipeline to convert videos of human activity into virtual streams of IMU data. These virtual IMU streams represent accelerometry at a wide variety of locations on the human body. We show how the virtually-generated IMU data improves the performance of a variety of models on known HAR datasets.
  arXiv  Detail & Related papers  (2020-05-29T21:50:38Z)
• Sensor Data for Human Activity Recognition: Feature Representation and Benchmarking [27.061240686613182]
  The field of Human Activity Recognition (HAR) focuses on obtaining and analysing data captured from monitoring devices (e.g. sensors) We address the issue of accurately recognising human activities using different Machine Learning (ML) techniques.
  arXiv  Detail & Related papers  (2020-05-15T00:46:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.