Contrastive Predictive Coding for Human Activity Recognition

• URL: http://arxiv.org/abs/2012.05333v1
• Date: Wed, 9 Dec 2020 21:44:36 GMT
• Title: Contrastive Predictive Coding for Human Activity Recognition
• Authors: Harish Haresamudram, Irfan Essa, Thomas Ploetz
• Abstract summary: We introduce the Contrastive Predictive Coding framework to human activity recognition, which captures the long-term temporal structure of sensor data streams. CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains. It leads to significantly improved recognition performance when only small amounts of labeled training data are available.
• Score: 5.766384728949437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data that are straightforward to collect in mobile and ubiquitous computing scenarios. We hypothesize and demonstrate that explicitly considering the temporality of sensor data at representation level plays an important role for effective HAR in challenging scenarios. We introduce the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the long-term temporal structure of sensor data streams. Through a range of experimental evaluations on real-life recognition tasks, we demonstrate its effectiveness for improved HAR. CPC-based pre-training is self-supervised, and the resulting learned representations can be integrated into standard activity chains. It leads to significantly improved recognition performance when only small amounts of labeled training data are available, thereby demonstrating the practical value of our approach.

Related papers

• Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition [0.2944538605197902]
  We present an approach for economic use of publicly available labeled HAR datasets for effective transfer learning. We introduce a novel transfer learning framework, Cross-Domain HAR, which follows the teacher-student self-training paradigm. We demonstrate the effectiveness of our approach for practically relevant few shot activity recognition scenarios.
  arXiv  Detail & Related papers  (2023-10-22T19:13:25Z)
• A Real-time Human Pose Estimation Approach for Optimal Sensor Placement in Sensor-based Human Activity Recognition [63.26015736148707]
  This paper introduces a novel methodology to resolve the issue of optimal sensor placement for Human Activity Recognition. The derived skeleton data provides a unique strategy for identifying the optimal sensor location. Our findings indicate that the vision-based method for sensor placement offers comparable results to the conventional deep learning approach.
  arXiv  Detail & Related papers  (2023-07-06T10:38:14Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition [7.086647707011785]
  Human activity recognition (HAR) in wearable computing is typically based on direct processing of sensor data. Recent advancements in Vector Quantization (VQ) to wearables applications enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors. This work presents a proof-of-concept for demonstrating how effective discrete representations can be derived.
  arXiv  Detail & Related papers  (2023-06-01T19:49:43Z)
• A Matter of Annotation: An Empirical Study on In Situ and Self-Recall Activity Annotations from Wearable Sensors [56.554277096170246]
  We present an empirical study that evaluates and contrasts four commonly employed annotation methods in user studies focused on in-the-wild data collection. For both the user-driven, in situ annotations, where participants annotate their activities during the actual recording process, and the recall methods, where participants retrospectively annotate their data at the end of each day, the participants had the flexibility to select their own set of activity classes and corresponding labels.
  arXiv  Detail & Related papers  (2023-05-15T16:02:56Z)
• Human Activity Recognition Using Self-Supervised Representations of Wearable Data [0.0]
  Development of accurate algorithms for human activity recognition (HAR) is hindered by the lack of large real-world labeled datasets. Here we develop a 6-class HAR model with strong performance when evaluated on real-world datasets not seen during training.
  arXiv  Detail & Related papers  (2023-04-26T07:33:54Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• ALLSH: Active Learning Guided by Local Sensitivity and Hardness [98.61023158378407]
  We propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks.
  arXiv  Detail & Related papers  (2022-05-10T15:39:11Z)
• Human Activity Recognition on wrist-worn accelerometers using self-supervised neural networks [0.0]
  Measures of Activity of Daily Living (ADL) are an important indicator of overall health but difficult to measure in-clinic. We propose a self-supervised learning paradigm to create a robust representation of accelerometer data that can generalize across devices and subjects. We also propose a segmentation algorithm which can identify segments of salient activity and boost HAR accuracy on continuous real-life data.
  arXiv  Detail & Related papers  (2021-12-22T23:35:20Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)
• Sensor Data for Human Activity Recognition: Feature Representation and Benchmarking [27.061240686613182]
  The field of Human Activity Recognition (HAR) focuses on obtaining and analysing data captured from monitoring devices (e.g. sensors) We address the issue of accurately recognising human activities using different Machine Learning (ML) techniques.
  arXiv  Detail & Related papers  (2020-05-15T00:46:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.