EgoCHARM: Resource-Efficient Hierarchical Activity Recognition using an Egocentric IMU Sensor

• URL: http://arxiv.org/abs/2504.17735v1
• Date: Thu, 24 Apr 2025 16:48:45 GMT
• Title: EgoCHARM: Resource-Efficient Hierarchical Activity Recognition using an Egocentric IMU Sensor
• Authors: Akhil Padmanabha, Saravanan Govindarajan, Hwanmun Kim, Sergio Ortiz, Rahul Rajan, Doruk Senkal, Sneha Kadetotad,
• Abstract summary: We introduce a resource (memory, compute, power, sample) efficient machine learning algorithm, EgoCHARM, for recognizing both high level and low level activities.<n>Our hierarchical algorithm employs a semi-supervised learning strategy, requiring primarily high level activity labels for training, to learn generalizable low level motion embeddings.<n>We evaluate our method on 9 high level and 3 low level activities achieving 0.826 and 0.855 F1 scores on high level and low level activity recognition respectively, with just 63k high level and 22k low level model parameters.
• Score: 1.2833579625749376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human activity recognition (HAR) on smartglasses has various use cases, including health/fitness tracking and input for context-aware AI assistants. However, current approaches for egocentric activity recognition suffer from low performance or are resource-intensive. In this work, we introduce a resource (memory, compute, power, sample) efficient machine learning algorithm, EgoCHARM, for recognizing both high level and low level activities using a single egocentric (head-mounted) Inertial Measurement Unit (IMU). Our hierarchical algorithm employs a semi-supervised learning strategy, requiring primarily high level activity labels for training, to learn generalizable low level motion embeddings that can be effectively utilized for low level activity recognition. We evaluate our method on 9 high level and 3 low level activities achieving 0.826 and 0.855 F1 scores on high level and low level activity recognition respectively, with just 63k high level and 22k low level model parameters, allowing the low level encoder to be deployed directly on current IMU chips with compute. Lastly, we present results and insights from a sensitivity analysis and highlight the opportunities and limitations of activity recognition using egocentric IMUs.

Related papers

• MR-GDINO: Efficient Open-World Continual Object Detection [58.066277387205325]
  We propose an open-world continual object detection task requiring detectors to generalize to old, new, and unseen categories.<n>We present a challenging yet practical OW-COD benchmark to assess detection abilities.<n>To mitigate forgetting in unseen categories, we propose MR-GDINO, a strong, efficient and scalable baseline via memory and retrieval mechanisms.
  arXiv  Detail & Related papers  (2024-12-20T15:22:51Z)
• Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity [51.40558987254471]
  Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations. This paper addresses the question of reinforcement learning under $textitgeneral$ latent dynamics from a statistical and algorithmic perspective.
  arXiv  Detail & Related papers  (2024-10-23T14:22:49Z)
• Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes [70.66864668709677]
  We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures.
  arXiv  Detail & Related papers  (2024-07-13T01:41:12Z)
• HARMamba: Efficient and Lightweight Wearable Sensor Human Activity Recognition Based on Bidirectional Mamba [7.412537185607976]
  Wearable sensor-based human activity recognition (HAR) is a critical research domain in activity perception. This study introduces HARMamba, an innovative light-weight and versatile HAR architecture that combines selective bidirectional State Spaces Model and hardware-aware design. HarMamba outperforms contemporary state-of-the-art frameworks, delivering comparable or better accuracy with significantly reducing computational and memory demands.
  arXiv  Detail & Related papers  (2024-03-29T13:57:46Z)
• Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning [66.00600682711995]
  Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks. One generalizable and scalable strategy for HOI detection is to use weak supervision, learning from image-level annotations only. This is inherently challenging due to ambiguous human-object associations, large search space of detecting HOIs and highly noisy training signal. We develop a CLIP-guided HOI representation capable of incorporating the prior knowledge at both image level and HOI instance level, and adopt a self-taught mechanism to prune incorrect human-object associations.
  arXiv  Detail & Related papers  (2023-03-02T14:41:31Z)
• CHARM: A Hierarchical Deep Learning Model for Classification of Complex Human Activities Using Motion Sensors [0.9594432031144714]
  CHARM is a hierarchical deep learning model for classification of complex human activities using motion sensors. It outperforms state-of-the-art supervised learning approaches for high-level activity recognition in terms of average accuracy and F1 scores. The ability to learn low-level user activities when trained using only high-level activity labels may pave the way to semi-supervised learning of HAR tasks.
  arXiv  Detail & Related papers  (2022-07-16T01:36:54Z)
• Classifying Human Activities using Machine Learning and Deep Learning Techniques [0.0]
  Human Activity Recognition (HAR) describes the machines ability to recognize human actions. Challenge in HAR is to overcome the difficulties of separating human activities based on the given data. Deep Learning techniques like Long Short-Term Memory (LSTM), Bi-Directional LS classifier, Recurrent Neural Network (RNN), and Gated Recurrent Unit (GRU) are trained. Experiment results proved that the Linear Support Vector in machine learning and Gated Recurrent Unit in Deep Learning provided better accuracy for human activity recognition.
  arXiv  Detail & Related papers  (2022-05-19T05:20:04Z)
• Hierarchical Skills for Efficient Exploration [70.62309286348057]
  In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-20T22:29:32Z)
• Incremental Learning Techniques for Online Human Activity Recognition [0.0]
  We propose a human activity recognition (HAR) approach for the online prediction of physical movements. We develop a HAR system containing monitoring software and a mobile application that collects accelerometer and gyroscope data. Six incremental learning algorithms are employed and evaluated in this work and compared with several batch learning algorithms commonly used for developing offline HAR systems.
  arXiv  Detail & Related papers  (2021-09-20T11:33:09Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)
• On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake? [31.563514432259897]
  We present a novel framework integrating self-supervised pretraining, active learning, and consistency-regularized self-training. Our experiments reveal two key insights: (i) Self-supervised pre-training significantly improves semi-supervised learning, especially in the few-label regime. We fail to observe any additional benefit of state-of-the-art active learning algorithms when combined with state-of-the-art S4L techniques.
  arXiv  Detail & Related papers  (2020-11-16T17:34:55Z)
• HHAR-net: Hierarchical Human Activity Recognition using Neural Networks [2.4530909757679633]
  This research aims at building a hierarchical classification with Neural Networks to recognize human activities. We evaluate our model on the Extrasensory dataset; a dataset collected in the wild and containing data from smartphones and smartwatches.
  arXiv  Detail & Related papers  (2020-10-28T17:06:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.