HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition

• URL: http://arxiv.org/abs/2608.27233v1
• Date: Thu, 27 Aug 2026 15:17:12 GMT
• Title: HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition
• Abstract summary: We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model.<n>Heterogeneity-Aware Language-aligned Open-set model addresses two key challenges through a two-stage training framework.<n>Heterogeneity-Aware Language-aligned Open-set model improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points.
• Score: 11.807649073272245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human Activity Recognition (HAR) using inertial measurement units (IMUs) enables a wide range of applications, yet the field still lacks a unified model that can generalize across diverse subjects, devices, and activities. Training such a model is difficult due to two key challenges: sensing heterogeneity -- differences in sampling rates, channel configurations, and sensor placements -- and poor generalization to unseen activities and label vocabularies. We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model that addresses both challenges through a two-stage training framework. Stage 1 pretrains the IMU encoder with heterogeneity-aware self-supervised learning, including adaptive-pooling tokenization, channel-independent feature extraction, and contextualized sensor conditioning that injects natural-language sensor descriptions into each channel embedding. Stage 2 aligns this IMU encoder with text embeddings via synonym-aware soft contrastive learning, enabling open-set recognition via cosine-similarity retrieval without per-dataset classifiers. Trained on 10 public HAR datasets and evaluated on 7 held-out datasets, HALO outperforms five state-of-the-art baselines on all 8 aggregate metrics, and still leads on 3 of 4 settings under baseline-matched inputs. Despite using only ~35M trainable parameters -- 10x fewer than the latest foundation model MOMENT (341.2M) -- HALO improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points. On two further datasets with severe distribution shift, every model including HALO collapses zero-shot. A video demonstration of HALO's performance in real world is available at https://youtu.be/rooVKragtFU

Related papers

• Closing the Modality Gap in Zero-Shot HAR: Contrastive Training and Separability-Optimized Prototypes on IMU Data [0.0]
  We evaluate seven configurations combining three inference methods with two training pipelines on the PAMAP2 dataset.<n>We find that the modality gap is a training-time phenomenon governed by the objective.<n>We also demonstrate that overall accuracy is a misleading primary metric when test-set class distributions are imbalanced.
  arXiv  Detail & Related papers  (2026-06-09T12:39:41Z)
• Pearl: A Foundation Model for Placing Every Atom in the Right Location [52.35027831422145]
  We introduce Pearl, a foundation model for protein-ligand cofolding at scale.<n>Pearl establishes a new state-of-the-art performance in protein-ligand cofolding.<n>Pearl surpasses AlphaFold 3 and other open source baselines on the public Runs N' Poses and PoseBusters benchmarks.
  arXiv  Detail & Related papers  (2025-10-28T17:36:51Z)
• Preliminary Use of Vision Language Model Driven Extraction of Mouse Behavior Towards Understanding Fear Expression [5.170961907232911]
  This work establishes a vision-language model (VLM) that encodes videos with text input in order to classify various behaviors of a mouse.<n>We use the open-source Qwen2.5-VL model and enhance its performance through prompts, in-context learning (ICL) with labeled examples, and frame-level preprocessing.
  arXiv  Detail & Related papers  (2025-10-22T01:33:39Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)
• Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models [81.62767292169225]
  We investigate knowledge forgetting in large language models with a focus on its generalisation.<n>We propose PerMU, a novel probability perturbation-based unlearning paradigm.<n>Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE.
  arXiv  Detail & Related papers  (2025-02-27T11:03:33Z)
• DeepHeteroIoT: Deep Local and Global Learning over Heterogeneous IoT Sensor Data [9.531834233076934]
  We propose a novel deep learning model that incorporates both Convolutional Neural Network and Bi-directional Gated Recurrent Unit to learn local and global features respectively. In particular, the model achieves an average absolute improvement of 3.37% in Accuracy and 2.85% in F1-Score across datasets.
  arXiv  Detail & Related papers  (2024-03-29T06:24:07Z)
• WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition [22.988658969594276]
  We introduce WEAR, an outdoor sports dataset for both vision- and inertial-based human activity recognition (HAR) Data from 22 participants performing a total of 18 different workout activities was collected with synchronized inertial (acceleration) and camera (egocentric video) data recorded at 11 different outside locations. Benchmark results show that through our sensor placement, each modality interestingly offers complementary strengths and weaknesses in their prediction performance.
  arXiv  Detail & Related papers  (2023-04-11T09:31:07Z)
• End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation [86.41437210485932]
  We aim at advancing zero-shot HOI detection to detect both seen and unseen HOIs simultaneously. We propose a novel end-to-end zero-shot HOI Detection framework via vision-language knowledge distillation. Our method outperforms the previous SOTA by 8.92% on unseen mAP and 10.18% on overall mAP.
  arXiv  Detail & Related papers  (2022-04-01T07:27:19Z)
• Disentangle Your Dense Object Detector [82.22771433419727]
  Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. We propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art detectors.
  arXiv  Detail & Related papers  (2021-07-07T00:52:16Z)
• Invariant Feature Learning for Sensor-based Human Activity Recognition [11.334750079923428]
  We present an invariant feature learning framework (IFLF) that extracts common information shared across subjects and devices. Experiments demonstrated that IFLF is effective in handling both subject and device diversion across popular open datasets and an in-house dataset.
  arXiv  Detail & Related papers  (2020-12-14T21:56:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.