AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

• URL: http://arxiv.org/abs/2605.22715v2
• Date: Sat, 23 May 2026 13:34:44 GMT
• Title: AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild
• Abstract summary: We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling.<n>AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals.<n>We evaluate AnyMo on three complementary tasks: zero-shot activity recognition, cross-modal retrieval, and wearable IMU motion captioning.
• Score: 10.47918824187894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream datasets, cross-modal retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7\%/11.6\%/22.6\% on HAR, increases zero-shot IMU-to-text and text-to-IMU retrieval MRR by 15.9\% and 28.6\%, respectively, and improves zero-shot captioning BERT-F1 by 18.8\%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.

Related papers

• HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition [11.807649073272245]
  We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model.<n>Heterogeneity-Aware Language-aligned Open-set model addresses two key challenges through a two-stage training framework.<n>Heterogeneity-Aware Language-aligned Open-set model improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points.
  arXiv  Detail & Related papers  (2026-08-27T15:17:12Z)
• Towards Real-World Wearable Motion Reconstruction [68.1355376294881]
  We present a large-scale dataset synchronizing consumer-grade sensors with ground-truth 3D motion.<n>Second, we propose WHIP, a baseline generative model that reconstructs motion from arbitrary subsets of available sensors.<n>Third, we conduct a systematic study of sensor complementarity, quantifying how different modalities complement one another.
  arXiv  Detail & Related papers  (2026-07-08T11:56:02Z)
• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry [10.274521730302029]
  Inertial odometry (IO) provides a lightweight solution for human motion tracking in augmented reality (AR) and wearable devices.<n>Recent learning-based IO methods have improved the generalizability of inertial localization through large-scale pretraining on human motion datasets.<n>We propose to ground IO in human kinematics through a learned IMU-inferred pose prior, which promotes physically consistent motion constraints.
  arXiv  Detail & Related papers  (2026-06-02T01:12:28Z)
• AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling [60.13338774249153]
  We introduce OmniHuMo, a dataset of over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations.<n>We propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer.<n>Experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
  arXiv  Detail & Related papers  (2026-05-28T07:15:19Z)
• Garment Inertial Denoiser (GID): Endowing Accurate Motion Capture via Loose IMU Denoiser [24.814697238576088]
  We propose GID (Garment Inertial Denoiser), a lightweight, plug-and-play Transformer that factorizes loosewear MoCap into three stages.<n>GID uses a location-aware expert architecture, where a shared-temporal backbone models global motion while per-IMU expert heads specialize in local dynamics.<n>Experiments show that GID enables accurate, real-time denoising from single-user training and generalizes across unseen users, motions, and garments.
  arXiv  Detail & Related papers  (2026-01-04T04:08:21Z)
• MotionTeller: Multi-modal Integration of Wearable Time-Series with LLMs for Health and Behavioral Understanding [4.158479111055355]
  MotionTeller is a generative framework that integrates minute-level wearable activity data with large language models (LLMs)<n>We construct a novel dataset of 54383 (actigraphy, text) pairs derived from real-world NHANES recordings, and train the model using cross-entropy loss with supervision only on the language tokens.<n>MotionTeller achieves high semantic fidelity (BERT-F1 = 0.924) and lexical accuracy (ROUGE-1 = 0.722), outperforming prompt-based baselines by 7 percent in ROUGE-1.
  arXiv  Detail & Related papers  (2025-12-25T04:37:07Z)
• Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Models [25.20942802233326]
  Garment Inertial Poser (GaIP) is a method for estimating full-body poses from sparse and loosely attached IMU sensors.<n>Our experiments show that our diffusion methods trained on simulated and synthetic data outperform state-of-the-art inertial full-body pose estimators.
  arXiv  Detail & Related papers  (2025-06-18T09:16:36Z)
• Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input [62.51283548975632]
  This work focuses on tracking and understanding human motion using consumer wearable devices, such as VR/AR headsets, smart glasses, cellphones, and smartwatches.<n>We present Ego4o (o for omni), a new framework for simultaneous human motion capture and understanding from multi-modal egocentric inputs.
  arXiv  Detail & Related papers  (2025-04-11T11:18:57Z)
• Suite-IN: Aggregating Motion Features from Apple Suite for Robust Inertial Navigation [10.634236058278722]
  Motion data captured by sensors on different body parts contains both local and global motion information. We propose a multi-device deep learning framework named Suite-IN, aggregating motion data from Apple Suite for inertial navigation.
  arXiv  Detail & Related papers  (2024-11-12T14:23:52Z)
• Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition [24.217068565936117]
  We present a novel method for action recognition that integrates motion data from body-worn IMUs with egocentric video. To model the complex relation of multiple IMU devices placed across the body, we exploit the collaborative dynamics in multiple IMU devices. Experiments show our method can achieve state-of-the-art performance on multiple public datasets.
  arXiv  Detail & Related papers  (2024-07-09T07:53:16Z)
• Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
  We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation. M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
  arXiv  Detail & Related papers  (2023-08-28T10:40:16Z)
• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars [80.05743236282564]
  Real-time tracking of human body motion is crucial for immersive experiences in AR/VR. We present a reinforcement learning framework that takes in sparse signals from an HMD and two controllers. We show that a single policy can be robust to diverse locomotion styles, different body sizes, and novel environments.
  arXiv  Detail & Related papers  (2022-09-20T00:25:54Z)
• MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
  We present MoDi, an unconditional generative model that synthesizes diverse motions. Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset. We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
  arXiv  Detail & Related papers  (2022-06-16T09:06:25Z)
• Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs [79.72586714047199]
  We propose an attention-based deep learning method to reconstruct full-body motion from six IMU sensors in real-time. Our method achieves new state-of-the-art results both quantitatively and qualitatively, while being simple to implement and smaller in size.
  arXiv  Detail & Related papers  (2022-03-29T16:24:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.