VENOM: Versatile Embodied Network for Omni-bodied Motion tracking

• URL: http://arxiv.org/abs/2606.16696v1
• Date: Mon, 15 Jun 2026 13:31:37 GMT
• Title: VENOM: Versatile Embodied Network for Omni-bodied Motion tracking
• Abstract summary: VENOM is a cross-embodiment full-body motion tracking model for humanoids in simulation.<n>It is trained on multiple humanoid data that can track the entire body without the requirement to split into upper and lower body control.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving expert-level expressive full-body motion tracking across multiple humanoids solely from demonstration data remains a challenging and relatively an underexplored problem in humanoid robot learning. Cross-embodiment motion tracking policies are mostly trained by decoupling the control problem into upper and lower body control. This work proposes VENOM, a cross-embodiment full-body motion tracking model for humanoids in simulation. VENOM is a GPT-based motion tracker trained on multiple humanoid data that can track the entire body without the requirement to split into upper and lower body control. We curate a multi-humanoid motion tracking dataset called the VENOM dataset that contains states, actions, and rewards and train VENOM and the baselines on this dataset. In this letter, we evaluate VENOM's performance against baselines and show that we can achieve a stable motion tracker across different humanoids more capable than an MLP trained on multiple humanoid data with supervised learning alone, and also show that despite lack of reward feedback, VENOM closely matches the tracking capability of experts that were trained using asymmetric-actor critic reinforcement learning.

Related papers

• HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  We propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection.<n>We use lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions.<n>Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2026-06-25T16:23:56Z)
• BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  BifrostUMI is a portable, efficient, and robot-free data collection framework for humanoid robots.<n>We demonstrate the efficacy and versatility of the proposed framework across two distinct experimental scenarios.
  arXiv  Detail & Related papers  (2026-05-05T07:35:09Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control [85.91101551600978]
  We show that scaling up model capacity, data, and compute yields a generalist humanoid controller capable of creating natural and robust whole-body movements.<n>We build a foundation model for motion tracking by scaling along three axes: network size, dataset volume, and compute.<n>We show the practical utility of our model through two mechanisms: (1) a real-time universal kinematic planner that bridges motion tracking to downstream task execution, enabling natural and interactive control, and (2) a unified token space that supports various motion input interfaces.
  arXiv  Detail & Related papers  (2025-11-11T04:37:40Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• Learning to Track Any Points from Human Motion [55.831218129679144]
  We propose an automated pipeline to generate pseudo-labeled training data for point tracking.<n>A point tracking model trained on AnthroTAP achieves annotated state-of-the-art performance on the TAP-Vid benchmark.
  arXiv  Detail & Related papers  (2025-07-08T17:59:58Z)
• ExBody2: Advanced Expressive Humanoid Whole-Body Control [16.69009772546575]
  We propose a method for producing whole-body tracking controllers that are trained on both human motion capture and simulated data.<n>We use a teacher policy to produce intermediate data that better conforms to the robot's kinematics.<n>We observed significant improvement of tracking performance after fine-tuning on a small amount of data.
  arXiv  Detail & Related papers  (2024-12-17T18:59:51Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots [13.229028132036321]
  Masked Humanoid Controller (MHC) supports standing, walking, and mimicry of whole and partial-body motions.<n>MHC imitates partially masked motions from a library of behaviors spanning standing, walking, optimized reference trajectories, re-targeted video clips, and human motion capture data.<n>We demonstrate sim-to-real transfer on the real-world Digit V3 humanoid robot.
  arXiv  Detail & Related papers  (2024-07-30T09:10:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.