ActiveMimic: Egocentric Video Pretraining with Active Perception

• URL: http://arxiv.org/abs/2606.06194v1
• Date: Thu, 04 Jun 2026 14:01:01 GMT
• Title: ActiveMimic: Egocentric Video Pretraining with Active Perception
• Abstract summary: Egocentric human video offers a scalable alternative to robot data for pretraining.<n>We present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera.<n>We show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data.
• Score: 84.2999803878421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.

Related papers

• Co-training with Ego-centric Video and Demonstration for Robot Navigation Task [0.0]
  We propose a framework that converts egocentric walking videos into datasets for mobile robot imitation learning.<n>The proposed method estimates camera motion from human videos and transforms it into action representations compatible with ground mobile robots.<n> Experiments on a fruit-search navigation task demonstrate that human egocentric videos provide an effective and scalable data source for mobile robot learning.
  arXiv  Detail & Related papers  (2026-06-01T09:12:22Z)
• ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstration [51.69384671495837]
  ActiveGlasses is a system for learning robot manipulation from ego-centric human demonstrations with active vision.<n>A stereo camera mounted on smart glasses serves as the sole perception device for both data collection and policy inference.<n>To enable zero-transfer, we extract object trajectories from demonstrations and use an object-centric point-cloud policy to jointly predict manipulation and head movement.
  arXiv  Detail & Related papers  (2026-04-09T17:59:08Z)
• EgoAVFlow: Robot Policy Learning with Active Vision from Human Egocentric Videos via 3D Flow [17.993528705818008]
  EgoAVFlow learns manipulation and active vision from egocentric videos through a shared 3D flow representation.<n>We show that EgoAVFlow consistently outperforms prior human-demo-based baselines.
  arXiv  Detail & Related papers  (2026-02-25T22:50:51Z)
• Mitty: Diffusion-based Human-to-Robot Video Generation [57.494785199352975]
  We present Mitty, a Diffusion Transformer that enables video In-Context Learning for end-to-end Human2Robot video generation.<n>Built on a pretrained video diffusion model, Mitty leverages strong visual-temporal priors to translate human demonstrations into robot-execution videos without action labels or intermediate abstractions.<n> Experiments on Human2Robot and EPIC-Kitchens show that Mitty delivers state-of-the-art results, strong generalization to unseen environments, and new insights for scalable robot learning from human observations.
  arXiv  Detail & Related papers  (2025-12-19T05:52:15Z)
• MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training [40.45924128424013]
  We propose MimicDreamer, a framework that turns low-cost human demonstrations into robot-usable supervision.<n>For visual alignment, we propose H2R Aligner, a video diffusion model that generates high-fidelity robot demonstration videos.<n>For viewpoint stabilization, EgoStabilizer is proposed, which canonicalizes egocentric videos via homography.<n>For action alignment, we map human hand trajectories to the robot frame and apply a constrained inverse kinematics solver.
  arXiv  Detail & Related papers  (2025-09-26T11:05:10Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• AR-VRM: Imitating Human Motions for Visual Robot Manipulation with Analogical Reasoning [5.371855090716962]
  Visual Robot Manipulation (VRM) aims to enable a robot to follow natural language instructions based on robot states and visual observations.<n>Existing approaches have employed vision-language pretraining with large-scale data.<n>We propose to learn from large-scale human action video datasets in an explicit way.
  arXiv  Detail & Related papers  (2025-08-11T05:09:58Z)
• EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos [49.820119587446655]
  In this paper, we explore training Vision-Language-Action (VLA) models using egocentric human videos.<n>With a VLA trained on human video that predicts human wrist and hand actions, we can perform Inverse Kinematics and convert the human actions to robot actions.<n>We propose a simulation benchmark called Ego Humanoid Manipulation Benchmark, where we design diverse bimanual manipulation tasks with demonstrations.
  arXiv  Detail & Related papers  (2025-07-16T17:27:44Z)
• Learning Video-Conditioned Policies for Unseen Manipulation Tasks [83.2240629060453]
  Video-conditioned Policy learning maps human demonstrations of previously unseen tasks to robot manipulation skills. We learn our policy to generate appropriate actions given current scene observations and a video of the target task. We validate our approach on a set of challenging multi-task robot manipulation environments and outperform state of the art.
  arXiv  Detail & Related papers  (2023-05-10T16:25:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.