HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

• URL: http://arxiv.org/abs/2607.25895v1
• Date: Tue, 28 Jul 2026 15:52:02 GMT
• Title: HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone
• Abstract summary: We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, and field of view.<n>We demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot.<n>We open-source HiFi-UMI-2K, 2,000 hours of micro-synchronized, ultra-wide-FoV demonstrations as a large-scale, high-fidelity resource for the robot-learning community.
• Score: 19.886495773199023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning deployable manipulation policies is bottlenecked by the scarcity of data that is both high-fidelity and scalable. Real-robot teleoperation is accurate but costly to scale; robot-free UMI capture scales readily, and current practice uses the resulting data mainly for pre-training, adding a small real-robot "anchor" at post-training. We ask whether raising the fidelity of robot-free UMI data, rather than shrinking the real-robot fraction, can remove that anchor. We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, synchronization, and field of view: head-mounted offline stereo-inertial SLAM, native rather than reconstructed relative pose, a shared microsecond GPIO trigger, and two wide-angle cameras per hand covering ~200 degrees. It reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure. Using this corpus, we demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches in-domain teleoperation across three backbones spanning the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on StarVLA-QwenPI, OpenPI-pi_0.5, and LingBot-VA; the strongest policy reaches 85% on a precision insertion task, even though the teleoperation baseline is collected in the evaluation scene and no HiFi-UMI trajectory is. Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41% and, on StarVLA-QwenPI, raises real-robot success by a further 18.1 percentage points. We open-source HiFi-UMI-2K, 2,000 hours of microsecond-synchronized, ultra-wide-FoV demonstrations, each automatically reconstructed and validated through simulation replay, as a large-scale, high-fidelity resource for the robot-learning community.

Related papers

• The Robot Data Factory [35.46679263015648]
  We introduce the Robot Data Factory (RDF), a mission-driven infrastructure for continuously generating, validating, benchmarking, and reusing experience.<n>RDF organizes heterogeneous robots and environment-specific training grounds through reproducible missions, skill curricula, synchronized multimodal sensing, external ground truth, an agentic robot network, data pipelines, and living benchmarks.<n>We further formalize robot experience and its quality, introduce a mission-task-skill-episode-dataset-benchmark-capability hierarchy, and derive quantitative scaling laws and an algorithmic procedure connecting robot fleet size, sensor rates, storage, learning representations, tokenization, training compute
  arXiv  Detail & Related papers  (2026-09-15T06:30:11Z)
• Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots [5.3871099789895665]
  Reinforcement learning (RL) has shown considerable promise for robotic decision-making, yet deploying multi-agent RL (MARL) on physical multi-robot systems in industrial environments remains challenging.<n>We propose Feature-fusion Multi-Agent Proximal Policy Optimization (FMAPPO), which fuses 2D LiDAR measurements with task-specific state information to enable safe decentralized multi-robot task assignment and navigation.
  arXiv  Detail & Related papers  (2026-09-13T14:53:42Z)
• Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization [32.62028759076795]
  AcoustoBots is a mobile acoustophoretic data-physicalization platform.<n>TurtleBot3 robots carry upward-facing 8 x 8 ultrasonic phased arrays.<n>Each array levitates a particle whose height encodes a local urban scalar value, such as population density, noise, or traffic.
  arXiv  Detail & Related papers  (2026-07-07T17:59:44Z)
• From Foundation to Application: Improving VLA Models in Practice [62.1552185059214]
  We present LingBot-VLA 2.0, which advances LingBot-VLA through improvements in three functional domains.<n>Compared to the previous version, we revamp the data processing pipeline and curate around 60,000 hours of data for pretraining.<n>In particular, our system accommodates degrees of freedom for the heads, waists, mobile bases, and dexterous hands, thereby empowering the robots to tackle more complex tasks in practical scenarios.
  arXiv  Detail & Related papers  (2026-07-07T15:33:12Z)
• Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models [95.75234389806654]
  We present Qwen-RobotManip, a generalizable Vision-Language-Action foundation model built on Qwen-VL.<n>Qwen-RobotManip introduces a unified alignment framework across the representation, motion, and behavioral dimensions of manipulation.<n>A human-to-robot synthesis pipeline converts egocentric hand demonstrations into robot trajectories across 15 platforms.
  arXiv  Detail & Related papers  (2026-06-16T12:14:39Z)
• XRZero-G0: Pushing the Frontier of Dexterous Robotic Manipulation with Interfaces, Quality and Ratios [10.349987856533723]
  XRZero-G0 is a hardware-software co-designed system for embodied data collection and policy learning.<n>We construct a 2,000-hour robot-free dataset that enables zero-shot cross-embodiment transfer to a target physical robot.
  arXiv  Detail & Related papers  (2026-04-14T17:34:21Z)
• MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation [56.976138997544005]
  We show that zero-shot transfer to the real world is possible, but effective for both static and mobile manipulation.<n>We release MolmoBot-Data, a dataset of 1.8 million expert trajectories for articulated object manipulation and pick-and-place tasks.<n>Our results demonstrate that procedural environment generation combined with diverse articulated assets can produce robust manipulation policies.
  arXiv  Detail & Related papers  (2026-03-17T17:59:03Z)
• InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation [77.07565723756119]
  We present InternVLA-A1, a vision-language-action model with dynamic predictive capabilities.<n>We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos.<n>We evaluate InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark.
  arXiv  Detail & Related papers  (2026-01-05T18:54:29Z)
• InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy [138.89177083578213]
  We introduce InternVLA-M1, a unified framework for spatial grounding and robot control.<n>InternVLA-M1 employs a two-stage pipeline: (i) spatial grounding pre-training on over 2.3M spatial reasoning data, and (ii) spatially guided action post-training.<n>Results: InternVLA-M1 outperforms its variant without spatial guidance by +14.6% on SimplerEnv Google Robot, +17% on WidowX, and +4.3% on LIBERO Franka.
  arXiv  Detail & Related papers  (2025-10-15T17:30:05Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.