OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation

• URL: http://arxiv.org/abs/2606.08548v1
• Date: Sun, 07 Jun 2026 10:01:53 GMT
• Title: OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation
• Abstract summary: OASIS is a simulation-data-driven framework for humanoid loco-manipulation.<n>It reconstructs realistic object assets from real-world images using a 3D generative model.<n>Under zero-shot deployment, it achieves higher success rates on most tasks than that trained on real-robot teleoperation data.
• Score: 51.37436817948304
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent progress in robot manipulation has been largely driven by learning from large-scale demonstrations. For humanoid robot loco-manipulation tasks, however, existing data sources force an unsatisfying tradeoff between trajectory quality and scalability. Real-world teleoperation provides the highest-quality trajectories but requires dedicated physical space and time-consuming scene resets. Simulation offers an alternative way out of this dilemma: it can produce clean, embodiment-aligned data at scale without any physical hardware. In this paper, we propose OASIS, a simulation-data-driven framework for humanoid loco-manipulation. OASIS automatically reconstructs realistic object assets from real-world images using a 3D generative model. Based on these assets, trajectories are first collected through teleoperation in simulation, and then augmented under diverse domain randomizations in a post-processing stage. With the resulting simulation data, we further design a hierarchical visuomotor policy for humanoid loco-manipulation. Extensive experiments on the real humanoid robot show that, under zero-shot deployment, the policy trained on our simulation data achieves higher success rates on most tasks than that trained on real-robot teleoperation data, owing largely to the broad lighting and environmental variations covered by our simulation rendering, which real-robot data fails to capture. The project page is available at https://oasis-humanoid.github.io/.

Related papers

• Pre-training Visual Dexterity in Simulation [67.17477115017896]
  We introduce a pre-training framework for dexterous manipulation that uses data entirely collected in simulation.<n>With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective.<n>We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation.
  arXiv  Detail & Related papers  (2026-08-16T20:21:33Z)
• DataLadder: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid [63.45560198934907]
  Generalist robot policies require trustworthy evaluation and robot-usable training data.<n>We present DataLadder, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation.<n>We package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud.
  arXiv  Detail & Related papers  (2026-06-15T14:21:35Z)
• SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation [33.50539650112054]
  We present SIMPLE, a unified simulation testbed for humanoid policy learning and evaluation.<n> SIMPLE couples the accurate contact-rich dynamics of MuJoCo with the rendering of IsaacSim.<n>It provides a large-scale environment comprising 60 diverse whole-body tasks, 50 indoor scenes, and over 1,000 object assets.
  arXiv  Detail & Related papers  (2026-06-06T17:55:43Z)
• ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation [90.4702774169675]
  We propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation.<n>Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data.<n>We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments.
  arXiv  Detail & Related papers  (2026-04-13T12:25:45Z)
• SoftMimicGen: A Data Generation System for Scalable Robot Learning in Deformable Object Manipulation [45.71119158785489]
  We introduce SoftMimicGen, an automated data generation pipeline for deformable object manipulation tasks.<n>We introduce a suite of high-fidelity simulation environments that encompasses a wide range of deformable objects.<n>We apply SoftMimicGen to generate datasets across the task suite, train high-performing policies from the data, and systematically analyze the data generation system.
  arXiv  Detail & Related papers  (2026-03-26T17:58:40Z)
• Sim-and-Human Co-training for Data-Efficient and Generalizable Robotic Manipulation [113.13282853889818]
  SimHum is a framework to simultaneously extract kinematic prior from simulated robot actions and visual prior from real-world human observations.<n>Based on the two complementary priors, we achieve data-efficient and generalizable robotic manipulation in real-world tasks.
  arXiv  Detail & Related papers  (2026-01-27T09:41:28Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• RoboPearls: Editable Video Simulation for Robot Manipulation [81.18434338506621]
  RoboPearls is an editable video simulation framework for robotic manipulation.<n>Built on 3D Gaussian Splatting (3DGS), RoboPearls enables the construction of photo-realistic, view-consistent simulations.<n>We conduct extensive experiments on multiple datasets and scenes, including RLBench, COLOSSEUM, Ego4D, Open X-Embodiment, and a real-world robot.
  arXiv  Detail & Related papers  (2025-06-28T05:03:31Z)
• Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation [40.96453435496208]
  We present a recipe for utilizing simulation data to solve vision-based robotic manipulation tasks.<n>Using two domains--a robot arm and a humanoid--we demonstrate that simulation data can enhance real-world task performance by an average of 38%.
  arXiv  Detail & Related papers  (2025-03-31T17:39:38Z)
• Robot Learning with Super-Linear Scaling [13.053949644385932]
  CASHER is a pipeline for scaling up data collection and learning in simulation where the performance scales superlinearly with human effort.<n>We show that CASHER enables fine-tuning of pre-trained policies to a target scenario using a video scan without any additional human effort.
  arXiv  Detail & Related papers  (2024-12-02T18:12:02Z)
• RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots [25.650235551519952]
  We present RoboCasa, a large-scale simulation framework for training generalist robots in everyday environments. We provide thousands of 3D assets across over 150 object categories and dozens of interactable furniture and appliances. Our experiments show a clear scaling trend in using synthetically generated robot data for large-scale imitation learning.
  arXiv  Detail & Related papers  (2024-06-04T17:41:31Z)
• Reactive Long Horizon Task Execution via Visual Skill and Precondition Models [59.76233967614774]
  We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner. We show an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines.
  arXiv  Detail & Related papers  (2020-11-17T15:24:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.