ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
- URL: http://arxiv.org/abs/2604.11386v1
- Date: Mon, 13 Apr 2026 12:25:45 GMT
- Title: ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
- Abstract summary: We propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation.<n>Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data.<n>We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments.
- Score: 90.4702774169675
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advancements in foundational models, such as large language models and world models, have greatly enhanced the capabilities of robotics, enabling robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation to generate accurate action-video pairs while maintaining real-world consistency. Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data to generate diverse, large-scale training datasets that cover a broader spectrum of real-world scenarios. We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments. Through extensive experiments, we demonstrate that our method significantly reduces the sim2real domain gap, resulting in higher success rates in real-world policy model training. Our approach offers a scalable solution for generating robust training data and bridging the gap between simulated and real-world robotics.
Related papers
- OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation [51.37436817948304]
OASIS is a simulation-data-driven framework for humanoid loco-manipulation.<n>It reconstructs realistic object assets from real-world images using a 3D generative model.<n>Under zero-shot deployment, it achieves higher success rates on most tasks than that trained on real-robot teleoperation data.
arXiv Detail & Related papers (2026-06-07T10:01:53Z) - From Seeing to Simulating: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation [11.382656265384854]
We present a generative framework that establishes a generative real-to-sim mapping from real-world panoramas to high-fidelity simulation scenes.<n>Combined with high-quality physics engines and realistic assets, the generated scenes support interactive manipulation tasks.<n>Experiments demonstrate a strong sim-to-real correlation validating our platform's fidelity, and show that extensively scaling up data generation leads to significantly better generalization to unseen scene and object variations.
arXiv Detail & Related papers (2026-04-17T08:06:26Z) - Interactive World Simulator for Robot Policy Training and Evaluation [21.481187472784047]
We present Interactive World Simulator, a framework for building interactive world models from a robot interaction dataset.<n>In our experiments, the learned world models produce interaction-consistent pixel-level predictions.<n>We find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data.
arXiv Detail & Related papers (2026-03-09T16:13:32Z) - Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
arXiv Detail & Related papers (2026-01-24T07:43:57Z) - PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies [88.78188489161028]
We introduce Policy Evaluation and Environment Reconstruction in Simulation (PolaRiS)<n>PolaRiS is a scalable real-to-sim framework for high-fidelity simulated robot evaluation.<n>We show that PolaRiS evaluations provide a much stronger correlation to real world generalist policy performance than existing simulated benchmarks.
arXiv Detail & Related papers (2025-12-18T18:49:41Z) - The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio [138.07247714782412]
MultiGen is a framework that integrates large-scale generative models into traditional physics simulators.<n>We demonstrate effective zero-shot transfer to real-world pouring with novel containers and liquids.
arXiv Detail & Related papers (2025-07-03T17:59:58Z) - Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation [40.96453435496208]
We present a recipe for utilizing simulation data to solve vision-based robotic manipulation tasks.<n>Using two domains--a robot arm and a humanoid--we demonstrate that simulation data can enhance real-world task performance by an average of 38%.
arXiv Detail & Related papers (2025-03-31T17:39:38Z) - Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
arXiv Detail & Related papers (2025-01-17T10:39:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.