SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds

• URL: http://arxiv.org/abs/2604.08544v2
• Date: Fri, 10 Apr 2026 10:13:54 GMT
• Title: SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds
• Abstract summary: We introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world.<n>The system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via diffusion-based trajectory generation.<n>Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment.
• Score: 36.87909899472009
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Robotic manipulation with deformable objects represents a data-intensive regime in embodied learning, where shape, contact, and topology co-evolve in ways that far exceed the variability of rigids. Although simulation promises relief from the cost of real-world data acquisition, prevailing sim-to-real pipelines remain rooted in rigid-body abstractions, producing mismatched geometry, fragile soft dynamics, and motion primitives poorly suited for cloth interaction. We posit that simulation fails not for being synthetic, but for being ungrounded. To address this, we introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world. Given limited demonstrations, the system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via diffusion-based trajectory generation with quality filtering. This pipeline transforms sparse observations into scaled synthetic supervision with near-demonstration fidelity. Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment. These results validate physics-aligned simulation as scalable supervision for deformable manipulation and a practical pathway for data-efficient policy learning.

Related papers

• GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models [66.77745044586173]
  GAUGE is a real-world-grounded diagnostic benchmark for evaluating how numerical simulators and generative video world models reproduce or deviate from real-world physics.<n>We benchmark Isaac Sim, Genesis, and Newton on 14 task families using generalized trajectory errors, and evaluate 6 image-to-video models on 5 rigid-body tasks.<n>Our results reveal no uniformly faithful physics engine, with the largest discrepancies arising in impulsive contact, rapid textile motion, and volumetric deformation.
  arXiv  Detail & Related papers  (2026-08-06T12:19:38Z)
• DeformX: A Versatile Co-Simulation Framework for Deformable Linear Objects [30.23545134439942]
  DeformX is a co-simulation framework that integrates a dedicated Cosserat rod physics engine with NVIDIA Isaac Sim.<n>Our Cosserat rod engine simulates the dynamics and self-collisions of DLOs, and contact interactions with arbitrary free-form meshes.<n>DeformX is the one of the first frameworks for DLO simulation that unifies realistic visualization, principled physics, and compatibility with robot learning pipelines.
  arXiv  Detail & Related papers  (2026-06-20T15:57:29Z)
• Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes [4.759823735082845]
  Real2Sim is an unified framework that combines 4D Gaussian Splatting (4DGS) with a differentiable Material Point Method (MPM) solver.<n>Real2Sim explicitly reconstructs dynamic driving scenes as temporally continuous Gaussian primitives, supports instance-level editing, and simulates realistic object-object and object-environment interactions.<n> Experiments on the Open dataset validate Real2Sim's capabilities in rendering, reconstruction, editing, and physics simulation, demonstrating its potential as a scalable tool for data generation in downstream tasks such as perception, tracking, trajectory prediction, and end-to-end policy learning.
  arXiv  Detail & Related papers  (2026-05-13T14:26:25Z)
• ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation [90.4702774169675]
  We propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation.<n>Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data.<n>We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments.
  arXiv  Detail & Related papers  (2026-04-13T12:25:45Z)
• Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling [9.084754731100311]
  We propose a few-shot real-to-sim approach that combines analytical formulations with the representational capacity of graph neural network (GNN)-based models.<n>Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions.<n> Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories.
  arXiv  Detail & Related papers  (2026-03-06T12:32:56Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• Contact-Aware Neural Dynamics [42.83872574010789]
  We propose an implicit sim-to-real framework that learns to align the simulator's dynamics with contact information.<n>We show that using tactile contact information from robotic hands can effectively model the non-smooth discontinuities inherent in contact-rich tasks.
  arXiv  Detail & Related papers  (2026-01-19T07:56:13Z)
• RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data [25.53943767088309]
  We introduce RealPDEBench, the first benchmark for scientific Machine Learning (ML) that integrates real-world measurements with paired numerical simulations.<n>RealPDEBench consists of five datasets, three tasks, eight metrics, and ten baselines.<n> Experiments reveal significant discrepancies between simulated and real-world data, while showing that pretraining with simulated data consistently improves both accuracy and convergence.
  arXiv  Detail & Related papers  (2026-01-05T06:49:13Z)
• SimScale: Learning to Drive via Real-World Simulation at Scale [45.08991279559151]
  We introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs.<n>Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations.<n>We develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision.
  arXiv  Detail & Related papers  (2025-11-28T17:17:38Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• Inferring Articulated Rigid Body Dynamics from RGBD Video [18.154013621342266]
  We introduce a pipeline that combines inverse rendering with differentiable simulation to create digital twins of real-world articulated mechanisms. Our approach accurately reconstructs the kinematic tree of an articulated mechanism being manipulated by a robot.
  arXiv  Detail & Related papers  (2022-03-20T08:19:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.