Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling

• URL: http://arxiv.org/abs/2603.06218v1
• Date: Fri, 06 Mar 2026 12:32:56 GMT
• Title: Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling
• Abstract summary: We propose a few-shot real-to-sim approach that combines analytical formulations with the representational capacity of graph neural network (GNN)-based models.<n>Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions.<n> Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories.
• Score: 9.084754731100311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate physics simulation is essential for robotic learning and control, yet analytical simulators often fail to capture complex contact dynamics, while learning-based simulators typically require large amounts of costly real-world data. To bridge this gap, we propose a few-shot real-to-sim approach that combines the physical consistency of analytical formulations with the representational capacity of graph neural network (GNN)-based models. Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions. On this foundation, we introduce a mesh-based GNN that implicitly models rigid-body forward dynamics and derive surrogate gradients for collision detection, achieving full differentiability. Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories. In addition, the differentiable design supports gradient-based optimization, which we validate through simulation-based policy learning in multi-object interaction scenarios. Extensive experiments show that our framework not only improves simulation fidelity with minimal supervision but also increases the efficiency of policy learning. Taken together, these findings suggest that differentiable simulation with few-shot real-world grounding provides a powerful direction for advancing future robotic manipulation and control.

Related papers

• RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning [19.985714275376726]
  We introduce RLFTSim, a reinforcement-learning-based fine-tuning framework.<n>It enhances scenario realism by aligning simulator rollouts with real-world data distributions.<n>We also demonstrate the effectiveness of our approach for distilling traffic simulation controllability through goal conditioning.
  arXiv  Detail & Related papers  (2026-05-18T18:57:13Z)
• ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation [90.4702774169675]
  We propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation.<n>Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data.<n>We train a neural simulator to transform classical simulation videos into real-world representations, improving the accuracy of policy models trained in real-world environments.
  arXiv  Detail & Related papers  (2026-04-13T12:25:45Z)
• SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds [36.87909899472009]
  We introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world.<n>The system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via diffusion-based trajectory generation.<n>Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment.
  arXiv  Detail & Related papers  (2026-04-09T17:59:52Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• Contact-Aware Neural Dynamics [42.83872574010789]
  We propose an implicit sim-to-real framework that learns to align the simulator's dynamics with contact information.<n>We show that using tactile contact information from robotic hands can effectively model the non-smooth discontinuities inherent in contact-rich tasks.
  arXiv  Detail & Related papers  (2026-01-19T07:56:13Z)
• SimScale: Learning to Drive via Real-World Simulation at Scale [45.08991279559151]
  We introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs.<n>Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations.<n>We develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision.
  arXiv  Detail & Related papers  (2025-11-28T17:17:38Z)
• Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning [20.72669976554826]
  Learned Graph Network Simulators (GNSs) offer a promising alternative to traditional mesh-based physics simulators.<n>Their speed and inherent differentiability make them well suited for applications that require fast and accurate simulations, such as robotic manipulation or manufacturing optimization.<n>We utilize movement primitives to directly predict fast, stable and accurate simulations from a single model call.
  arXiv  Detail & Related papers  (2025-11-07T13:34:02Z)
• Simulation Priors for Data-Efficient Deep Learning [56.525770511247934]
  SimPEL is a method that efficiently combines first-principles models with data-driven learning.<n>We evaluate SimPEL on diverse systems, including biological, agricultural, and robotic domains.<n>For decision-making, we demonstrate that SimPEL bridges the sim-to-real gap in model-based reinforcement learning.
  arXiv  Detail & Related papers  (2025-09-06T14:36:41Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap [15.1962264049463]
  We present a differentiable simulation architecture for articulated rigid-body dynamics that enables the augmentation of analytical models with neural networks at any point of the computation.
  arXiv  Detail & Related papers  (2020-07-12T17:27:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.