Differentiable Physics Simulation of Dynamics-Augmented Neural Objects
- URL: http://arxiv.org/abs/2210.09420v2
- Date: Thu, 20 Oct 2022 04:51:53 GMT
- Title: Differentiable Physics Simulation of Dynamics-Augmented Neural Objects
- Authors: Simon Le Cleac'h, Hong-Xing Yu, Michelle Guo, Taylor A. Howell, Ruohan
Gao, Jiajun Wu, Zachary Manchester, Mac Schwager
- Abstract summary: We present a differentiable pipeline for simulating the motion of objects that represent their geometry as a continuous density field parameterized as a deep network.
We estimate the dynamical properties of the object, including its mass, center of mass, and inertia matrix.
This allows a robot to autonomously build object models that are visually and dynamically accurate from still images and videos of objects in motion.
- Score: 40.587385809005355
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a differentiable pipeline for simulating the motion of objects
that represent their geometry as a continuous density field parameterized as a
deep network. This includes Neural Radiance Fields (NeRFs), and other related
models. From the density field, we estimate the dynamical properties of the
object, including its mass, center of mass, and inertia matrix. We then
introduce a differentiable contact model based on the density field for
computing normal and friction forces resulting from collisions. This allows a
robot to autonomously build object models that are visually and dynamically
accurate from still images and videos of objects in motion. The resulting
Dynamics-Augmented Neural Objects (DANOs) are simulated with an existing
differentiable simulation engine, Dojo, interacting with other standard
simulation objects, such as spheres, planes, and robots specified as URDFs. A
robot can use this simulation to optimize grasps and manipulation trajectories
of neural objects, or to improve the neural object models through
gradient-based real-to-simulation transfer. We demonstrate the pipeline to
learn the coefficient of friction of a bar of soap from a real video of the
soap sliding on a table. We also learn the coefficient of friction and mass of
a Stanford bunny through interactions with a Panda robot arm from synthetic
data, and we optimize trajectories in simulation for the Panda arm to push the
bunny to a goal location.
Related papers
- MonoForce: Learnable Image-conditioned Physics Engine [1.03590082373586]
We propose a novel model for the prediction of robot trajectories on rough offroad terrain from the onboard camera images.
The proposed hybrid model integrates a black-box component that predicts robot-terrain interaction forces with a neural-symbolic layer.
The differentiability, in conjunction with the rapid simulation speed, makes the model well-suited for various applications.
arXiv Detail & Related papers (2025-02-14T13:36:00Z) - GauSim: Registering Elastic Objects into Digital World by Gaussian Simulator [55.02281855589641]
GauSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.
We leverage continuum mechanics, modeling each kernel as a continuous piece of matter to account for realistic deformations without idealized assumptions.
GauSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
arXiv Detail & Related papers (2024-12-23T18:58:17Z) - Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materials [43.99845081513279]
This work introduces a novel Differentiable Physics-based System Identification (DPSI) framework that enables a robot arm to infer the physics parameters of elastoplastic materials and the environment using simple manipulation motions and incomplete 3D point clouds.
With only a single real-world interaction, the estimated parameters can accurately simulate visually and physically realistic behaviours induced by unseen and long-horizon manipulation motions.
arXiv Detail & Related papers (2024-11-01T13:04:25Z) - Physics-Encoded Graph Neural Networks for Deformation Prediction under
Contact [87.69278096528156]
In robotics, it's crucial to understand object deformation during tactile interactions.
We introduce a method using Physics-Encoded Graph Neural Networks (GNNs) for such predictions.
We've made our code and dataset public to advance research in robotic simulation and grasping.
arXiv Detail & Related papers (2024-02-05T19:21:52Z) - DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative
Diffusion Models [102.13968267347553]
We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks.
We showcase a range of simulated and fabricated robots along with their capabilities.
arXiv Detail & Related papers (2023-11-28T18:58:48Z) - DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via
Physics Simulation [81.11585774044848]
We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network.
Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
arXiv Detail & Related papers (2023-10-11T05:34:36Z) - Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videos [8.012771454339353]
We propose a novel approach for real-to-sim which tracks rigid objects in 3D from RGB-D images and infers physical properties of the objects.
We demonstrate and evaluate our approach on a real-world dataset.
arXiv Detail & Related papers (2023-09-27T14:46:01Z) - RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects
with Graph Networks [32.00371492516123]
We present a model-based planning framework for modeling and manipulating elasto-plastic objects.
Our system, RoboCraft, learns a particle-based dynamics model using graph neural networks (GNNs) to capture the structure of the underlying system.
We show through experiments that with just 10 minutes of real-world robotic interaction data, our robot can learn a dynamics model that can be used to synthesize control signals to deform elasto-plastic objects into various target shapes.
arXiv Detail & Related papers (2022-05-05T20:28:15Z) - Virtual Elastic Objects [18.228492027143307]
We build virtual objects that behave like their real-world counterparts, even when subject to novel interactions.
We use a differentiable, particle-based simulator to use deformation fields to find representative material parameters.
We present our results using a dataset of 12 objects under a variety of force fields, which will be shared with the community.
arXiv Detail & Related papers (2022-01-12T18:59:03Z) - Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform.
We produce a closed-loop controller to reactively push objects in a continuous action space.
We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
arXiv Detail & Related papers (2021-11-15T18:50:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.