A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation

• URL: http://arxiv.org/abs/2112.05068v1
• Date: Thu, 9 Dec 2021 17:50:54 GMT
• Title: A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation
• Authors: Rika Antonova, Jingyun Yang, Priya Sundaresan, Dieter Fox, Fabio Ramos, Jeannette Bohg
• Abstract summary: We propose a novel methodology for extracting state information from image sequences via a technique to represent the state of a deformable object as a distribution embedding. Our experiments confirm that we can estimate posterior distributions of physical properties, such as elasticity, friction and scale of highly deformable objects, such as cloth and ropes.
• Score: 59.29922697476789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deformable object manipulation remains a challenging task in robotics research. Conventional techniques for parameter inference and state estimation typically rely on a precise definition of the state space and its dynamics. While this is appropriate for rigid objects and robot states, it is challenging to define the state space of a deformable object and how it evolves in time. In this work, we pose the problem of inferring physical parameters of deformable objects as a probabilistic inference task defined with a simulator. We propose a novel methodology for extracting state information from image sequences via a technique to represent the state of a deformable object as a distribution embedding. This allows to incorporate noisy state observations directly into modern Bayesian simulation-based inference tools in a principled manner. Our experiments confirm that we can estimate posterior distributions of physical properties, such as elasticity, friction and scale of highly deformable objects, such as cloth and ropes. Overall, our method addresses the real-to-sim problem probabilistically and helps to better represent the evolution of the state of deformable objects.

Related papers

• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policy [23.72998685542652]
  We introduce GenDOM, a framework that allows the manipulation policy to handle different deformable objects with only a single real-world demonstration. At the time of inference, given a new object, GenDOM can estimate the deformable object parameters with only a single real-world demonstration. Empirical validations on both simulated and real-world object manipulation setups clearly show that our method can manipulate different objects with a single demonstration.
  arXiv  Detail & Related papers  (2023-09-16T17:18:23Z)
• ShapeShift: Superquadric-based Object Pose Estimation for Robotic Grasping [85.38689479346276]
  Current techniques heavily rely on a reference 3D object, limiting their generalizability and making it expensive to expand to new object categories. This paper proposes ShapeShift, a superquadric-based framework for object pose estimation that predicts the object's pose relative to a primitive shape which is fitted to the object.
  arXiv  Detail & Related papers  (2023-04-10T20:55:41Z)
• Predicting Physical Object Properties from Video [28.19031441659854]
  We present a novel approach to estimating physical properties of objects from video. Our approach consists of a physics engine and a correction estimator. We demonstrate faster and more robust convergence of the learned method in several simulated 2D scenarios.
  arXiv  Detail & Related papers  (2022-06-02T08:46:22Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)
• Sequential Topological Representations for Predictive Models of Deformable Objects [18.190326379178995]
  We construct compact topological representations to capture the state of highly deformable objects. We develop an approach that tracks the evolution of this topological state through time. Our experiments with highly deformable objects in simulation show that the proposed multistep predictive models yield more precise results than those obtained from computational topology libraries.
  arXiv  Detail & Related papers  (2020-11-23T19:45:15Z)
• Identifying Mechanical Models through Differentiable Simulations [16.86640234046472]
  This paper proposes a new method for manipulating unknown objects through a sequence of non-prehensile actions. The proposed method leverages recent progress in differentiable physics models to identify unknown mechanical properties of manipulated objects.
  arXiv  Detail & Related papers  (2020-05-11T20:19:20Z)
• Visual Grounding of Learned Physical Models [66.04898704928517]
  Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors. Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
  arXiv  Detail & Related papers  (2020-04-28T17:06:38Z)
• Predicting the Physical Dynamics of Unseen 3D Objects [65.49291702488436]
  We focus on predicting the dynamics of 3D objects on a plane that have just been subjected to an impulsive force. Our approach can generalize to object shapes and initial conditions that were unseen during training. Our model can support training with data from both a physics engine or the real world.
  arXiv  Detail & Related papers  (2020-01-16T06:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.