GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policy

• URL: http://arxiv.org/abs/2309.09051v3
• Date: Fri, 23 Feb 2024 07:51:31 GMT
• Title: GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policy
• Authors: So Kuroki, Jiaxian Guo, Tatsuya Matsushima, Takuya Okubo, Masato Kobayashi, Yuya Ikeda, Ryosuke Takanami, Paul Yoo, Yutaka Matsuo, Yusuke Iwasawa
• Abstract summary: We introduce GenDOM, a framework that allows the manipulation policy to handle different deformable objects with only a single real-world demonstration. At the time of inference, given a new object, GenDOM can estimate the deformable object parameters with only a single real-world demonstration. Empirical validations on both simulated and real-world object manipulation setups clearly show that our method can manipulate different objects with a single demonstration.
• Score: 23.72998685542652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the inherent uncertainty in their deformability during motion, previous methods in deformable object manipulation, such as rope and cloth, often required hundreds of real-world demonstrations to train a manipulation policy for each object, which hinders their applications in our ever-changing world. To address this issue, we introduce GenDOM, a framework that allows the manipulation policy to handle different deformable objects with only a single real-world demonstration. To achieve this, we augment the policy by conditioning it on deformable object parameters and training it with a diverse range of simulated deformable objects so that the policy can adjust actions based on different object parameters. At the time of inference, given a new object, GenDOM can estimate the deformable object parameters with only a single real-world demonstration by minimizing the disparity between the grid density of point clouds of real-world demonstrations and simulations in a differentiable physics simulator. Empirical validations on both simulated and real-world object manipulation setups clearly show that our method can manipulate different objects with a single demonstration and significantly outperforms the baseline in both environments (a 62% improvement for in-domain ropes and a 15% improvement for out-of-distribution ropes in simulation, as well as a 26% improvement for ropes and a 50% improvement for cloths in the real world), demonstrating the effectiveness of our approach in one-shot deformable object manipulation.

Related papers

• RPMArt: Towards Robust Perception and Manipulation for Articulated Objects [56.73978941406907]
  We propose a framework towards Robust Perception and Manipulation for Articulated Objects ( RPMArt) RPMArt learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. We introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-03-24T05:55:39Z)
• Learning Extrinsic Dexterity with Parameterized Manipulation Primitives [8.7221770019454]
  We learn a sequence of actions that utilize the environment to change the object's pose. Our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace.
  arXiv  Detail & Related papers  (2023-10-26T21:28:23Z)
• Learning Generalizable Manipulation Policies with Object-Centric 3D Representations [65.55352131167213]
  GROOT is an imitation learning method for learning robust policies with object-centric and 3D priors. It builds policies that generalize beyond their initial training conditions for vision-based manipulation. GROOT's performance excels in generalization over background changes, camera viewpoint shifts, and the presence of new object instances.
  arXiv  Detail & Related papers  (2023-10-22T18:51:45Z)
• Learning visual-based deformable object rearrangement with local graph neural networks [4.333220038316982]
  We propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We also propose a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments.
  arXiv  Detail & Related papers  (2023-10-16T11:42:54Z)
• GenORM: Generalizable One-shot Rope Manipulation with Parameter-Aware Policy [17.682208882809487]
  We introduce GenORM, a framework that allows the manipulation policy to handle different deformable ropes with a single real-world demonstration. At the time of inference, given a new rope, GenORM estimates the deformable rope parameters by minimizing the disparity between the grid density of point clouds of real-world demonstrations and simulations. Empirical validations on both simulated and real-world rope manipulation setups clearly show that our method can manipulate different ropes with a single demonstration.
  arXiv  Detail & Related papers  (2023-06-14T03:37:55Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation [135.10594078615952]
  We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects. A benchmark contains over 17,000 action trajectories with six types of plush toys and 78 variants. Our model achieves the best performance in geometry, correspondence, and dynamics predictions.
  arXiv  Detail & Related papers  (2022-03-14T04:56:55Z)
• Virtual Elastic Objects [18.228492027143307]
  We build virtual objects that behave like their real-world counterparts, even when subject to novel interactions. We use a differentiable, particle-based simulator to use deformation fields to find representative material parameters. We present our results using a dataset of 12 objects under a variety of force fields, which will be shared with the community.
  arXiv  Detail & Related papers  (2022-01-12T18:59:03Z)
• A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation [59.29922697476789]
  We propose a novel methodology for extracting state information from image sequences via a technique to represent the state of a deformable object as a distribution embedding. Our experiments confirm that we can estimate posterior distributions of physical properties, such as elasticity, friction and scale of highly deformable objects, such as cloth and ropes.
  arXiv  Detail & Related papers  (2021-12-09T17:50:54Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.