PlaNet-ClothPick: Effective Fabric Flattening Based on Latent Dynamic Planning

• URL: http://arxiv.org/abs/2303.01345v2
• Date: Tue, 28 Nov 2023 12:22:30 GMT
• Title: PlaNet-ClothPick: Effective Fabric Flattening Based on Latent Dynamic Planning
• Authors: Halid Abdulrahim Kadi and Kasim Terzic
• Abstract summary: Recent work has attributed this to the blurry prediction of the observation, which makes it difficult to plan directly in the latent space. We find that the sharp discontinuity of the transition function on the contour of the fabric makes it difficult to learn an accurate latent dynamic model. Our model exhibits a faster action inference and requires fewer transitional model parameters than the state-of-the-art robotic systems in this domain.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Why do Recurrent State Space Models such as PlaNet fail at cloth manipulation tasks? Recent work has attributed this to the blurry prediction of the observation, which makes it difficult to plan directly in the latent space. This paper explores the reasons behind this by applying PlaNet in the pick-and-place fabric-flattening domain. We find that the sharp discontinuity of the transition function on the contour of the fabric makes it difficult to learn an accurate latent dynamic model, causing the MPC planner to produce pick actions slightly outside of the article. By limiting picking space on the cloth mask and training on specially engineered trajectories, our mesh-free PlaNet-ClothPick surpasses visual planning and policy learning methods on principal metrics in simulation, achieving similar performance as state-of-the-art mesh-based planning approaches. Notably, our model exhibits a faster action inference and requires fewer transitional model parameters than the state-of-the-art robotic systems in this domain. Other supplementary materials are available at: https://sites.google.com/view/planet-clothpick.

Related papers

• Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories [28.701879490459675]
  We aim to learn an implicit motion field parameterized by a neural network to predict the movement of novel points within same domain. We exploit intrinsic regularization provided by SIREN, and modify the input layer to produce atemporally smooth motion field. Our experiments assess the model's performance in predicting unseen point trajectories and its application in temporal mesh alignment with deformation.
  arXiv  Detail & Related papers  (2024-06-05T21:02:10Z)
• A New View on Planning in Online Reinforcement Learning [19.35031543927374]
  This paper investigates a new approach to model-based reinforcement learning using background planning. We show that our GSP algorithm can propagate value from an abstract space in a manner that helps a variety of base learners learn significantly faster in different domains.
  arXiv  Detail & Related papers  (2024-06-03T17:45:19Z)
• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)
• Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking [18.288330275993328]
  We propose a self-supervised method to finetune a mesh reconstruction model in the real world. We show that we can improve the quality of the reconstructed mesh without requiring human annotations.
  arXiv  Detail & Related papers  (2023-02-19T07:48:12Z)
• Goal-Space Planning with Subgoal Models [18.43265820052893]
  This paper investigates a new approach to model-based reinforcement learning using background planning. We show that our GSP algorithm can propagate value from an abstract space in a manner that helps a variety of base learners learn significantly faster in different domains.
  arXiv  Detail & Related papers  (2022-06-06T20:59:07Z)
• Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation [18.288330275993328]
  Self-occlusion is challenging for cloth manipulation, as it makes it difficult to estimate the full state of the cloth. We leverage recent advances in pose estimation for cloth to build a system that uses explicit occlusion reasoning to unfold a crumpled cloth.
  arXiv  Detail & Related papers  (2022-06-06T20:15:02Z)
• Visual Learning-based Planning for Continuous High-Dimensional POMDPs [81.16442127503517]
  Visual Tree Search (VTS) is a learning and planning procedure that combines generative models learned offline with online model-based POMDP planning. VTS bridges offline model training and online planning by utilizing a set of deep generative observation models to predict and evaluate the likelihood of image observations in a Monte Carlo tree search planner. We show that VTS is robust to different observation noises and, since it utilizes online, model-based planning, can adapt to different reward structures without the need to re-train.
  arXiv  Detail & Related papers  (2021-12-17T11:53:31Z)
• N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks [69.94313958962165]
  We present a novel mesh-based learning approach (N-Cloth) for plausible 3D cloth deformation prediction. We use graph convolution to transform the cloth and object meshes into a latent space to reduce the non-linearity in the mesh space. Our approach can handle complex cloth meshes with up to $100$K triangles and scenes with various objects corresponding to SMPL humans, Non-SMPL humans, or rigid bodies.
  arXiv  Detail & Related papers  (2021-12-13T03:13:11Z)
• Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning [51.74463056899926]
  This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene. We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations.
  arXiv  Detail & Related papers  (2021-10-02T12:36:58Z)
• NRST: Non-rigid Surface Tracking from Monocular Video [97.2743051142748]
  We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-rigidly to each frame. Results demonstrate the effectiveness of our method on both general textured non-rigid objects and monochromatic fabrics.
  arXiv  Detail & Related papers  (2021-07-06T06:06:45Z)
• Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation [74.88956115580388]
  Planning is performed in a low-dimensional latent state space that embeds images. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them.
  arXiv  Detail & Related papers  (2020-03-19T18:43:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.