Robotic Fabric Flattening with Wrinkle Direction Detection

• URL: http://arxiv.org/abs/2303.04909v3
• Date: Thu, 26 Oct 2023 10:53:03 GMT
• Title: Robotic Fabric Flattening with Wrinkle Direction Detection
• Authors: Yulei Qiu, Jihong Zhu, Cosimo Della Santina, Michael Gienger, Jens Kober
• Abstract summary: Perception is considered one of the major challenges in DOM due to the complex dynamics and high degree of freedom of deformable objects. We develop a novel image-processing algorithm based on Gabor filters to extract useful features from cloth. Our algorithm can determine the direction of wrinkles on the cloth accurately in simulation as well as in real robot experiments.
• Score: 9.822493398088127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deformable Object Manipulation (DOM) is an important field of research as it contributes to practical tasks such as automatic cloth handling, cable routing, surgical operation, etc. Perception is considered one of the major challenges in DOM due to the complex dynamics and high degree of freedom of deformable objects. In this paper, we develop a novel image-processing algorithm based on Gabor filters to extract useful features from cloth, and based on this, devise a strategy for cloth flattening tasks. We also evaluate the overall framework experimentally and compare it with three human operators. The results show that our algorithm can determine the direction of wrinkles on the cloth accurately in simulation as well as in real robot experiments. Furthermore, our dewrinkling strategy compares favorably to baseline methods. The experiment video is available on https://sites.google.com/view/robotic-fabric-flattening/home

Related papers

• Offline Imitation Learning Through Graph Search and Retrieval [57.57306578140857]
  Imitation learning is a powerful machine learning algorithm for a robot to acquire manipulation skills. We propose GSR, a simple yet effective algorithm that learns from suboptimal demonstrations through Graph Search and Retrieval. GSR can achieve a 10% to 30% higher success rate and over 30% higher proficiency compared to baselines.
  arXiv  Detail & Related papers  (2024-07-22T06:12:21Z)
• RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation [50.10282876199739]
  This paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects. It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow. A real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.
  arXiv  Detail & Related papers  (2023-11-21T07:19:47Z)
• Robust Visual Sim-to-Real Transfer for Robotic Manipulation [79.66851068682779]
  Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR)
  arXiv  Detail & Related papers  (2023-07-28T05:47:24Z)
• Learning Fabric Manipulation in the Real World with Human Videos [10.608723220309678]
  Fabric manipulation is a long-standing challenge in robotics due to the enormous state space and complex dynamics. Most prior methods rely heavily on simulation, which is still limited by the large sim-to-real gap of deformable objects. A promising alternative is to learn fabric manipulation directly from watching humans perform the task.
  arXiv  Detail & Related papers  (2022-11-05T07:09:15Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• VisuoSpatial Foresight for Physical Sequential Fabric Manipulation [22.008305401551418]
  We build upon the Visual Foresight framework to learn fabric dynamics that can be efficiently reused to accomplish different sequential fabric manipulation tasks. In this work, we vary 4 components of VSF, including data generation, the choice of visual dynamics model, cost function, and optimization procedure. Results suggest that training visual dynamics models using longer, corner-based actions can improve the efficiency of fabric folding by 76% and enable a physical sequential fabric folding task that VSF could not previously perform with 90% reliability.
  arXiv  Detail & Related papers  (2021-02-19T06:06:49Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)
• Robotic Arm Control and Task Training through Deep Reinforcement Learning [6.249276977046449]
  We show that Trust Region Policy Optimization and DeepQ-Network with Normalized Advantage Functions perform better than Deep Deterministic Policy Gradient and Vanilla Policy Gradient. Real-world experiments let show that our polices, if correctly trained on simulation, can be transferred and executed in a real environment with almost no changes.
  arXiv  Detail & Related papers  (2020-05-06T07:34:28Z)
• Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics [35.84249614544505]
  We learn visual correspondences for deformable fabrics across different configurations in simulation. The learned correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. Results also suggest to fabrics of various colors, sizes, and shapes.
  arXiv  Detail & Related papers  (2020-03-28T04:06:20Z)
• VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation [24.262746504997683]
  We extend the Visual Foresight framework to learn fabric dynamics that can be efficiently reused to accomplish different fabric manipulation tasks. We experimentally evaluate VSF on multi-step fabric smoothing and folding tasks against 5 baseline methods in simulation and on the da Vinci Research Kit (dVRK) surgical robot without any demonstrations at train or test time.
  arXiv  Detail & Related papers  (2020-03-19T23:12:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.