UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Folding

• URL: http://arxiv.org/abs/2311.01267v1
• Date: Thu, 2 Nov 2023 14:25:10 GMT
• Title: UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Folding
• Authors: Han Xue, Yutong Li, Wenqiang Xu, Huanyu Li, Dongzhe Zheng, Cewu Lu
• Abstract summary: UniFolding is a sample-efficient, scalable, and generalizable robotic system for unfolding and folding garments. UniFolding employs the proposed UFONet neural network to integrate unfolding and folding decisions into a single policy model. The system is tested on two garment types: long-sleeve and short-sleeve shirts.
• Score: 53.38503172679482
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This paper explores the development of UniFolding, a sample-efficient, scalable, and generalizable robotic system for unfolding and folding various garments. UniFolding employs the proposed UFONet neural network to integrate unfolding and folding decisions into a single policy model that is adaptable to different garment types and states. The design of UniFolding is based on a garment's partial point cloud, which aids in generalization and reduces sensitivity to variations in texture and shape. The training pipeline prioritizes low-cost, sample-efficient data collection. Training data is collected via a human-centric process with offline and online stages. The offline stage involves human unfolding and folding actions via Virtual Reality, while the online stage utilizes human-in-the-loop learning to fine-tune the model in a real-world setting. The system is tested on two garment types: long-sleeve and short-sleeve shirts. Performance is evaluated on 20 shirts with significant variations in textures, shapes, and materials. More experiments and videos can be found in the supplementary materials and on the website: https://unifolding.robotflow.ai

Related papers

• SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation [82.61572106180705]
  This paper presents a unified approach using vision-language models (VLMs) to improve keypoint prediction across various garment categories. We created a large-scale synthetic dataset using advanced simulation techniques, allowing scalable training without extensive real-world data. Experimental results indicate that the VLM-based method significantly enhances keypoint detection accuracy and task success rates.
  arXiv  Detail & Related papers  (2024-09-26T17:26:16Z)
• ClothFit: Cloth-Human-Attribute Guided Virtual Try-On Network Using 3D Simulated Dataset [5.260305201345232]
  We propose a novel virtual try-on method called ClothFit. It can predict the draping shape of a garment on a target body based on the actual size of the garment and human attributes. Our experimental results demonstrate that ClothFit can significantly improve the existing state-of-the-art methods in terms of photo-realistic virtual try-on results.
  arXiv  Detail & Related papers  (2023-06-24T08:57:36Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Towards Multi-Layered 3D Garments Animation [135.77656965678196]
  Existing approaches mostly focus on single-layered garments driven by only human bodies and struggle to handle general scenarios. We propose a novel data-driven method, called LayersNet, to model garment-level animations as particle-wise interactions in a micro physics system. Our experiments show that LayersNet achieves superior performance both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-05-17T17:53:04Z)
• Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing [85.74977256940855]
  We propose an Arbitrary Virtual Try-On Network (AVTON) for all-type clothes. AVTON can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.
  arXiv  Detail & Related papers  (2021-11-24T08:59:56Z)
• Wearable-based Classification of Running Styles with Deep Learning [8.422257363944295]
  We develop a system capable of classifying running styles using wearables. Five wearable devices are used to record accelerometer data from different parts of the lower body. We show that the proposed model is capable of automatically classifying different running styles in a subject-dependant manner.
  arXiv  Detail & Related papers  (2021-09-01T19:55:06Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics [35.84249614544505]
  We learn visual correspondences for deformable fabrics across different configurations in simulation. The learned correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. Results also suggest to fabrics of various colors, sizes, and shapes.
  arXiv  Detail & Related papers  (2020-03-28T04:06:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.