GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulation

• URL: http://arxiv.org/abs/2503.09243v1
• Date: Wed, 12 Mar 2025 10:39:12 GMT
• Title: GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulation
• Authors: Ruihai Wu, Ziyu Zhu, Yuran Wang, Yue Chen, Jiarui Wang, Hao Dong,
• Abstract summary: Unlike single-garment manipulation, cluttered scenarios require managing complex garment entanglements and interactions.<n>We learn point-level affordance, the dense representation modeling the complex space and multi-modal manipulation candidates.<n>We introduce an adaptation module, guided by learned affordance, to reorganize highly-entangled garments into states plausible for manipulation.
• Score: 14.604134812602044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cluttered garments manipulation poses significant challenges due to the complex, deformable nature of garments and intricate garment relations. Unlike single-garment manipulation, cluttered scenarios require managing complex garment entanglements and interactions, while maintaining garment cleanliness and manipulation stability. To address these demands, we propose to learn point-level affordance, the dense representation modeling the complex space and multi-modal manipulation candidates, while being aware of garment geometry, structure, and inter-object relations. Additionally, as it is difficult to directly retrieve a garment in some extremely entangled clutters, we introduce an adaptation module, guided by learned affordance, to reorganize highly-entangled garments into states plausible for manipulation. Our framework demonstrates effectiveness over environments featuring diverse garment types and pile configurations in both simulation and the real world. Project page: https://garmentpile.github.io/.

Related papers

• GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoning [27.756766557197746]
  Garment manipulation has attracted increasing attention due to its critical role in home-assistant robotics.<n>We propose a novel garment retrieval pipeline that can not only follow language instruction to execute safe and clean retrieval but also guarantee exactly one garment is retrieved per attempt.<n>Our pipeline seamlessly integrates vision-language reasoning with visual affordance perception, fully leveraging the high-level reasoning and planning capabilities of VLMs.
  arXiv  Detail & Related papers  (2026-03-04T15:13:40Z)
• DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy [74.9519138296936]
  Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations.<n>We propose DexGarmentLab, the first environment specifically designed for dexterous (especially bimanual) garment manipulation.<n>It features large-scale high-quality 3D assets for 15 task scenarios, and refines simulation techniques tailored for garment modeling to reduce the sim-to-real gap.
  arXiv  Detail & Related papers  (2025-05-16T09:26:59Z)
• AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models [7.534556848810697]
  We propose a novel AnyDressing method for customizing characters conditioned on any combination of garments and personalized text prompts.<n>AnyDressing comprises two primary networks named GarmentsNet and DressingNet, which are respectively dedicated to extracting detailed clothing features.<n>We introduce a Garment-Enhanced Texture Learning strategy to improve the fine-grained texture details of garments.
  arXiv  Detail & Related papers  (2024-12-05T13:16:47Z)
• UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence [6.9061350009929185]
  Garment manipulation is essential for future robots to accomplish home-assistant tasks. We leverage the property that, garments in a certain category have similar structures. We then learn the topological dense (point-level) visual correspondence among garments in the category level with different deformations.
  arXiv  Detail & Related papers  (2024-05-11T04:18:41Z)
• AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model [58.035758145894846]
  We introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos. A pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. Our method is able to render natural garment dynamics that deviate highly from the body and well to generalize to both unseen views and poses.
  arXiv  Detail & Related papers  (2024-01-27T08:48:18Z)
• High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos [51.8323369577494]
  We propose the first method to recover high-quality animatable dynamic garments from monocular videos without depending on scanned data. To generate reasonable deformations for various unseen poses, we propose a learnable garment deformation network. We show that our method can reconstruct high-quality dynamic garments with coherent surface details, which can be easily animated under unseen poses.
  arXiv  Detail & Related papers  (2023-11-02T13:16:27Z)
• ClothCombo: Modeling Inter-Cloth Interaction for Draping Multi-Layered Clothes [3.8079353598215757]
  We present ClothCombo, a pipeline to drape arbitrary combinations of clothes on 3D human models. Our method utilizes a GNN-based network to efficiently model the interaction between clothes in different layers.
  arXiv  Detail & Related papers  (2023-04-07T06:23:54Z)
• DIG: Draping Implicit Garment over the Human Body [56.68349332089129]
  We propose an end-to-end differentiable pipeline that represents garments using implicit surfaces and learns a skinning field conditioned on shape and pose parameters of an articulated body model. We show that our method, thanks to its end-to-end differentiability, allows to recover body and garments parameters jointly from image observations.
  arXiv  Detail & Related papers  (2022-09-22T08:13:59Z)
• Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing [85.74977256940855]
  We propose an Arbitrary Virtual Try-On Network (AVTON) for all-type clothes. AVTON can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.
  arXiv  Detail & Related papers  (2021-11-24T08:59:56Z)
• Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN [66.3650689395967]
  We propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. To disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module.
  arXiv  Detail & Related papers  (2021-11-20T08:36:12Z)
• Per Garment Capture and Synthesis for Real-time Virtual Try-on [15.128477359632262]
  Existing image-based works try to synthesize a try-on image from a single image of a target garment. It is difficult to reproduce the change of wrinkles caused by pose and body size change, as well as pulling and stretching of the garment by hand. We propose an alternative per garment capture and synthesis workflow to handle such rich interactions by training the model with many systematically captured images.
  arXiv  Detail & Related papers  (2021-09-10T03:49:37Z)
• SMPLicit: Topology-aware Generative Model for Clothed People [65.84665248796615]
  We introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people.
  arXiv  Detail & Related papers  (2021-03-11T18:57:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.