Synthetic Data Generation and Vision-based Wrinkle and Keypoint Detection for Bimanual Cloth Manipulation

• URL: http://arxiv.org/abs/2606.06292v1
• Date: Thu, 04 Jun 2026 15:30:49 GMT
• Title: Synthetic Data Generation and Vision-based Wrinkle and Keypoint Detection for Bimanual Cloth Manipulation
• Abstract summary: We present a perception framework integrating a CNN for permutation-invariant keypoint detection and a YOLOv8-OpenCV pipeline to extract grasping points from structural wrinkles.<n>A proposed bimanual algorithm uses this system to stretch fully folded garments via wrinkles, transitioning to keypoint-based ironing once corners emerge.
• Score: 1.873444918172383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic manipulation of textiles remains challenging because continuous deformation and self-occlusions hinder the robust visual perception required to estimate the cloth's state. To address the lack of annotated real-world data, we developed a Blender-based synthetic pipeline exporting auto-annotated keypoints, and combined manually labeled renders with real-world data to train a wrinkle detector. We present a perception framework integrating a CNN for permutation-invariant keypoint detection and a YOLOv8-OpenCV pipeline to extract grasping points from structural wrinkles. A proposed bimanual algorithm uses this system to stretch fully folded garments via wrinkles, transitioning to keypoint-based ironing once corners emerge. The keypoint model achieves a Mean Position Error (MPE) of 1.7615 pixels. The perception system transfers to physical fabrics without fine-tuning, outperforming baselines that fail in high-occlusion states or yield false positives on severe folds.

Related papers

• ViP-Rig: Visual-Prompted Controllable Rigging [66.07397903377584]
  ViP-Rig is a visual-prompted framework that supports both prompt-first rigging and result-guided editing.<n>We show that ViP-Rig more accurately recovers target skeletons and skinning weights than geometry-conditioned baselines under prompt-guided evaluation.
  arXiv  Detail & Related papers  (2026-07-30T10:26:37Z)
• ScratchSim: A Procedural Synthetic Data Pipeline for Surface Scratch Detection [7.759492039775529]
  This paper presents a procedural rendering pipeline that generates large-scale annotated synthetic training data using Blender.<n>We evaluate four training strategies, namely synthetic-only, real-only, mixed, and fine-tuning from synthetic weights.<n>The proposed approach enables scalable defect detection without the burden of large real annotated datasets.
  arXiv  Detail & Related papers  (2026-07-29T15:54:01Z)
• Visual Geometry Transformer in the Wild: Distractor-Free 3D Reconstruction [77.75911117889915]
  Current end-to-end multi-view 3D reconstruction methods achieve impressive results, but rely on a restrictive static assumption.<n>This reliance on idealized inputs causes even the most advanced methods to fail in real-world settings.<n>We propose Visual Geometry Transformer in the Wild, an end-to-end framework for robust reconstruction from inconsistent views.
  arXiv  Detail & Related papers  (2026-06-22T02:52:22Z)
• RoboHitch: Learning Visual Affordance from Disordered Keypoints for Hitch Knots Tying [4.437059562306781]
  Existing robotic knot tying methods rely on precise topological state tracking with ordered keypoints and explicit edge connectivity.<n>We introduce RoboHitch, a novel framework that learns to perform hitch knot tying from human demonstrations using only disordered 3D keypoints and RGB images.
  arXiv  Detail & Related papers  (2026-05-23T04:38:58Z)
• ChartZero: Synthetic Priors Enable Zero Shot Chart Data Extraction [2.825872203744638]
  We introduce ChartZero, a parsing framework that leverages synthetic priors to enable robust zero-shot chart data extraction.<n>We overcome curve fragmentation via a novel Global Orthogonal Instance (GOI) loss, and replace brittle spatial rules with an open-vocabulary, Vision-Language Model (VLM)-guided legend matching strategy.
  arXiv  Detail & Related papers  (2026-05-07T07:57:48Z)
• Mesh Denoising Transformer [104.5404564075393]
  Mesh denoising is aimed at removing noise from input meshes while preserving their feature structures. SurfaceFormer is a pioneering Transformer-based mesh denoising framework. New representation known as Local Surface Descriptor captures local geometric intricacies. Denoising Transformer module receives the multimodal information and achieves efficient global feature aggregation.
  arXiv  Detail & Related papers  (2024-05-10T15:27:43Z)
• Anomaly Detection in Automated Fibre Placement: Learning with Data Limitations [3.103778949672542]
  We present a comprehensive framework for defect detection and localization in Automated Fibre Placement. Our approach combines unsupervised deep learning and classical computer vision algorithms. It efficiently detects various surface issues while requiring fewer images of composite parts for training.
  arXiv  Detail & Related papers  (2023-07-15T22:13:36Z)
• Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud Analysis [66.49788145564004]
  We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology. Our methods perform favorably against the current state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-12-17T15:05:25Z)
• UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detection [52.91782218300844]
  We propose a novel Unsupervised Inconsistency-Aware method based on Vision Transformer, called UIA-ViT. Due to the self-attention mechanism, the attention map among patch embeddings naturally represents the consistency relation, making the vision Transformer suitable for the consistency representation learning.
  arXiv  Detail & Related papers  (2022-10-23T15:24:47Z)
• VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling [2.0624279915507047]
  Training 3D scene parsing models with sparse supervision is an intriguing alternative. We term this task as data-efficient 3D scene parsing. We propose an effective two-stage framework named VIBUS to resolve it.
  arXiv  Detail & Related papers  (2022-10-20T17:59:57Z)
• Leveraging Deepfakes to Close the Domain Gap between Real and Synthetic Images in Facial Capture Pipelines [8.366597450893456]
  We propose an end-to-end pipeline for building and tracking 3D facial models from personalized in-the-wild video data. We present a method for automatic data curation and retrieval based on a hierarchical clustering framework typical of collision algorithms in traditional computer graphics pipelines. We outline how we train a motion capture regressor, leveraging the aforementioned techniques to avoid the need for real-world ground truth data.
  arXiv  Detail & Related papers  (2022-04-22T15:09:49Z)
• Point Set Self-Embedding [63.23565826873297]
  This work presents an innovative method for point set self-embedding, that encodes structural information of a dense point set into its sparser version in a visual but imperceptible form. The self-embedded point set can function as the ordinary downsampled one and be visualized efficiently on mobile devices. We can leverage the self-embedded information to fully restore the original point set for detailed analysis on remote servers.
  arXiv  Detail & Related papers  (2022-02-28T07:03:33Z)
• Data Augmentation for Object Detection via Differentiable Neural Rendering [71.00447761415388]
  It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data. We introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
  arXiv  Detail & Related papers  (2021-03-04T06:31:06Z)
• SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization [52.20602782690776]
  It is expensive and tedious to obtain large scale paired sparse-canned point sets for training from real scanned sparse data. We propose a self-supervised point cloud upsampling network, named SPU-Net, to capture the inherent upsampling patterns of points lying on the underlying object surface. We conduct various experiments on both synthetic and real-scanned datasets, and the results demonstrate that we achieve comparable performance to the state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2020-12-08T14:14:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.