Learning Bimanual Cloth Manipulation with Vision-based Tactile Sensing via Single Robotic Arm

• URL: http://arxiv.org/abs/2603.10609v1
• Date: Wed, 11 Mar 2026 10:15:42 GMT
• Title: Learning Bimanual Cloth Manipulation with Vision-based Tactile Sensing via Single Robotic Arm
• Abstract summary: Touch G.O.G. is a vision-based tactile gripper and perception/control framework for single-arm bimanual cloth manipulation.<n>Experiments show 96% accuracy in distinguishing edges, corners, interior regions, and grasp failures.<n>Results highlight Touch G.O.G. as a compact and cost-effective solution for deformable object manipulation.
• Score: 5.190090354079479
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic cloth manipulation remains challenging due to the high-dimensional state space of fabrics, their deformable nature, and frequent occlusions that limit vision-based sensing. Although dual-arm systems can mitigate some of these issues, they increase hardware and control complexity. This paper presents Touch G.O.G., a compact vision-based tactile gripper and perception/control framework for single-arm bimanual cloth manipulation. The proposed framework combines three key components: (1) a novel gripper design and control strategy for in-gripper cloth sliding with a single robot arm, (2) a Vision Foundation Model-backboned Vision Transformer pipeline for cloth part classification (PC-Net) and edge pose estimation (PE-Net) using real and synthetic tactile images, and (3) an encoder-decoder synthetic data generator (SD-Net) that reduces manual annotation by producing high-fidelity tactile images. Experiments show 96% accuracy in distinguishing edges, corners, interior regions, and grasp failures, together with sub-millimeter edge localization and 4.5° orientation error. Real-world results demonstrate reliable cloth unfolding, even for crumpled fabrics, using only a single robotic arm. These results highlight Touch G.O.G. as a compact and cost-effective solution for deformable object manipulation.

Related papers

• One Print, Many Moves: Monolithic Origami-inspired Folding Actuator for Composable Soft Multi-DoF Systems [5.952185594637419]
  We introduce a MONOlithic ORIGAMI-inspired soft folding actuator design (MONORIGAMI)<n>The design is fully 3D-printable through a single-material, single-print process that requires no assembly.<n>We demonstrate three 3D-printed soft multi-DoF robotic systems spanning distinct application domains.
  arXiv  Detail & Related papers  (2026-09-01T05:31:22Z)
• VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation [22.38178317703772]
  This paper presents a tactile-reactive gripper that integrates a Visuo-Tactile Active Palm (VTAP) and compliant, reconfigurable fingers equipped with array sensors.<n>The design exploits structured finger-palm synergy and multi-modal tactile perception to achieve both robust grasping and fine manipulation.
  arXiv  Detail & Related papers  (2026-07-16T20:41:00Z)
• AetheRock: An Arm-Worn Robot Teaching System for Force-Guided Vision-Tactile Learning [56.20593626288444]
  AetheRock is an arm-worn device featuring a modular and easily manufactured visuo-tactile sensor, GelSlim-MiniFab, at the fingertip.<n>We propose ForceVT, a representation learning framework that uses force and vision to guide fidelity-agnostic tactile learning.
  arXiv  Detail & Related papers  (2026-06-08T17:38:57Z)
• Form-Fitting, Large-Area Sensor Mounting for Obstacle Detection [99.11587290390071]
  We introduce a low-cost method for mounting sensors onto robot links for large-area sensing coverage.<n>Using computer aided design (CAD), a robot skin covering, or skin unit, can be procedurally generated to fit around a nondevelopable surface.<n>The skin unit embeds mounts for printed circuit boards of any size to keep sensors in fixed and known locations.
  arXiv  Detail & Related papers  (2026-03-24T21:20:57Z)
• UniForce: A Unified Latent Force Model for Robot Manipulation with Diverse Tactile Sensors [51.88112610411651]
  We propose a novel unified tactile representation learning framework that learns a shared latent force space across diverse tactile sensors.<n>UniForce reduces cross-sensor domain shift by jointly modeling inverse dynamics (image-to-force) and forward dynamics (force-to-image)<n>To avoid reliance on expensive external force/torque (F/T) sensors, we exploit static equilibrium and collect force-paired data via direct sensor--object--sensor interactions.
  arXiv  Detail & Related papers  (2026-02-01T11:03:01Z)
• A High-Fidelity Digital Twin for Robotic Manipulation Based on 3D Gaussian Splatting [0.0]
  We present a framework that constructs high-quality digital twins within minutes from sparse RGB inputs.<n>Our system employs 3D Gaussian Splatting (3DGS) for fast, photorealistic reconstruction as a unified scene representation.<n>In experiments with a Franka Emika Panda robot, we demonstrate that this enhanced geometric accuracy effectively supports robust manipulation in real-world trials.
  arXiv  Detail & Related papers  (2026-01-06T17:29:10Z)
• ControlTac: Force- and Position-Controlled Tactile Data Augmentation with a Single Reference Image [24.223013595239916]
  ControlTac is a controllable framework that generates realistic tactile images conditioned on a single reference tactile image, contact force, and contact position.<n>We demonstrate that ControlTac can effectively augment tactile datasets and lead to consistent gains.
  arXiv  Detail & Related papers  (2025-05-26T20:01:17Z)
• Visuo-Tactile Object Pose Estimation for a Multi-Finger Robot Hand with Low-Resolution In-Hand Tactile Sensing [9.970043211592058]
  Accurate 3D pose estimation of grasped objects is an important prerequisite for robots to perform assembly or in-hand manipulation tasks.<n>We propose that combining visual information and proprioception with binary, low-resolution tactile contact measurements can mitigate this issue.
  arXiv  Detail & Related papers  (2025-03-25T17:53:53Z)
• TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing [29.691786688595762]
  Humans rely on their visual and tactile senses to develop a comprehensive 3D understanding of their physical environment. We propose TouchSDF, a Deep Learning approach for tactile 3D shape reconstruction. Our technique consists of two components: (1) a Convolutional Neural Network that maps tactile images into local meshes representing the surface at the touch location, and (2) an implicit neural function that predicts a signed distance function to extract the desired 3D shape.
  arXiv  Detail & Related papers  (2023-11-21T13:43:06Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps [100.72245315180433]
  We present a reconfigurable data glove design to capture different modes of human hand-object interactions. The glove operates in three modes for various downstream tasks with distinct features. We evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses.
  arXiv  Detail & Related papers  (2023-01-14T05:35:50Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation [16.54834671357377]
  General purpose in-hand manipulation remains one of the unsolved challenges of robotics. We introduce DIGIT, an inexpensive, compact, and high-resolution tactile sensor geared towards in-hand manipulation.
  arXiv  Detail & Related papers  (2020-05-29T17:07:54Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.