ShapeGrasp: Simultaneous Visuo-Haptic Shape Completion and Grasping for Improved Robot Manipulation

• URL: http://arxiv.org/abs/2605.02347v1
• Date: Mon, 04 May 2026 08:49:52 GMT
• Title: ShapeGrasp: Simultaneous Visuo-Haptic Shape Completion and Grasping for Improved Robot Manipulation
• Abstract summary: The proposed method is an iterative grasp-and-complete pipeline that couples implicit surface visuo-haptic shape completion with physics-based grasp planning.<n>From a single RGB-D view, ShapeGrasp infers a complete shape, generates candidate grasps via rigid-body simulation, and executes the best feasible grasp.<n>We evaluate ShapeGrasp in the real world using two different robots and grippers.
• Score: 3.415147604573136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans grasp unfamiliar objects by combining an initial visual estimate with tactile and proprioceptive feedback during interaction. We present ShapeGrasp, a robotic implementation of this approach. The proposed method is an iterative grasp-and-complete pipeline that couples implicit surface visuo-haptic shape completion (creation of full 3D shape from partial information) with physics-based grasp planning. From a single RGB-D view, ShapeGrasp infers a complete shape (point cloud or triangular mesh), generates candidate grasps via rigid-body simulation, and executes the best feasible grasp. Each grasp attempt yields additional geometric constraints -- tactile surface contacts and space occupied by the gripper body -- which are fused to update the object shape. Failures trigger pose re-estimation and regrasping using the refined shape. We evaluate ShapeGrasp in the real world using two different robots and grippers. To the best of our knowledge, this is the first approach that updates shape representations following a real-world grasp. We achieved superior results over baselines for both grippers (grasp success rate of 84% with a three-finger gripper and 91% with a two-finger gripper), while improving the 3D shape reconstruction quality in all evaluation metrics used.

Related papers

• MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping [82.63833405368159]
  Cross-embodiment dexterous grasping aims to synthesize stable grasps across heterogeneous multi-fingered hands with little or no embodiment-specific tuning.<n>We propose MANGO-Grasp, an anisotropic interaction framework that represents objects as geometry-oriented 3D Gaussian primitives and robot hands as surface keypoints encoded into morpho-kinematic descriptors.
  arXiv  Detail & Related papers  (2026-08-03T10:12:15Z)
• Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation [53.09168514034483]
  Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions.<n>We propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model.<n>Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap.
  arXiv  Detail & Related papers  (2026-02-27T08:54:20Z)
• Single-View Shape Completion for Robotic Grasping in Clutter [5.084516012213421]
  In vision-based robot manipulation, a single camera view can only capture one side of objects of interest.<n>We leverage diffusion models to perform category-level 3D shape completion from partial depth observations.<n>Our approach consistently results in better grasp success rates than a naive baseline without shape completion by 23%.
  arXiv  Detail & Related papers  (2025-12-18T12:11:05Z)
• WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction [13.645442589551354]
  WIR3D is a technique for abstracting 3D shapes through a sparse set of visually meaningful curves in 3D.<n>We optimize the parameters of Bezier curves such that they faithfully represent both the geometry and salient visual features.<n>We successfully apply our method for shape abstraction over a broad dataset of shapes.
  arXiv  Detail & Related papers  (2025-05-07T21:28:05Z)
• PCF-Grasp: Converting Point Completion to Geometry Feature to Enhance 6-DoF Grasp [14.909551737430473]
  We propose a novel 6-DoF grasping framework that converts the point completion results as object shape features to train the 6-DoF grasp network.<n>Our method achieves a 17.8% success rate higher than the state-of-the-art method in real-world experiments.
  arXiv  Detail & Related papers  (2025-04-22T23:37:05Z)
• Robust 3D Tracking with Quality-Aware Shape Completion [67.9748164949519]
  We propose a synthetic target representation composed of dense and complete point clouds depicting the target shape precisely by shape completion for robust 3D tracking. Specifically, we design a voxelized 3D tracking framework with shape completion, in which we propose a quality-aware shape completion mechanism to alleviate the adverse effect of noisy historical predictions.
  arXiv  Detail & Related papers  (2023-12-17T04:50:24Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction [1.290382979353427]
  We propose a Deep Mesh Relation Capturing Graph Convolution Network, DC-GNet, with a shape completion task for 3D human shape reconstruction. Our approach encodes mesh structure from more subtle relations between nodes in a more distant region. Our shape completion module alleviates the performance degradation issue in the outdoor scene.
  arXiv  Detail & Related papers  (2021-08-27T16:43:32Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• From Points to Multi-Object 3D Reconstruction [71.17445805257196]
  We propose a method to detect and reconstruct multiple 3D objects from a single RGB image. A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes. The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
  arXiv  Detail & Related papers  (2020-12-21T18:52:21Z)
• Learning 3D Human Shape and Pose from Dense Body Parts [117.46290013548533]
  We propose a Decompose-and-aggregate Network (DaNet) to learn 3D human shape and pose from dense correspondences of body parts. Messages from local streams are aggregated to enhance the robust prediction of the rotation-based poses. Our method is validated on both indoor and real-world datasets including Human3.6M, UP3D, COCO, and 3DPW.
  arXiv  Detail & Related papers  (2019-12-31T15:09:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.