GraspSense: Physically Grounded Grasp and Grip Planning for a Dexterous Robotic Hand via Language-Guided Perception and Force Maps

• URL: http://arxiv.org/abs/2604.05697v2
• Date: Mon, 13 Apr 2026 12:02:59 GMT
• Title: GraspSense: Physically Grounded Grasp and Grip Planning for a Dexterous Robotic Hand via Language-Guided Perception and Force Maps
• Abstract summary: We present a pipeline for grasp selection and force regulation in a five-fingered robotic hand.<n>From an operator command, the system identifies the target object, reconstructs its 3D geometry using SAM3D, and imports the model into Isaac Sim.<n>A physics-informed geometric analysis then computes a force map that encodes the maximum lateral contact force admissible at each surface location.
• Score: 2.1645011609137295
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Dexterous robotic manipulation requires more than geometrically valid grasps: it demands physically grounded contact strategies that account for the spatially non-uniform mechanical properties of the object. However, existing grasp planners typically treat the surface as structurally homogeneous, even though contact in a weak region can damage the object despite a geometrically perfect grasp. We present a pipeline for grasp selection and force regulation in a five-fingered robotic hand, based on a map of locally admissible contact loads. From an operator command, the system identifies the target object, reconstructs its 3D geometry using SAM3D, and imports the model into Isaac Sim. A physics-informed geometric analysis then computes a force map that encodes the maximum lateral contact force admissible at each surface location without deformation. Grasp candidates are filtered by geometric validity and task-goal consistency. When multiple candidates are comparable under classical metrics, they are re-ranked using a force-map-aware criterion that favors grasps with contacts in mechanically admissible regions. An impedance controller scales the stiffness of each finger according to the locally admissible force at the contact point, enabling safe and reliable grasp execution. Validation on paper, plastic, and glass cups shows that the proposed approach consistently selects structurally stronger contact regions and keeps grip forces within safe bounds. In this way, the work reframes dexterous manipulation from a purely geometric problem into a physically grounded joint planning problem of grasp selection and grip execution for future humanoid systems.

Related papers

• Control Architecture for Safe Grasping of Fragile Objects Using a Coarse Position-Controlled Gripper [0.26999000177990923]
  We present a force/torque-based contact estimation method for a coarse position-controlled gripper.<n>The desired contact forces are estimated online to keep stable contact with objects of unknown properties.<n> Experiments show that the proposed algorithm not only prevents slippage but also applies minimal force to safely grasp an object.
  arXiv  Detail & Related papers  (2026-09-11T11:37:40Z)
• MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping [82.63833405368159]
  Cross-embodiment dexterous grasping aims to synthesize stable grasps across heterogeneous multi-fingered hands with little or no embodiment-specific tuning.<n>We propose MANGO-Grasp, an anisotropic interaction framework that represents objects as geometry-oriented 3D Gaussian primitives and robot hands as surface keypoints encoded into morpho-kinematic descriptors.
  arXiv  Detail & Related papers  (2026-08-03T10:12:15Z)
• DeSeG: Decoupling Semantic Intent and Geometric Constraints for Physically Plausible Human-Scene Interaction [50.06148080975928]
  We propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints.<n>DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
  arXiv  Detail & Related papers  (2026-07-07T03:21:08Z)
• DCGrasp: Distance-aware Controllable Grasp Generation [64.30284326160826]
  We present DCGrasp, a controllable grasp generation system built on a novel grasp energy term.<n>This term computes Distance Profile, a signed distance from each hand to the nearest object point.<n>Our experiments show that DCGrasp produces high-quality, physically plausible grasps with flexible user control.
  arXiv  Detail & Related papers  (2026-06-29T08:02:02Z)
• Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation [51.463992799813816]
  A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data.<n>We introduce Center-of-Pressure, an effective tactile representation grounded in physical principles.<n>We evaluate CoP on two blind, challenging contact-rich manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-27T17:59:02Z)
• SECOND-Grasp: Semantic Contact-guided Dexterous Grasping [60.1519218638742]
  Second-Grasp (SEmantic CONtact-guided Dexterous Grasping) is a unified framework that enables robotic hands to adjust grasping strategies based on semantic reasoning.<n>We show that our approach consistently outperforms baselines in lifting success rate on both seen and unseen categories.
  arXiv  Detail & Related papers  (2026-05-13T07:37:00Z)
• arg-VU: Affordance Reasoning with Physics-Aware 3D Geometry for Visual Understanding in Robotic Surgery [2.3780752271138965]
  Arg-VU is a physics-aware affordance reasoning framework.<n>It integrates temporally consistent geometry tracking with constraint-induced mechanical modeling.<n>Results show that arg-VU yields more stable, physically consistent, and interpretable affordance predictions than kinematic baselines.
  arXiv  Detail & Related papers  (2026-03-26T17:14:28Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• Simultaneous Extrinsic Contact and In-Hand Pose Estimation via Distributed Tactile Sensing [20.162368645967153]
  We propose a method that pairs local observations from sensing with the physical constraints of contact.<n>We formalize our problem as a factor graph, allowing for efficient estimation.<n>In our experiments, we demonstrate that our method outperforms existing geometric and contact-informed estimation pipelines.
  arXiv  Detail & Related papers  (2025-12-29T20:45:38Z)
• Graph Neural Network Surrogates for Contacting Deformable Bodies with Necessary and Sufficient Contact Detection [0.0]
  Surrogate models for the rapid inference of nonlinear boundary value problems in mechanics are helpful in a broad range of engineering applications.<n>Existing methods are confined to rigid body contact or, at best, contact between rigid and soft objects with well-defined contact planes.<n>We present a graph neural network architecture that utilizes continuous collision detection and, for the first time, incorporates sufficient conditions designed for contact between soft deformable bodies.
  arXiv  Detail & Related papers  (2025-07-17T18:09:19Z)
• Flatten Anything: Unsupervised Neural Surface Parameterization [76.4422287292541]
  We introduce the Flatten Anything Model (FAM), an unsupervised neural architecture to achieve global free-boundary surface parameterization. Compared with previous methods, our FAM directly operates on discrete surface points without utilizing connectivity information. Our FAM is fully-automated without the need for pre-cutting and can deal with highly-complex topologies.
  arXiv  Detail & Related papers  (2024-05-23T14:39:52Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• Semi-Supervised Disentanglement of Tactile Contact~Geometry from Sliding-Induced Shear [12.004939546183355]
  The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information.
  arXiv  Detail & Related papers  (2022-08-26T08:30:19Z)
• Leveraging distributed contact force measurements for slip detection: a physics-based approach enabled by a data-driven tactile sensor [5.027571997864706]
  This paper describes a novel model-based slip detection pipeline that can predict possibly failing grasps in real-time. A vision-based tactile sensor that accurately estimates distributed forces was integrated into a grasping setup composed of a six degrees-of-freedom cobot and a two-finger gripper. Results show that the system can reliably predict slip while manipulating objects of different shapes, materials, and weights.
  arXiv  Detail & Related papers  (2021-09-23T17:12:46Z)
• Physics-Based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning [52.37106940303246]
  We learn a model that maps noisy input hand poses to target virtual poses. The agent is trained in a residual setting by using a model-free hybrid RL+IL approach. We test our framework in two applications that use hand pose estimates for dexterous manipulations: hand-object interactions in VR and hand-object motion reconstruction in-the-wild.
  arXiv  Detail & Related papers  (2020-08-07T17:34:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.