SAGP: Semantic Affordance-Guided Grasp Planning via Coarse-Zone VLM Reasoning

• URL: http://arxiv.org/abs/2607.29374v1
• Date: Fri, 31 Jul 2026 12:59:05 GMT
• Title: SAGP: Semantic Affordance-Guided Grasp Planning via Coarse-Zone VLM Reasoning
• Abstract summary: Geometry-based grasp planners ensure physically valid grasps but ignore functional semantics.<n>No practical, training-free framework has yet been proposed that robustly links high-level semantic reasoning to geometric grasp planning.<n>We introduce Semantic Affordance-Guided Grasp Planning (SAGP), a training-free pipeline built on a coarse-zone abstraction layer.
• Score: 1.8988957322012407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Geometry-based grasp planners ensure physically valid grasps but ignore functional semantics, often generating grasps that are antipodal and collision-free yet practically inappropriate, for example, gripping a mug by its rim, a knife by the blade, or a bottle near its cap. These inconsistencies cause the downstream task to fail even when traditional grasp metrics are met. Existing vision-language model (VLM) approaches either depend on fine-grained, category-specific part segmentation or attempt to directly infer grasp poses, with the latter prone to spatial hallucinations. As a result, no practical, training-free framework has yet been proposed that robustly links high-level semantic reasoning to geometric grasp planning. We introduce Semantic Affordance-Guided Grasp Planning (SAGP), a training-free pipeline built on a coarse-zone abstraction layer. The method first partitions the object point cloud into spatial regions (top, middle, bottom, lateral sides, and protrusions) by applying PCA-based alignment followed by distance-driven DBSCAN clustering, entirely bypassing learned segmentation. A pre-trained VLM then assesses the grasp quality of each region through a structured zero-shot query, and the resulting zone-wise scores are fused with geometric, reachability, and task-alignment signals to re-rank antipodal grasp candidates. Experiments on YCB objects in PyBullet with a Franka Panda robot show that SAGP preserves the high success rate of geometry-only planning while substantially improving the functional appropriateness of selected grasps, particularly on asymmetric, handle-bearing objects where geometry alone is uninformative. The introduced coarse-zone abstraction offers an effective, training-free bridge between VLM-based reasoning and geometric grasp planning, without the need for fine-grained part segmentation.

Related papers

• LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields [7.898784066160524]
  We present LeCropFollow, a visual navigation framework that bypasses explicit geometric modeling in favor of a learned latent representation.<n>We show that LeCropFollow matches state-of-the-art baselines in unstructured rows but significantly outperforms them in plantation gaps.<n>These results suggest that latent planning offers a robust alternative to geometric estimation for operations in heterogeneous agricultural environments.
  arXiv  Detail & Related papers  (2026-06-30T16:47:08Z)
• Learning Unions of Convex Sets via Invertible Latent Decomposition for Path Planning [19.14344979340464]
  Invertible Latent Decomposition (ILD) is a framework that learns an invertible mapping and a union of explicit convex polytopes in the resulting latent space.<n>ILD achieves broader coverage, better inter-set connectivity, and higher path-planning success rates than prior baselines.<n>On a 14-DoF bimanual manipulator, we demonstrate real-time collision-free planning, with test-time refinement adapting to scene-geometry changes.
  arXiv  Detail & Related papers  (2026-06-10T12:49:47Z)
• SOCC-ICP: Semantics-Assisted Odometry based on Occupancy Grids and ICP [1.2744523252873352]
  SOCC-ICP is a semantics-assisted odometry framework that jointly performs Semantic OCCupancy grid mapping and LiDAR scan alignment.<n>It achieves performance competitive with state-of-the-art LiDAR odometry and remains robust in geometrically degenerate environments.<n>By unifying odometry and semantic occupancy grid mapping within a single representation, SOCC-ICP eliminates redundant map structures.
  arXiv  Detail & Related papers  (2026-05-14T17:00:38Z)
• SphereVAD: Training-Free Video Anomaly Detection via Geodesic Inference on the Unit Hypersphere [90.85001795656633]
  Video anomaly detection (VAD) aims to automatically identify events that deviate from normal patterns in untrimmed surveillance videos.<n>We propose SphereVAD, a fully training-free, zero-shot VAD framework that recasts anomaly discrimination as von Mises-Fisher (vMF) likelihood-ratio geodesic inference on the unit hypersphere.
  arXiv  Detail & Related papers  (2026-05-08T16:57:38Z)
• GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models [70.61152292499737]
  We argue that this gap does not arise from insufficient geometric priors, but from a misalignment in the training paradigm.<n>Existing approaches typically resort to naive feature concatenation and optimize directly for downstream tasks without geometry-specific supervision.<n>We propose GAP-MLLM, a Geometry-Aligned Pre-training paradigm that explicitly activates structural perception before downstream adaptation.
  arXiv  Detail & Related papers  (2026-03-17T12:43:48Z)
• Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA [50.494504099850325]
  We introduce the Geodesic Hypothesis, positing that token sequences trace geodesics on a smooth semantic manifold and are therefore locally linear.<n>We show this constraint improves signal-to-noise ratio, and preserves diversity by preventing collisions during trajectory.<n>We demonstrate that geometric priors can surpass brute-force scaling.
  arXiv  Detail & Related papers  (2026-02-26T04:45:07Z)
• Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Remote Sensing [8.731693840957716]
  Think2Seg-RS is a framework that trains an LVLM prompter to control a frozen Segment Anything Model (SAM) via structured geometric prompts.<n>The framework achieves state-of-the-art performance on the EarthReason dataset.<n> compact segmenters outperform larger ones under semantic-level supervision, and that negative prompts are ineffective in heterogeneous aerial backgrounds.
  arXiv  Detail & Related papers  (2025-12-22T11:46:42Z)
• Geometrically-Constrained Agent for Spatial Reasoning [53.93718394870856]
  Vision Language Models exhibit a fundamental semantic-to-geometric gap in spatial reasoning.<n>Current paradigms fail to bridge this gap.<n>We propose a training-free agentic paradigm that resolves this gap by introducing a formal task constraint.
  arXiv  Detail & Related papers  (2025-11-27T17:50:37Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• Spatial Pyramid Based Graph Reasoning for Semantic Segmentation [67.47159595239798]
  We apply graph convolution into the semantic segmentation task and propose an improved Laplacian. The graph reasoning is directly performed in the original feature space organized as a spatial pyramid. We achieve comparable performance with advantages in computational and memory overhead.
  arXiv  Detail & Related papers  (2020-03-23T12:28:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.