GeoLanG: Geometry-Aware Language-Guided Grasping with Unified RGB-D Multimodal Learning

• URL: http://arxiv.org/abs/2602.04231v1
• Date: Wed, 04 Feb 2026 05:42:55 GMT
• Title: GeoLanG: Geometry-Aware Language-Guided Grasping with Unified RGB-D Multimodal Learning
• Abstract summary: GeoLanG is an end-to-end multi-task framework for language-guided grasping.<n>It unifies visual and linguistic inputs into a shared representation space for robust semantic alignment and improved generalization.<n>We show that GeoLanG enables precise and robust language-guided grasping in complex, cluttered environments.
• Score: 17.389094942979455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-guided grasping has emerged as a promising paradigm for enabling robots to identify and manipulate target objects through natural language instructions, yet it remains highly challenging in cluttered or occluded scenes. Existing methods often rely on multi-stage pipelines that separate object perception and grasping, which leads to limited cross-modal fusion, redundant computation, and poor generalization in cluttered, occluded, or low-texture scenes. To address these limitations, we propose GeoLanG, an end-to-end multi-task framework built upon the CLIP architecture that unifies visual and linguistic inputs into a shared representation space for robust semantic alignment and improved generalization. To enhance target discrimination under occlusion and low-texture conditions, we explore a more effective use of depth information through the Depth-guided Geometric Module (DGGM), which converts depth into explicit geometric priors and injects them into the attention mechanism without additional computational overhead. In addition, we propose Adaptive Dense Channel Integration, which adaptively balances the contributions of multi-layer features to produce more discriminative and generalizable visual representations. Extensive experiments on the OCID-VLG dataset, as well as in both simulation and real-world hardware, demonstrate that GeoLanG enables precise and robust language-guided grasping in complex, cluttered environments, paving the way toward more reliable multimodal robotic manipulation in real-world human-centric settings.

Related papers

• OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following [59.53720386342017]
  We propose a unified vision-language framework that adapts foundational Vision-Language Models for scalable multi-person gaze reasoning.<n>By modeling all individuals, OmniGF seamlessly integrates precise spatial gaze target estimation, semantic gaze prediction, and complex social gaze reasoning.
  arXiv  Detail & Related papers  (2026-05-26T00:08:06Z)
• LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models [27.76634830542925]
  spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world.<n>We propose LAST, a unified framework for tool-augmented spatial reasoning.<n> LAST features an interactive sandbox, LAST-Box, which invocations heterogeneous tool invocations into atomic instructions.<n>We also design a three-stage progressive training strategy that guides models from understanding tool outputs to proficient and adaptive tool invocation.
  arXiv  Detail & Related papers  (2026-04-08T06:28:03Z)
• GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learning [8.819428500071007]
  Vision-language pretraining models have made significant progress in bridging remote sensing imagery with natural language.<n>We propose GeoAlignCLIP, a unified framework that achieves fine-grained alignment in remote sensing tasks.<n>We construct RSFG-100k, a fine-granular remote sensing dataset containing scene descriptions, region-level annotations, and challenging hard-negative samples.
  arXiv  Detail & Related papers  (2026-03-10T12:12:11Z)
• Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining [59.2578488860426]
  Heterogeneous multi-modal remote sensing object detection aims to accurately detect objects from diverse sensors.<n>Existing approaches largely adopt a late alignment paradigm, in which modality alignment and task-specific optimization are entangled during downstream fine-tuning.<n>We propose BabelRS, a unified language-pivoted pretraining framework that explicitly decouples modality alignment from downstream task learning.
  arXiv  Detail & Related papers  (2026-03-02T11:38:12Z)
• Revisiting Multi-Task Visual Representation Learning [52.93947931352643]
  We introduce MTV, a principled multi-task visual pretraining framework.<n>We leverage high-capacity "expert" models to synthesize dense, structured pseudo-labels at scale.<n>Our results demonstrate that MTV achieves "best-of-both-worlds" performance.
  arXiv  Detail & Related papers  (2026-01-20T11:59:19Z)
• Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation [31.386822229629455]
  We propose Language-Guided Grasp Detection (LGGD) with a coarse-to-fine learning paradigm for robotic manipulation.<n>This design enables fine-grained visual-semantic alignment and improves the feasibility of the predicted grasps with respect to task instructions.<n>Experiments on the OCID-VLG and Grasp-Anything++ datasets show that LGGD surpasses existing language-guided grasping methods.
  arXiv  Detail & Related papers  (2025-12-24T09:16:42Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Hierarchical Language Models for Semantic Navigation and Manipulation in an Aerial-Ground Robotic System [8.88014241557266]
  Heterogeneous multirobot systems show great potential in complex tasks requiring coordinated hybrid cooperation.<n>Existing methods that rely on static or task-specific models often lack generalizability across diverse tasks and dynamic environments.<n>We propose a hierarchical multimodal framework that integrates a prompted large language model (LLM) with a fine-tuned vision-language model (VLM)
  arXiv  Detail & Related papers  (2025-06-05T13:27:41Z)
• Agentic 3D Scene Generation with Spatially Contextualized VLMs [67.31920821192323]
  We introduce a new paradigm that enables vision-language models to generate, understand, and edit complex 3D environments.<n>We develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context.<n>Results show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work.
  arXiv  Detail & Related papers  (2025-05-26T15:28:17Z)
• NuGrounding: A Multi-View 3D Visual Grounding Framework in Autonomous Driving [7.007334645975593]
  We introduce NuGrounding, the first large-scale benchmark for multi-view 3D visual grounding in autonomous driving.<n>We propose a novel paradigm that seamlessly combines instruction comprehension abilities of multi-modal LLMs with precise localization abilities of specialist detection models.
  arXiv  Detail & Related papers  (2025-03-28T13:55:16Z)
• Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation [69.01029651113386]
  Embodied-RAG is a framework that enhances the model of an embodied agent with a non-parametric memory system.<n>At its core, Embodied-RAG's memory is structured as a semantic forest, storing language descriptions at varying levels of detail.<n>We demonstrate that Embodied-RAG effectively bridges RAG to the robotics domain, successfully handling over 250 explanation and navigation queries.
  arXiv  Detail & Related papers  (2024-09-26T21:44:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.