Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics

• URL: http://arxiv.org/abs/2606.24814v2
• Date: Tue, 30 Jun 2026 07:10:41 GMT
• Title: Vision-Language Model Reasoning for Contextual Semantic Mapping in Intralogistics
• Abstract summary: We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, and VLM multi-view reasoning.<n>The pipeline infers contextual object properties without requiring task-specific training or predefined object categories.<n>The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.
• Score: 2.8218290601001854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.

Related papers

• SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models [63.28435103335999]
  We introduce SOCO, a new benchmark for Semantic Object Correspondence.<n>We show that vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories.<n>We also show that LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching.
  arXiv  Detail & Related papers  (2026-05-29T17:58:48Z)
• GaLa: Hypergraph-Guided Visual Language Models for Procedural Planning [14.265218749993956]
  Implicit spatial relations and deep semantic structures encoded in object attributes are crucial for procedural planning in embodied AI systems.<n>We propose GaLa, a vision language framework for multimodal procedural planning.<n>We show that GaLa significantly outperforms existing methods in terms of execution success rate, LCS, and planning correctness.
  arXiv  Detail & Related papers  (2026-04-19T04:04:02Z)
• OVI-MAP:Open-Vocabulary Instance-Semantic Mapping [108.66131262110095]
  OVI-MAP is a class-agnostic 3D instance map that is incrementally constructed from RGB-D input.<n> semantic features are extracted only from a small set of automatically selected views using vision-language models.<n>Our system operates in real time and outperforms state-of-the-art open-vocabulary mapping baselines on standard benchmarks.
  arXiv  Detail & Related papers  (2026-03-27T15:50:59Z)
• CMOMgen: Complex Multi-Ontology Alignment via Pattern-Guided In-Context Learning [0.45880283710344066]
  Complex multi-ontology matching (CMOM) aligns one source entity to composite logical expressions of multiple target entities.<n>We present CMOMgen, the first end-to-end strategy that generates semantic mappings without establishing any restrictions on the number of target or entities.
  arXiv  Detail & Related papers  (2025-10-24T17:12:22Z)
• LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance [54.683384204063934]
  Large multi-modal models (LMMs) struggle with inaccurate segmentation and hallucinated comprehension.<n>We propose LIRA, a framework that capitalizes on the complementary relationship between visual comprehension and segmentation.<n>LIRA achieves state-of-the-art performance in both segmentation and comprehension tasks.
  arXiv  Detail & Related papers  (2025-07-08T07:46:26Z)
• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework that learns spatial-semantic features and discriminative object queries.<n>The proposed method achieves state-of-the-art performance on benchmark data sets, including the DAVIS 2017 test (textbf87.8%), YoutubeVOS 2019 (textbf88.1%), MOSE val (textbf74.0%), and LVOS test (textbf73.0%)
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• Exploiting Contextual Target Attributes for Target Sentiment Classification [53.30511968323911]
  Existing PTLM-based models for TSC can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. We present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes.
  arXiv  Detail & Related papers  (2023-12-21T11:45:28Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)
• An Object SLAM Framework for Association, Mapping, and High-Level Tasks [12.62957558651032]
  We present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. A range of public datasets and real-world results have been used to evaluate the proposed object SLAM framework for its efficient performance.
  arXiv  Detail & Related papers  (2023-05-12T08:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.