OVI-MAP:Open-Vocabulary Instance-Semantic Mapping

• URL: http://arxiv.org/abs/2603.26541v1
• Date: Fri, 27 Mar 2026 15:50:59 GMT
• Title: OVI-MAP:Open-Vocabulary Instance-Semantic Mapping
• Abstract summary: OVI-MAP is a class-agnostic 3D instance map that is incrementally constructed from RGB-D input.<n> semantic features are extracted only from a small set of automatically selected views using vision-language models.<n>Our system operates in real time and outperforms state-of-the-art open-vocabulary mapping baselines on standard benchmarks.
• Score: 108.66131262110095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Incremental open-vocabulary 3D instance-semantic mapping is essential for autonomous agents operating in complex everyday environments. However, it remains challenging due to the need for robust instance segmentation, real-time processing, and flexible open-set reasoning. Existing methods often rely on the closed-set assumption or dense per-pixel language fusion, which limits scalability and temporal consistency. We introduce OVI-MAP that decouples instance reconstruction from semantic inference. We propose to build a class-agnostic 3D instance map that is incrementally constructed from RGB-D input, while semantic features are extracted only from a small set of automatically selected views using vision-language models. This design enables stable instance tracking and zero-shot semantic labeling throughout online exploration. Our system operates in real time and outperforms state-of-the-art open-vocabulary mapping baselines on standard benchmarks.

Related papers

• ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation [36.94429692322632]
  We present ENEAS, a text-promptable method for instance tracking and semantic discovery.<n>We extend the geometrically robust SeC architecture, previously limited to point interactions, with a text-prompting adapter.<n>The verification layer combines high-speed visual embedding matching with conditional VLM refinement, invoking semantic reasoning only for ambiguous candidates.
  arXiv  Detail & Related papers  (2026-09-03T12:25:26Z)
• dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3 [36.6036728217708]
  Open-Vocabulary Semantics (OVSS) assigns pixel-level labels from an open set of text-defined categories, demanding reliable generalization to unseen classes at inference.<n>We introduce dinov3.seg, extending dinov3.txt into a dedicated framework for OVSS.<n>Our contributions are four-fold. First, we design a task-specific architecture tailored to this backbone, systematically adapting established design principles from prior open-vocabulary segmentation work.<n>Second, we jointly leverage text embeddings aligned with both the global [] token and local patch-level visual features from ViT-based encoder
  arXiv  Detail & Related papers  (2026-03-19T23:57:28Z)
• AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis [50.793806818677716]
  AffordGrasp produces physically stable and semantically faithful human grasps with high precision.<n>AffordGrasp integrates an affordance-aware latent representation of hand poses with a dual-conditioning diffusion process.<n>We evaluate AffordGrasp across four instruction-augmented benchmarks derived from HO-3D, OakInk, GRAB, and AffordPose.
  arXiv  Detail & Related papers  (2026-03-09T06:56:35Z)
• DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping [5.520073359436354]
  Open-set semantic mapping enables language-driven robotic perception.<n>Current instance-centric approaches are bottlenecked by context-depriving and computationally expensive crop-based feature extraction.<n>We introduce DISC (Dense Integrated Semantic Context), featuring a novel single-pass, distance-weighted extraction mechanism.
  arXiv  Detail & Related papers  (2026-03-04T10:47:06Z)
• VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection [67.09867723723934]
  3D semantic occupancy prediction aims to reconstruct the 3D geometry and semantics of the surrounding environment.<n>With dense voxel labels, prior works typically formulate it as a dense segmentation task, independently classifying each voxel.<n>We propose VoxDet, an instance-centric framework that reformulates the voxel-level occupancy prediction as dense object detection.
  arXiv  Detail & Related papers  (2025-06-05T04:31:55Z)
• OpenFusion++: An Open-vocabulary Real-time Scene Understanding System [4.470499157873342]
  We present OpenFusion++, a TSDF-based real-time 3D semantic-geometric reconstruction system.<n>Our approach refines 3D point clouds by fusing confidence maps from foundational models, dynamically updates global semantic labels via an adaptive cache based on instance area, and employs a dual-path encoding framework.<n>Experiments on the ICL, Replica, ScanNet, and ScanNet++ datasets demonstrate that OpenFusion++ significantly outperforms the baseline in both semantic accuracy and query responsiveness.
  arXiv  Detail & Related papers  (2025-04-27T14:46:43Z)
• EgoSplat: Open-Vocabulary Egocentric Scene Understanding with Language Embedded 3D Gaussian Splatting [108.15136508964011]
  EgoSplat is a language-embedded 3D Gaussian Splatting framework for open-vocabulary egocentric scene understanding.<n>EgoSplat achieves state-of-the-art performance in both localization and segmentation tasks on two datasets.
  arXiv  Detail & Related papers  (2025-03-14T12:21:26Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• FM-Fusion: Instance-aware Semantic Mapping Boosted by Vision-Language Foundation Models [24.77953131753715]
  Development of vision-language foundation models demonstrates a strong zero-shot transferability across data distribution. We propose a probabilistic label fusion method to predict close-set semantic classes from open-set label measurements. We integrate all the modules into a unified semantic mapping system. Reading a sequence of RGB-D input, our work incrementally reconstructs an instance-aware semantic map.
  arXiv  Detail & Related papers  (2024-02-07T03:19:02Z)
• LISNeRF Mapping: LiDAR-based Implicit Mapping via Semantic Neural Fields for Large-Scale 3D Scenes [2.822816116516042]
  Large-scale semantic mapping is crucial for outdoor autonomous agents to fulfill high-level tasks such as planning and navigation. This paper proposes a novel method for large-scale 3D semantic reconstruction through implicit representations from posed LiDAR measurements alone.
  arXiv  Detail & Related papers  (2023-11-04T03:55:38Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.