OMCL: Open-vocabulary Monte Carlo Localization

• URL: http://arxiv.org/abs/2512.15557v1
• Date: Wed, 17 Dec 2025 16:08:53 GMT
• Title: OMCL: Open-vocabulary Monte Carlo Localization
• Abstract summary: In this work, we extend Monte Carlo localization using vision-language features.<n>The abstract vision-language features enable to associate observations and map elements from different modalities.
• Score: 16.468655011980843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust robot localization is an important prerequisite for navigation planning. If the environment map was created from different sensors, robot measurements must be robustly associated with map features. In this work, we extend Monte Carlo Localization using vision-language features. These open-vocabulary features enable to robustly compute the likelihood of visual observations, given a camera pose and a 3D map created from posed RGB-D images or aligned point clouds. The abstract vision-language features enable to associate observations and map elements from different modalities. Global localization can be initialized by natural language descriptions of the objects present in the vicinity of locations. We evaluate our approach using Matterport3D and Replica for indoor scenes and demonstrate generalization on SemanticKITTI for outdoor scenes.

Related papers

• Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps [77.63233146945718]
  We introduce Cog3DMap, a framework that recurrently constructs an explicit 3D memory from multi-view images.<n>Our framework enables direct reasoning over a spatially structured 3D map, achieving state-of-the-art performance on various spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-24T10:05:32Z)
• VLG-Loc: Vision-Language Global Localization from Labeled Footprint Maps [5.851042749720534]
  A vision-language model (VLM) is used to search the robot's multi-directional image observations for the landmarks noted in the map.<n>The method then identifies robot poses within a Monte Carlo localization framework.<n> Experimental validation in simulated and real-world retail environments demonstrates superior robustness compared to existing scan-based methods.
  arXiv  Detail & Related papers  (2025-12-14T18:22:00Z)
• A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding [78.99798110890157]
  Open-vocabulary 3D visual grounding aims to localize target objects based on free-form language queries.<n>Existing language field methods struggle to accurately localize instances using spatial relations in language queries.<n>We propose SpatialReasoner, a novel neural representation-based framework with large language model (LLM)-driven spatial reasoning.
  arXiv  Detail & Related papers  (2025-07-09T10:20:38Z)
• Multimodal Spatial Language Maps for Robot Navigation and Manipulation [32.852583241593436]
  multimodal spatial language maps are a spatial map representation that fuses pretrained multimodal features with a 3D reconstruction of the environment.<n>We present two instances of our maps, which are visual-language maps (VLMaps) and their extension to audio-visual-language maps (AVLMaps)<n>These capabilities extend to mobile robots and tabletop manipulators, supporting navigation and interaction guided by visual, audio, and spatial cues.
  arXiv  Detail & Related papers  (2025-06-07T17:02:13Z)
• FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environment [16.987872206495897]
  FindAnything is an open-world mapping framework that incorporates vision-language information into dense volumetric submaps.<n>Our system is the first of its kind to be deployed on resource-constrained devices, such as MAVs.
  arXiv  Detail & Related papers  (2025-04-11T15:12:05Z)
• ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning [68.4209681278336]
  Open-vocabulary 3D visual grounding and reasoning aim to localize objects in a scene based on implicit language descriptions.<n>Current methods struggle because they rely heavily on fine-tuning with 3D annotations and mask proposals.<n>We propose ReasonGrounder, an LVLM-guided framework that uses hierarchical 3D feature Gaussian fields for adaptive grouping.
  arXiv  Detail & Related papers  (2025-03-30T03:40:35Z)
• Towards Localizing Structural Elements: Merging Geometrical Detection with Semantic Verification in RGB-D Data [0.0]
  This paper presents a real-time pipeline for localizing building components, including wall and ground surfaces, by integrating geometric calculations for pure 3D plane detection. It has a parallel multi-thread architecture to precisely estimate poses and equations of all the planes detected in the environment, filters the ones forming the map structure using a panoptic segmentation validation, and keeps only the validated building components. It can also ensure (re-)association of these detected components into a unified 3D scene graph, bridging the gap between geometric accuracy and semantic understanding.
  arXiv  Detail & Related papers  (2024-09-10T16:28:09Z)
• Pixel Aligned Language Models [94.32841818609914]
  We develop a vision-language model that can take locations as either inputs or outputs. When taking locations as inputs, the model performs location-conditioned captioning, which generates captions for the indicated object or region. Our model is pre-trained on the Localized Narrative dataset, which contains pixel-word-aligned captioning from human attention.
  arXiv  Detail & Related papers  (2023-12-14T18:57:58Z)
• Neural Implicit Dense Semantic SLAM [83.04331351572277]
  We propose a novel RGBD vSLAM algorithm that learns a memory-efficient, dense 3D geometry, and semantic segmentation of an indoor scene in an online manner. Our pipeline combines classical 3D vision-based tracking and loop closing with neural fields-based mapping. Our proposed algorithm can greatly enhance scene perception and assist with a range of robot control problems.
  arXiv  Detail & Related papers  (2023-04-27T23:03:52Z)
• Visual Language Maps for Robot Navigation [30.33041779258644]
  Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data. We propose VLMaps, a spatial map representation that directly fuses pretrained visual-language features with a 3D reconstruction of the physical world.
  arXiv  Detail & Related papers  (2022-10-11T18:13:20Z)
• Semantic Image Alignment for Vehicle Localization [111.59616433224662]
  We present a novel approach to vehicle localization in dense semantic maps using semantic segmentation from a monocular camera. In contrast to existing visual localization approaches, the system does not require additional keypoint features, handcrafted localization landmark extractors or expensive LiDAR sensors.
  arXiv  Detail & Related papers  (2021-10-08T14:40:15Z)
• LanguageRefer: Spatial-Language Model for 3D Visual Grounding [72.7618059299306]
  We develop a spatial-language model for a 3D visual grounding problem. We show that our model performs competitively on visio-linguistic datasets proposed by ReferIt3D.
  arXiv  Detail & Related papers  (2021-07-07T18:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.