Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization

• URL: http://arxiv.org/abs/2606.24767v1
• Date: Tue, 23 Jun 2026 16:27:04 GMT
• Title: Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization
• Abstract summary: OpenReLoc is a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities.<n>We propose a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching.<n> Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets.
• Score: 93.79141221821858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.

Related papers

• Direct 3D-Aware Object Insertion via Decomposed Visual Proxies [101.76006356727015]
  DIRECT (Decomposed Injection for Reference Composition and Target-integration) is a novel framework that integrates interactive pose manipulation with high-fidelity 2D image synthesis.<n>Our method decomposes the insertion conditions into three complementary components: appearance guidance capturing visual details from the reference object, geometry guidance derived from the user-adjusted 3D proxy, and context guidance from the target background.<n>Experiments show that DIRECT outperforms previous methods in both geometric controllability and visual quality.
  arXiv  Detail & Related papers  (2026-06-04T18:00:34Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimation [81.24160191975781]
  This paper presents a general RGB-based approach for object pose estimation, specifically designed to address challenges in sparse-view settings.<n>To overcome these limitations, we introduce corner points of the object bounding box as an intermediate representation of the object pose.<n>The 3D object corners can be reliably recovered from sparse input views, while the 2D corner points in the target view are estimated through a novel reference-based point datasets.
  arXiv  Detail & Related papers  (2025-04-10T17:58:35Z)
• Hierarchical Cross-Modal Alignment for Open-Vocabulary 3D Object Detection [45.68105299990119]
  Open-vocabulary 3D object detection (OV-3DOD) aims at localizing and classifying novel objects beyond closed sets.<n>We propose a hierarchical framework, named HCMA, to simultaneously learn local object and global scene information for OV-3DOD.
  arXiv  Detail & Related papers  (2025-03-10T17:55:22Z)
• Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description [56.69740649781989]
  3D scene understanding is a long-standing challenge in computer vision and a key component in enabling mixed reality, wearable computing, and embodied AI.<n>We introduce Articulate3D, an expertly curated 3D dataset featuring high-quality manual annotations on 280 indoor scenes.<n>We also present USDNet, a novel unified framework capable of simultaneously predicting part segmentation along with a full specification of motion attributes for articulated objects.
  arXiv  Detail & Related papers  (2024-12-02T11:33:55Z)
• LidaRefer: Context-aware Outdoor 3D Visual Grounding for Autonomous Driving [1.0589208420411014]
  3D visual grounding aims to locate objects or regions within 3D scenes guided by natural language descriptions.<n>Large-scale outdoor LiDAR scenes are dominated by background points and contain limited foreground information.<n>LidaRefer is a context-aware 3D VG framework for outdoor scenes.
  arXiv  Detail & Related papers  (2024-11-07T01:12:01Z)
• Semantic Object-level Modeling for Robust Visual Camera Relocalization [14.998133272060695]
  We propose a novel method of automatic object-level voxel modeling for accurate ellipsoidal representations of objects. All of these modules are entirely intergrated into visual SLAM system.
  arXiv  Detail & Related papers  (2024-02-10T13:39:44Z)
• CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection [57.44434974289945]
  We propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework. Our framework takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene. In addition to 3D object detection, we investigate the effectiveness of our framework for the problem of 3D object counting.
  arXiv  Detail & Related papers  (2022-09-13T05:26:09Z)
• Sim2Real Object-Centric Keypoint Detection and Description [40.58367357980036]
  Keypoint detection and description play a central role in computer vision. We propose the object-centric formulation, which requires further identifying which object each interest point belongs to. We develop a sim2real contrastive learning mechanism that can generalize the model trained in simulation to real-world applications.
  arXiv  Detail & Related papers  (2022-02-01T15:00:20Z)
• Compositional Scalable Object SLAM [29.349829139625403]
  We present a fast, scalable, and accurate Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of objects. We show that a compositional scalable object mapping formulation is amenable to a robust SLAM solution for drift-free large scale indoor reconstruction.
  arXiv  Detail & Related papers  (2020-11-05T04:46:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.