Memory Over Maps: 3D Object Localization Without Reconstruction

• URL: http://arxiv.org/abs/2603.20530v1
• Date: Fri, 20 Mar 2026 21:57:51 GMT
• Title: Memory Over Maps: 3D Object Localization Without Reconstruction
• Abstract summary: Target localization is a prerequisite for embodied tasks such as navigation and manipulation.<n>Recent advances in vision-language models suggest that rich semantic reasoning can be performed directly on 2D observations.<n>We propose a map-free pipeline that stores only posed RGB-Ds as a lightweight visual memory without constructing any global 3D representation of the scene.
• Score: 44.29345996003297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Target localization is a prerequisite for embodied tasks such as navigation and manipulation. Conventional approaches rely on constructing explicit 3D scene representations to enable target localization, such as point clouds, voxel grids, or scene graphs. While effective, these pipelines incur substantial mapping time, storage overhead, and scalability limitations. Recent advances in vision-language models suggest that rich semantic reasoning can be performed directly on 2D observations, raising a fundamental question: is a complete 3D scene reconstruction necessary for object localization? In this work, we revisit object localization and propose a map-free pipeline that stores only posed RGB-D keyframes as a lightweight visual memory--without constructing any global 3D representation of the scene. At query time, our method retrieves candidate views, re-ranks them with a vision-language model, and constructs a sparse, on-demand 3D estimate of the queried target through depth backprojection and multi-view fusion. Compared to reconstruction-based pipelines, this design drastically reduces preprocessing cost, enabling scene indexing that is over two orders of magnitude faster to build while using substantially less storage. We further validate the localized targets on downstream object-goal navigation tasks. Despite requiring no task-specific training, our approach achieves strong performance across multiple benchmarks, demonstrating that direct reasoning over image-based scene memory can effectively replace dense 3D reconstruction for object-centric robot navigation. Project page: https://ruizhou-cn.github.io/memory-over-maps/

Related papers

• EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation [81.54723508469617]
  EvoMemNav is an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation.<n>VSMGraph organizes views with semantic cues and topological relations into a room-view-object hierarchy.<n>Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL.
  arXiv  Detail & Related papers  (2026-06-02T11:27:44Z)
• Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances [23.103482730965595]
  AFFORDMEM is a framework that grounds 3D functional affordances by remembering geometry at two levels.<n>As the agent processes the scene, it organizes candidate instances and their 3D spatial relations into a structured scene graph.<n>Our method improves AP50 over the prior training-free state of the art by 3.23 on Split 0 and 3.7 on Split 1.
  arXiv  Detail & Related papers  (2026-05-12T06:44:47Z)
• $L^3$:Scene-agnostic Visual Localization in the Wild [8.93640624067105]
  In this paper, we propose a novel map-free visual localization framework $L3$.<n> Specifically, by performing direct online 3D reconstruction on RGB images, followed by two-stage metric scale recovery and pose refinement based on 2D-3D correspondences, $L3$ achieves high accuracy without the need to pre-build or store any offline scene representations.
  arXiv  Detail & Related papers  (2026-03-09T04:04:24Z)
• ChangingGrounding: 3D Visual Grounding in Changing Scenes [92.00984845186679]
  Real-world robots localize objects from natural-language instructions while scenes around them keep changing.<n>Most of the existing 3D visual grounding (3DVG) method still assumes a reconstructed and up-to-date point cloud.<n>We introduce ChangingGrounding, the first benchmark that explicitly measures how well an agent can exploit past observations.
  arXiv  Detail & Related papers  (2025-10-16T17:59:16Z)
• Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory [72.75478398447396]
  We propose Point3R, an online framework targeting dense streaming 3D reconstruction.<n>To be specific, we maintain an explicit spatial pointer memory directly associated with the 3D structure of the current scene.<n>Our method achieves competitive or state-of-the-art performance on various tasks with low training costs.
  arXiv  Detail & Related papers  (2025-07-03T17:59:56Z)
• SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality [50.179377002092416]
  We propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches.
  arXiv  Detail & Related papers  (2024-09-21T08:46:16Z)
• Improved Scene Landmark Detection for Camera Localization [11.56648898250606]
  Method based on scene landmark detection (SLD) was recently proposed to address these limitations. It involves training a convolutional neural network (CNN) to detect a few predetermined, salient, scene-specific 3D points or landmarks. We show that the accuracy gap was due to insufficient model capacity and noisy labels during training.
  arXiv  Detail & Related papers  (2024-01-31T18:59:12Z)
• Lazy Visual Localization via Motion Averaging [89.8709956317671]
  We show that it is possible to achieve high localization accuracy without reconstructing the scene from the database. Experiments show that our visual localization proposal, LazyLoc, achieves comparable performance against state-of-the-art structure-based methods.
  arXiv  Detail & Related papers  (2023-07-19T13:40:45Z)
• Visual Localization using Imperfect 3D Models from the Internet [54.731309449883284]
  This paper studies how imperfections in 3D models affect localization accuracy. We show that 3D models from the Internet show promise as an easy-to-obtain scene representation.
  arXiv  Detail & Related papers  (2023-04-12T16:15:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.