ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training

• URL: http://arxiv.org/abs/2510.11605v1
• Date: Mon, 13 Oct 2025 16:45:17 GMT
• Title: ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training
• Authors: Leonard Bruns, Axel Barroso-Laguna, Tommaso Cavallari, Áron Monszpart, Sowmya Munukutla, Victor Adrian Prisacariu, Eric Brachmann,
• Abstract summary: Scene coordinate regression (SCR) has established itself as a promising learning-based approach to visual relocalization.<n>We propose to separate the coordinate regressor and the map representation into a generic transformer and a scene-specific map code.<n>This separation allows us to pre-train the transformer on tens of thousands of scenes.
• Score: 26.573873458594303
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scene coordinate regression (SCR) has established itself as a promising learning-based approach to visual relocalization. After mere minutes of scene-specific training, SCR models estimate camera poses of query images with high accuracy. Still, SCR methods fall short of the generalization capabilities of more classical feature-matching approaches. When imaging conditions of query images, such as lighting or viewpoint, are too different from the training views, SCR models fail. Failing to generalize is an inherent limitation of previous SCR frameworks, since their training objective is to encode the training views in the weights of the coordinate regressor itself. The regressor essentially overfits to the training views, by design. We propose to separate the coordinate regressor and the map representation into a generic transformer and a scene-specific map code. This separation allows us to pre-train the transformer on tens of thousands of scenes. More importantly, it allows us to train the transformer to generalize from mapping images to unseen query images during pre-training. We demonstrate on multiple challenging relocalization datasets that our method, ACE-G, leads to significantly increased robustness while keeping the computational footprint attractive.

Related papers

• Scene Coordinate Reconstruction Priors [29.668748429324154]
  Scene coordinate regression (SCR) models have proven to be powerful implicit scene representations for 3D vision.<n>We present a probabilistic reinterpretation of training SCR models, which allows us to infuse high-level reconstruction priors.<n>Our priors help learning better scene representations, resulting in more coherent scene point clouds, higher registration rates and better camera poses.
  arXiv  Detail & Related papers  (2025-10-14T11:13:31Z)
• HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis [59.25751939710903]
  We propose a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos.<n>Our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately.<n>Results highlight a practical and scalable solution for EVS in real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-24T03:54:40Z)
• Rethinking Visual Content Refinement in Low-Shot CLIP Adaptation [31.023236232633213]
  Recent adaptations can boost the low-shot capability of Contrastive Vision-Language Pre-training. We propose a Visual Content Refinement (VCR) before the adaptation calculation during the test stage. We apply our method to 3 popular low-shot benchmark tasks with 13 datasets and achieve a significant improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-19T08:34:23Z)
• Breaking the Frame: Visual Place Recognition by Overlap Prediction [53.17564423756082]
  We propose a novel visual place recognition approach based on overlap prediction, called VOP.<n>VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone.<n>Our approach uses a voting mechanism to assess overlap scores for potential database images.
  arXiv  Detail & Related papers  (2024-06-23T20:00:20Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency [32.46493952272438]
  State-of-the-art visual localization approaches rely on a first image retrieval step. We improve this retrieval step and tailor it to the final localization task. We experimentally show that those changes translate into large improvements for the most challenging visual localization datasets.
  arXiv  Detail & Related papers  (2024-02-14T15:24:20Z)
• Vision-by-Language for Training-Free Compositional Image Retrieval [78.60509831598745]
  Compositional Image Retrieval (CIR) aims to retrieve the relevant target image in a database. Recent research sidesteps this need by using large-scale vision-language models (VLMs) We propose to tackle CIR in a training-free manner via Vision-by-Language (CIReVL)
  arXiv  Detail & Related papers  (2023-10-13T17:59:38Z)
• SACReg: Scene-Agnostic Coordinate Regression for Visual Localization [16.866303169903237]
  We propose a generalized SCR model trained once in new test scenes, regardless of their scale, without any finetuning. Instead of encoding the scene coordinates into the network weights, our model takes as input a database image with some sparse 2D pixel to 3D coordinate annotations. We show that the database representation of images and their 2D-3D annotations can be highly compressed with negligible loss of localization performance.
  arXiv  Detail & Related papers  (2023-07-21T16:56:36Z)
• Learning to Localize in Unseen Scenes with Relative Pose Regressors [5.672132510411465]
  Relative pose regressors (RPRs) localize a camera by estimating its relative translation and rotation to a pose-labelled reference. In practice, however, the performance of RPRs is significantly degraded in unseen scenes. We implement aggregation with concatenation, projection, and attention operations (Transformers) and learn to regress the relative pose parameters from the resulting latent codes. Compared to state-of-the-art RPRs, our model is shown to localize significantly better in unseen environments, across both indoor and outdoor benchmarks, while maintaining competitive performance in seen scenes.
  arXiv  Detail & Related papers  (2023-03-05T17:12:50Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• Leveraging Image Matching Toward End-to-End Relative Camera Pose Regression [13.233301155616616]
  This paper proposes a generalizable, end-to-end deep learning-based method for relative pose regression between two images. Inspired by the classical pipeline, our method leverages Image Matching (IM) as a pre-trained task for relative pose regression. We evaluate our method on several datasets and show that it outperforms previous end-to-end methods.
  arXiv  Detail & Related papers  (2022-11-27T22:01:47Z)
• Pretraining is All You Need for Image-to-Image Translation [59.43151345732397]
  We propose to use pretraining to boost general image-to-image translation. We show that the proposed pretraining-based image-to-image translation (PITI) is capable of synthesizing images of unprecedented realism and faithfulness.
  arXiv  Detail & Related papers  (2022-05-25T17:58:26Z)
• End-to-End Trainable Multi-Instance Pose Estimation with Transformers [68.93512627479197]
  We propose a new end-to-end trainable approach for multi-instance pose estimation by combining a convolutional neural network with a transformer. Inspired by recent work on end-to-end trainable object detection with transformers, we use a transformer encoder-decoder architecture together with a bipartite matching scheme to directly regress the pose of all individuals in a given image. Our model, called POse Estimation Transformer (POET), is trained using a novel set-based global loss that consists of a keypoint loss, a keypoint visibility loss, a center loss and a class loss.
  arXiv  Detail & Related papers  (2021-03-22T18:19:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.