Seeing Through the Weights: Privacy Leakage in Scene Coordinate Regression

• URL: http://arxiv.org/abs/2606.31164v1
• Date: Tue, 30 Jun 2026 05:54:23 GMT
• Title: Seeing Through the Weights: Privacy Leakage in Scene Coordinate Regression
• Abstract summary: Scene Coordinate Regression (SCR) methods are increasingly adopted for visual localization.<n>Because the scene is represented only through the network parameters and not stored explicitly as images or maps, such methods are often assumed to be privacy-preserving.<n>We introduce a query-based attack that reconstructs the 3D geometry of the training environment from an SCR model under different levels of model access.<n>This directly contradicts claims in the literature that SCR representations are privacy-preserving by design, and reveals a real risk when such systems are deployed in private or security-critical spaces.
• Score: 9.85962651843239
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scene Coordinate Regression (SCR) methods are increasingly adopted for visual localization. In these approaches, the scene is implicitly encoded within a neural network that regresses a 3D world coordinate for each image pixel. Because the scene is represented only through the network parameters and not stored explicitly as images or maps, such methods are often assumed to be privacy-preserving. In this work, we show that this assumption is incorrect in practice. Specifically, we introduce a query-based attack that reconstructs the 3D geometry of the training environment from an SCR model under different levels of model access. To do so, we repeatedly query the model with batches of proxy images unrelated to the target scene to obtain dense pixel-wise 3D coordinates. Reliable points are identified through their stability under small input perturbations and can be further refined in a white-box setting. These stable points are accumulated across independent query batches to recover the scene geometry. From the recovered 3D representation, we also invert the network features to synthesize images from arbitrary viewpoints, revealing additional appearance information. Experiments on indoor and outdoor datasets demonstrate that substantial portions of training environments can be reconstructed with high geometric fidelity. Beyond geometry, we also recover an approximate color appearance, which exposes recognizable layout and potentially sensitive scene elements. This directly contradicts claims in the literature that SCR representations are privacy-preserving by design, and reveals a real risk when such systems are deployed in private or security-critical spaces. The project page is available at https://jaeminch0.github.io/seeing-through-the-weights-privacy-leakage-in-scene-coordinate-regressio n.

Related papers

• HGGT: Robust and Flexible 3D Hand Mesh Reconstruction from Uncalibrated Images [81.42866295265443]
  High-fidelity 3D hand geometry is a critical task in computer vision.<n> scalable applications demand both accuracy and deployment flexibility.<n>We propose a feed-forward architecture that infers 3D hand meshes and camera poses from uncalibrated views.
  arXiv  Detail & Related papers  (2026-03-25T06:54:34Z)
• Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory [72.75478398447396]
  We propose Point3R, an online framework targeting dense streaming 3D reconstruction.<n>To be specific, we maintain an explicit spatial pointer memory directly associated with the 3D structure of the current scene.<n>Our method achieves competitive or state-of-the-art performance on various tasks with low training costs.
  arXiv  Detail & Related papers  (2025-07-03T17:59:56Z)
• A-SCoRe: Attention-based Scene Coordinate Regression for wide-ranging scenarios [1.2093553114715083]
  A-ScoRe is an Attention-based model which leverage attention on descriptor map level to produce meaningful and high-semantic 2D descriptors.<n>Results show our methods achieve comparable performance with State-of-the-art methods on multiple benchmark while being light-weighted and much more flexible.
  arXiv  Detail & Related papers  (2025-03-18T07:39:50Z)
• Obfuscation Based Privacy Preserving Representations are Recoverable Using Neighborhood Information [43.14648227833812]
  Research on privacy-preserving localization has focused on preventing inversion attacks on query image keypoints and the 3D points of the scene map.<n>We point to a common weakness of these obfuscations that allows to recover approximations of the original point positions under the assumption of known neighborhoods.<n>Our results show that these schemes should not be considered privacy-preserving, even though they are claimed to be privacy-preserving.
  arXiv  Detail & Related papers  (2024-09-17T20:13:54Z)
• Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement [32.335953514942474]
  This paper proposes to jointly learn the scene representation along with a 3D dense feature field and a 2D feature extractor. We learn the underlying geometry of the scene with an implicit field through volumetric rendering and design our feature field to leverage intermediate geometric information encoded in the implicit field. Visual localization is then achieved by aligning the image-based features and the rendered volumetric features.
  arXiv  Detail & Related papers  (2024-06-12T17:51:53Z)
• GLACE: Global Local Accelerated Coordinate Encoding [66.87005863868181]
  Scene coordinate regression methods are effective in small-scale scenes but face significant challenges in large-scale scenes. We propose GLACE, which integrates pre-trained global and local encodings and enables SCR to scale to large scenes with only a single small-sized network. Our method achieves state-of-the-art results on large-scale scenes with a low-map-size model.
  arXiv  Detail & Related papers  (2024-06-06T17:59:50Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• SACReg: Scene-Agnostic Coordinate Regression for Visual Localization [16.866303169903237]
  We propose a generalized SCR model trained once in new test scenes, regardless of their scale, without any finetuning. Instead of encoding the scene coordinates into the network weights, our model takes as input a database image with some sparse 2D pixel to 3D coordinate annotations. We show that the database representation of images and their 2D-3D annotations can be highly compressed with negligible loss of localization performance.
  arXiv  Detail & Related papers  (2023-07-21T16:56:36Z)
• HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer [23.920690073252636]
  We present a new hierarchical scene coordinate network to predict pixel scene coordinates in a coarse-to-fine manner from a single RGB image. The proposed method, which is an extension of HSCNet, allows us to train compact models which scale robustly to large environments.
  arXiv  Detail & Related papers  (2023-05-05T15:00:14Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• Back to the Feature: Learning Robust Camera Localization from Pixels to Pose [114.89389528198738]
  We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching.
  arXiv  Detail & Related papers  (2021-03-16T17:40:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.