Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction

• URL: http://arxiv.org/abs/2609.04201v1
• Date: Thu, 03 Sep 2026 17:59:53 GMT
• Title: Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction
• Abstract summary: Scal3R reformulates online reconstruction as multi-reference relative pose.<n>It reduces the average ATE by over 60% on KITTI compared to the online baseline.
• Score: 43.823514226720846
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online 3D reconstruction models perform poorly on long videos. This happens because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. Small drifts accumulate and amplify into significant geometric collapse. However, we observe that per-frame depth remains stable throughout this failure. The backbone's local geometry remains intact; only the global pose head breaks down. Motivated by this decoupling, we introduce Scal3R. This approach reformulates online reconstruction as multi-reference relative pose querying. We use lightweight learnable tokens, which make up about ~1% of the parameters, and inject them into a completely frozen backbone via asymmetric attention. This setup queries poses relative to multiple past keyframes. An online pose-graph optimization system with loop closure suppresses long-range drift. Scal3R reaches convergence in 8 hours on a single GPU. It reduces the average ATE by over 60% on KITTI compared to the online baseline. It also achieves state-of-the-art performance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes. Project page: https://linjohnss.github.io/scal3r/

Related papers

• $R^3$: 3D Reconstruction via Relative Regression [45.709068651036795]
  $R3$ supports both full-context offline reconstruction and causal, bounded-memory streaming.<n>Our evaluation in both offline and streaming settings validates the effectiveness of our relative mechanism.
  arXiv  Detail & Related papers  (2026-05-26T04:03:38Z)
• Amortized Inverse Kinematics via Graph Attention for Real-Time Human Avatar Animation [16.714345824701386]
  Inverse kinematics is a core operation in animation, robotics, and biomechanics.<n>We introduce IK-GAT, a lightweight graph-attention network that reconstructs full-body joint orientations from 3D joint positions in a single forward pass.<n>IK-GAT produces animation-ready local rotations that can directly drive a rigged avatar or be converted to pose parameters of SMPL-like body models.
  arXiv  Detail & Related papers  (2026-04-17T18:30:20Z)
• Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself [38.95274911975506]
  Free Geometry is a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth.<n>Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets.
  arXiv  Detail & Related papers  (2026-04-15T16:24:03Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation [16.22677555146353]
  Fin3R is a simple, effective, and general fine-tuning method for feed-forward 3D reconstruction models.<n>We validate our method on a wide range of models, including DUSt3R, MASt3R, CUT3R, and VGGT.
  arXiv  Detail & Related papers  (2025-11-27T13:10:19Z)
• Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction [153.52406455209538]
  Gamba is an end-to-end 3D reconstruction model from a single-view image. It completes reconstruction within 0.05 seconds on a single NVIDIA A100 GPU.
  arXiv  Detail & Related papers  (2024-03-27T17:40:14Z)
• PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction [77.89935657608926]
  We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing a 3D object from a few unposed images. PF-LRM simultaneously estimates the relative camera poses in 1.3 seconds on a single A100 GPU.
  arXiv  Detail & Related papers  (2023-11-20T18:57:55Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• Back to the Feature: Learning Robust Camera Localization from Pixels to Pose [114.89389528198738]
  We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching.
  arXiv  Detail & Related papers  (2021-03-16T17:40:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.