ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction

• URL: http://arxiv.org/abs/2607.05356v1
• Date: Mon, 06 Jul 2026 17:35:18 GMT
• Title: ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction
• Abstract summary: Streaming 3D reconstruction relies on a compact recurrent scene state to process long image streams in linear time and bounded memory.<n>We introduce ReCal3R, a reliability-calibrated learning rate method for recurrent 3D reconstruction.
• Score: 57.10941470313318
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming 3D reconstruction relies on a compact recurrent scene state to process long image streams in linear time and bounded memory. However, repeated updates can gradually corrupt this state, causing reliable historical information to be overwritten by noisy or ambiguous observations. We introduce ReCal3R, a reliability-calibrated learning rate method for recurrent 3D reconstruction. Instead of directly applying a candidate learning rate, our method estimates state token reliability from the maintained scene state and uses it to calibrate a candidate learning rate derived from token alignment, state reconstruction residual, and recent update pressure. The resulting token-wise learning rate interpolates between a conservative base rate and the candidate rate, suppressing aggressive updates on unreliable tokens while preserving adaptation to informative frames. Applied to CUT3R as a training-free calibration rule, ReCal3R reaches strong performance on long sequences in pose, depth, and reconstruction quality, including a 3.7$\times$ reduction in ATE, with comparable runtime and memory. Code is available at: https://github.com/Powertony102/ReCal3R.

Related papers

• Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction [10.383154504465232]
  ABot-Recon is a simple streaming model that caches KV features from only the preceding 11 frames.<n>It predicts a point map in the current camera coordinate system together with an adjacent-frame relative pose.<n>To reduce accumulated drift, a lightweight temporal refiner improves relative rotations using recent visual and motion context.
  arXiv  Detail & Related papers  (2026-08-27T15:07:38Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• Mem3R: Streaming 3D Reconstruction with Hybrid Memory via Test-Time Training [8.037339305506656]
  We propose Mem3R, a streaming 3D reconstruction model with a hybrid memory design.<n>For camera tracking, Mem3R employs an implicit fast-weight memory implemented as a lightweight Multi-Layer Perceptron.<n>For geometric mapping, Mem3R maintains an explicit token-based fixed-size state.
  arXiv  Detail & Related papers  (2026-04-08T16:41:22Z)
• FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction [51.56484100374058]
  Streaming 3D reconstruction maintains a persistent latent state that is updated online from incoming frames.<n>FILT3R casts recurrent state updates as state estimation in token space.<n>Code will be released at https://github.com/jinotter3/FILT3R.
  arXiv  Detail & Related papers  (2026-03-19T04:56:36Z)
• SSR: A Training-Free Approach for Streaming 3D Reconstruction [53.19807901585702]
  Self-expressive Sequence Regularization (SSR) is a plug-and-play operator that enforces Grassmannian sequence regularity during inference.<n>We show that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-03-16T02:55:14Z)
• FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning [63.20028888397869]
  FOREVER (FORgEtting curVe-inspired mEmory) is a novel framework that aligns replay schedules with a model-centric notion of time.<n>Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay.
  arXiv  Detail & Related papers  (2026-01-07T13:55:14Z)
• TTT3R: 3D Reconstruction as Test-Time Training [69.51086319339662]
  We revisit the 3D reconstruction foundation models from a Test-Time Training perspective.<n>We leverage the alignment confidence between the memory state and incoming observations to derive a closed-form learning rate.<n>This training-free intervention, termed TTT3R, substantially improves length generalization.
  arXiv  Detail & Related papers  (2025-09-30T17:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.