FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction

• URL: http://arxiv.org/abs/2603.18493v1
• Date: Thu, 19 Mar 2026 04:56:36 GMT
• Title: FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction
• Abstract summary: Streaming 3D reconstruction maintains a persistent latent state that is updated online from incoming frames.<n>FILT3R casts recurrent state updates as state estimation in token space.<n>Code will be released at https://github.com/jinotter3/FILT3R.
• Score: 51.56484100374058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming 3D reconstruction maintains a persistent latent state that is updated online from incoming frames, enabling constant-memory inference. A key failure mode is the state update rule: aggressive overwrites forget useful history, while conservative updates fail to track new evidence, and both behaviors become unstable beyond the training horizon. To address this challenge, we propose FILT3R, a training-free latent filtering layer that casts recurrent state updates as stochastic state estimation in token space. FILT3R maintains a per-token variance and computes a Kalman-style gain that adaptively balances memory retention against new observations. Process noise -- governing how much the latent state is expected to change between frames -- is estimated online from EMA-normalized temporal drift of candidate tokens. Using extensive experiments, we demonstrate that FILT3R yields an interpretable, plug-in update rule that generalizes common overwrite and gating policies as special cases. Specifically, we show that gains shrink in stable regimes as uncertainty contracts with accumulated evidence, and rise when genuine scene change increases process uncertainty, improving long-horizon stability for depth, pose, and 3D reconstruction, compared to the existing methods. Code will be released at https://github.com/jinotter3/FILT3R.

Related papers

• Separating Stream Stability from Long-Term Recall in Language Models [14.319140213074045]
  An attention sink can stabilize autoregressive generation over an indefinitely long stream while the model remains unable to use content that has left its recent-token cache.<n>We introduce three horizons: the stability horizon, over which predictive behavior remains well behaved; the access horizon, over which past content can still causally affect the output; and the utility horizon, over which a task retains acceptable performance.<n>Applying the framework to attention-sink streaming clarifies its strength, constant-memory, stable generation, without treating anchor tokens as semantic memory.
  arXiv  Detail & Related papers  (2026-09-07T09:49:32Z)
• ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction [57.10941470313318]
  Streaming 3D reconstruction relies on a compact recurrent scene state to process long image streams in linear time and bounded memory.<n>We introduce ReCal3R, a reliability-calibrated learning rate method for recurrent 3D reconstruction.
  arXiv  Detail & Related papers  (2026-07-06T17:35:18Z)
• Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation [89.70897512515477]
  Action-controlled image-to-video generation is a promising paradigm for interactive world simulation, where each control signal should elicit an immediate visual response.<n>Maintaining visual fidelity and 3D consistency over long autoregressive rollouts remains challenging.<n>Existing 3D-aware methods often suffer from catastrophic drift due to two impediments: information loss from textitLatent--RGB Cycling, and the training--inference gap induced by the textiterror-free hypothesis.<n>We present textbfRobust Dreamer, a memory-augmented framework built around how to
  arXiv  Detail & Related papers  (2026-05-29T05:21:33Z)
• Learned Memory Attenuation in Sage-Husa Kalman Filters for Robust UAV State Estimation [0.0]
  Unmanned Aerial Vehicles in dynamic environments face telemetry outages, structural vibrations, and regime-dependent noise.<n>We introduce the N-Deep Recurrent Sage-Husa Filter (NDR-SHKF), which replaces the scalar parameter with a vector-valued memory attenuation policy.<n>A bifurcated architecture routes shallow recurrent states to capture instantaneous sensor anomalies and deep states to encode sustained dynamic trends.
  arXiv  Detail & Related papers  (2026-05-18T17:38:13Z)
• SSR: A Training-Free Approach for Streaming 3D Reconstruction [53.19807901585702]
  Self-expressive Sequence Regularization (SSR) is a plug-and-play operator that enforces Grassmannian sequence regularity during inference.<n>We show that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-03-16T02:55:14Z)
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising [52.237486207964245]
  HiAR is a hierarchical denoising framework that reverses the conventional generation order.<n>It performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level.<n>On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
  arXiv  Detail & Related papers  (2026-03-09T17:58:16Z)
• OnlineX: Unified Online 3D Reconstruction and Understanding with Active-to-Stable State Evolution [34.8105632078785]
  We introduce OnlineX, a feed-forward framework that reconstructs both 3D visual appearance and language fields in an online manner using only streaming images.<n>Our framework decouples the memory state into a dedicated active state and a persistent stable state, and then cohesively fuses the information from the former into the latter to achieve both fidelity and stability.
  arXiv  Detail & Related papers  (2026-03-02T17:52:02Z)
• TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction [11.124244013253806]
  Streaming recurrent models enable efficient 3D reconstruction by maintaining persistent state representations.<n>Recent methods alleviate this by deriving adaptive signals from attention perspective.<n>We propose a training-free framework termed TTSA3R that leverages both temporal state evolution and spatial observation quality.
  arXiv  Detail & Related papers  (2026-01-30T06:14:42Z)
• Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression [53.48692193399171]
  Gated KalmaNet (GKA) is a layer that reduces the gap by accounting for the full past when predicting the next token.<n>We solve an online ridge regression problem at test time, with constant memory and linear compute cost in the sequence length.<n>On long-context, GKA excels at real-world RAG and LongQA tasks up to 128k tokens, achieving more than $10$% relative improvement over other fading memory baselines.
  arXiv  Detail & Related papers  (2025-11-26T03:26:37Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• Stable Video Infinity: Infinite-Length Video Generation with Error Recycling [76.91310169118408]
  We propose Stable Video Infinity (SVI) that is able to generate infinite-length videos with high temporal consistency, plausible scene transitions, and controllable streaming storylines.<n> SVI incorporates Error-Recycling Fine-Tuning, a new type of efficient training that recycles the Diffusion Transformer's self-generated errors into supervisory prompts.<n>We evaluate SVI on three benchmarks, including consistent, creative, and conditional settings, thoroughly verifying its versatility and state-of-the-art role.
  arXiv  Detail & Related papers  (2025-10-10T09:45:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.