MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction

• URL: http://arxiv.org/abs/2603.15330v1
• Date: Mon, 16 Mar 2026 14:21:19 GMT
• Title: MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction
• Abstract summary: We present MeMix, a training-free, plug-and-play module that improves streaming reconstruction by recasting the recurrent state into a Memory Mixture.<n>MeMix partitions the state into multiple independent memory patches and updates only the least-aligned memory patches while preserving others.<n>It reduces reconstruction completeness error by 15.3% on average (up to 40.0%) across 300--500 frame streams on 7-Scenes.
• Score: 11.01217345606816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstruction is a fundamental task in 3D vision and a fundamental capability for spatial intelligence. Particularly, streaming 3D reconstruction is central to real-time spatial perception, yet existing recurrent online models often suffer from progressive degradation on long sequences due to state drift and forgetting, motivating inference-time remedies. We present MeMix, a training-free, plug-and-play module that improves streaming reconstruction by recasting the recurrent state into a Memory Mixture. MeMix partitions the state into multiple independent memory patches and updates only the least-aligned memory patches while exactly preserving others. This selective update mitigates catastrophic forgetting while retaining $O(1)$ inference memory, and requires no fine-tuning or additional learnable parameters, making it directly applicable to existing recurrent reconstruction models. Across standard benchmarks (ScanNet, 7-Scenes, KITTI, etc.), under identical backbones and inference settings, MeMix reduces reconstruction completeness error by 15.3% on average (up to 40.0%) across 300--500 frame streams on 7-Scenes. The code is available at https://dongjiacheng06.github.io/MeMix/

Related papers

• Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction [43.823514226720846]
  Scal3R reformulates online reconstruction as multi-reference relative pose.<n>It reduces the average ATE by over 60% on KITTI compared to the online baseline.
  arXiv  Detail & Related papers  (2026-09-03T17:59:53Z)
• ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction [57.10941470313318]
  Streaming 3D reconstruction relies on a compact recurrent scene state to process long image streams in linear time and bounded memory.<n>We introduce ReCal3R, a reliability-calibrated learning rate method for recurrent 3D reconstruction.
  arXiv  Detail & Related papers  (2026-07-06T17:35:18Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion [61.57938553036056]
  ARL2 is a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state.<n>We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame recurrent linear branch that maintains a fixed-size state for streaming context.<n>With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.
  arXiv  Detail & Related papers  (2026-05-15T19:33:45Z)
• Ray-Aware Pointer Memory with Adaptive Updates for Streaming 3D Reconstruction [19.132257919770947]
  3D reconstruction from continuous image streams requires both accurate geometric aggregation and stable memory management.<n>We propose a ray-aware pointer memory for streaming 3D reconstruction that explicitly models both spatial location and viewing direction.<n>Our approach provides a principled framework for scalable and drift-resistant online 3D reconstruction from image streams.
  arXiv  Detail & Related papers  (2026-05-07T06:42:11Z)
• Mem3R: Streaming 3D Reconstruction with Hybrid Memory via Test-Time Training [8.037339305506656]
  We propose Mem3R, a streaming 3D reconstruction model with a hybrid memory design.<n>For camera tracking, Mem3R employs an implicit fast-weight memory implemented as a lightweight Multi-Layer Perceptron.<n>For geometric mapping, Mem3R maintains an explicit token-based fixed-size state.
  arXiv  Detail & Related papers  (2026-04-08T16:41:22Z)
• LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory [97.14005794889134]
  We present LoGeR, a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization.<n>LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning.<n>This memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference.
  arXiv  Detail & Related papers  (2026-03-03T18:55:37Z)
• LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstruction [28.14207161940146]
  LASER is a training-free framework that converts an offline reconstruction model into a streaming system.<n>We introduce layer-wise scale alignment, which segments depth predictions into discrete layers.<n>Experiments show that LASER achieves state-of-the-art performance on camera pose estimation and point map reconstruction %quality with offline models.
  arXiv  Detail & Related papers  (2025-12-15T18:59:04Z)
• Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution [52.76038908826961]
  We propose $textbfReMe$ ($textitRemember Me, Refine Me$) to bridge the gap between static storage and dynamic reasoning.<n>ReMe innovates across the memory lifecycle via three mechanisms: $textitmulti-faceted distillation$, which extracts fine-grained experiences.<n>Experiments on BFCL-V3 and AppWorld demonstrate that ReMe establishes a new state-of-the-art in agent memory system.
  arXiv  Detail & Related papers  (2025-12-11T14:40:01Z)
• UniVerse: Unleashing the Scene Prior of Video Diffusion Models for Robust Radiance Field Reconstruction [73.29048162438797]
  We introduce UniVerse, a unified framework for robust reconstruction based on a video diffusion model.<n>Specifically, UniVerse first converts inconsistent images into initial videos, then uses a specially designed video diffusion model to restore them into consistent images.<n>Experiments on both synthetic and real-world datasets demonstrate the strong generalization capability and superior performance of our method in robust reconstruction.
  arXiv  Detail & Related papers  (2025-10-02T04:50:18Z)
• TTT3R: 3D Reconstruction as Test-Time Training [69.51086319339662]
  We revisit the 3D reconstruction foundation models from a Test-Time Training perspective.<n>We leverage the alignment confidence between the memory state and incoming observations to derive a closed-form learning rate.<n>This training-free intervention, termed TTT3R, substantially improves length generalization.
  arXiv  Detail & Related papers  (2025-09-30T17:59:51Z)
• Least Redundant Gated Recurrent Neural Network [0.0]
  We introduce a recurrent neural architecture called Deep Memory Update (DMU) It is based on updating the previous memory state with a deep transformation of the lagged state and the network input. Its training is stable and fast due to relating its learning rate to the size of the module.
  arXiv  Detail & Related papers  (2021-05-28T20:24:00Z)
• Tuned Compositional Feature Replays for Efficient Stream Learning [11.697781095636147]
  We propose a new continual learning algorithm, Compositional Replay Using Memory Blocks (CRUMB) CRUMB mitigates forgetting by replaying feature maps reconstructed by combining generic parts. We stress-tested CRUMB alongside 13 competing methods on 7 challenging datasets.
  arXiv  Detail & Related papers  (2021-04-06T00:53:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.