Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation

• URL: http://arxiv.org/abs/2606.02479v1
• Date: Mon, 01 Jun 2026 16:49:58 GMT
• Title: Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation
• Abstract summary: We propose Coverage-Maximizing Retrieval-Augmented Generation (COVRAG) for autoregressive video generation.<n>COVRAG uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D memory evidence.<n>Experiments on RealEstate10K and DL3DV10K show that COVRAG improves long-horizon geometric consistency while maintaining low latency compared to baselines.
• Score: 31.251444284229397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Maintaining long-term geometric consistency remains challenging for long-horizon autoregressive video generation. Memory-augmented generative models address this by retrieving historical frames, but their effectiveness depends on two key design choices: what 3D-geometric evidence should represent past observations, and how memory frames should be selected from this evidence. Existing methods often rely on camera poses or field-of-view overlap, which are lightweight but too coarse to reason about pixel-wise visibility, or use explicit 3D reconstruction, which provides fine-grained evidence but is costly to maintain over long rollouts. We propose Coverage-Maximizing Retrieval-Augmented Generation (COVRAG), a depth-based memory retrieval framework that uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D memory evidence. For frame selection, COVRAG maximizes residual coverage gain, iteratively retrieving frames that explain target-view regions not covered by the current context or previously selected memories. To improve scalability in long-video generation, we introduce sliding-window depth caching for efficient geometry estimation. Experiments on RealEstate10K and DL3DV10K show that COVRAG improves long-horizon geometric consistency while maintaining low latency compared to baselines.

Related papers

• GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors [14.045684695919116]
  3DGS has achieved remarkable success in synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts.<n>We propose a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction.<n>Our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency.
  arXiv  Detail & Related papers  (2026-08-22T08:47:29Z)
• Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning [53.96129188781529]
  We propose a question-guided geometric memory framework for video spatial reasoning.<n>ours injects camera-conditioned geometry into visual tokens and maintains two complementary memories.<n>ours achieves state-of-the-art performance among evaluated spatial reasoning models.
  arXiv  Detail & Related papers  (2026-05-26T17:26:29Z)
• Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval [86.05617542701644]
  Visual Geometry Grounded Transformer (VGGT) advances 3D reconstruction via scalable Transformer architecture.<n>StreamVGGT enables streaming with causal attention, yet its KV cache grows linearly with frames, causing memory overflow and quality degradation.<n>We present RetrieveVGGT, a training-free framework, which formulates context construction for VGGT as a retrieval problem.
  arXiv  Detail & Related papers  (2026-05-10T16:41:54Z)
• Ray-Aware Pointer Memory with Adaptive Updates for Streaming 3D Reconstruction [19.132257919770947]
  3D reconstruction from continuous image streams requires both accurate geometric aggregation and stable memory management.<n>We propose a ray-aware pointer memory for streaming 3D reconstruction that explicitly models both spatial location and viewing direction.<n>Our approach provides a principled framework for scalable and drift-resistant online 3D reconstruction from image streams.
  arXiv  Detail & Related papers  (2026-05-07T06:42:11Z)
• AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model [29.087865646035695]
  Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction.<n>We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs.<n>Experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
  arXiv  Detail & Related papers  (2026-04-21T17:59:47Z)
• Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction [50.5449251266956]
  This paper addresses the task of large-scale 3D scene reconstruction from long video sequences.<n>Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints.<n>We propose a novel neural global context representation that efficiently compresses and retains long-range scene information.
  arXiv  Detail & Related papers  (2026-04-09T17:59:50Z)
• I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation [56.33710337846449]
  I3DM is a novel implicit 3D-aware memory mechanism for consistent video scene generation.<n>At the core of our approach is a 3D-aware memory retrieval strategy.<n>To fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module.
  arXiv  Detail & Related papers  (2026-03-24T16:45:40Z)
• LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory [97.14005794889134]
  We present LoGeR, a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization.<n>LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning.<n>This memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference.
  arXiv  Detail & Related papers  (2026-03-03T18:55:37Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.