Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation

• URL: http://arxiv.org/abs/2606.04527v1
• Date: Wed, 03 Jun 2026 07:09:01 GMT
• Title: Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation
• Abstract summary: We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation.<n>It employs a learnable evolving memory to dynamically filter, abstract, and compress any-length history at constant cost.<n>It achieves state-of-the-art performance, and, to our knowledge, demonstrates promising 24-hour (>1.3 M frames) real-time rollouts for the first time.
• Score: 59.01737077739414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation that employs a learnable evolving memory to dynamically filter, abstract, and compress any-length history at constant cost. Existing methods mainly curate memory with predefined KV-cache schedules, fixed-ratio heuristic compression, or inference-time RoPE adaptation. These designs inevitably lose historical information and amplify compounding errors due to their limited cache window and ignorance of autoregressive generation noise. Inspired by human memory consolidation, Echo-Infinity replaces handcrafted memory curation with learnable Memory Query, which are updated by attention and a gating mechanism when past frames are evicted from the local window. The queries are optimized end-to-end with the video diffusion transformers (DiTs), forming an evolving memory that supports arbitrary compression ratios with constant computation independent of video length. They also act as a generalizable generation prior, improving quality even when only the optimized initial state is used. We further introduce Unified Relative RoPE Recipe, which anchors the sink frames to start from id 0 and lets the newest frame id grow at most to the DiTs' pretrained maximum temporal RoPE id throughout training and inference, freeing the model from the finite RoPE constraint and closing the train-test RoPE extrapolation gap. In long and short video generation, Echo-Infinity achieves state-of-the-art performance, and, to our knowledge, demonstrates promising 24-hour (>1.3 M frames) real-time rollouts for the first time, suggesting a practical path toward infinite video generation.

Related papers

• RECAP-Forcing: Retaining Content Appearances for Long Video Generation [37.756464872090596]
  We propose RECAP-Forcing, organizing memory by appearance novelty.<n>A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes.<n>As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content.
  arXiv  Detail & Related papers  (2026-08-27T06:24:59Z)
• Towards Error-Free Long Video Generation [56.86952045212838]
  In this paper, we introduce an infinite-length video generation framework that produces high-quality, dynamic, and identity-consistent single-shot long videos.<n>We first finetune a diffusion model as a video extension model on large-scale short video data to autoregressively generate temporally coherent clips.<n>Our framework establishes a new benchmark for realistic and coherent minute-level video synthesis.
  arXiv  Detail & Related papers  (2026-06-21T07:39:37Z)
• Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation [89.70897512515477]
  Action-controlled image-to-video generation is a promising paradigm for interactive world simulation, where each control signal should elicit an immediate visual response.<n>Maintaining visual fidelity and 3D consistency over long autoregressive rollouts remains challenging.<n>Existing 3D-aware methods often suffer from catastrophic drift due to two impediments: information loss from textitLatent--RGB Cycling, and the training--inference gap induced by the textiterror-free hypothesis.<n>We present textbfRobust Dreamer, a memory-augmented framework built around how to
  arXiv  Detail & Related papers  (2026-05-29T05:21:33Z)
• Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution [25.63670341165374]
  Video models should maintain evolving states when evidence is unobserved, yet current generators often freeze hidden states upon interruption.<n>We introduce ReMind, a framework eliciting dynamic memory behavior via memory-oriented data, event-aware training, and cacheated adaptation.
  arXiv  Detail & Related papers  (2026-05-25T01:30:41Z)
• Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos [0.05417521241272644]
  Train-free long video generation aims to enable foundation video generation models to produce longer videos.<n> Frame-level autoregressive frameworks, e.g., FIFO-diffusion, offer the advantage of generating infinitely long videos with constant memory consumption.<n>We propose textbfMIGA, a novel infinite-frame long video generation method.
  arXiv  Detail & Related papers  (2026-05-18T11:28:45Z)
• Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion [61.57938553036056]
  ARL2 is a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state.<n>We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame recurrent linear branch that maintains a fixed-size state for streaming context.<n>With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.
  arXiv  Detail & Related papers  (2026-05-15T19:33:45Z)
• Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation [48.476317015122625]
  Echo-Forcing is a training-free scene memory framework for interactive long video generation.<n>It supports smooth transitions, hard cuts, and long-range scene recall under a bounded cache budget.
  arXiv  Detail & Related papers  (2026-05-15T14:33:09Z)
• SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation [31.105007908298003]
  Streaming long-video generation faces a central challenge in continuous semantic switching.<n>Current approaches rely on cache rebuilding at prompt boundaries or fixed memory budgets.<n>We present SWIFT, a training-free framework for multi-prompt long-video generation.
  arXiv  Detail & Related papers  (2026-05-10T09:37:56Z)
• MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens [34.228121359393775]
  We introduce MemRoPE, a training-free framework with two co-designed components.<n>Online RoPE Indexing caches unrotated keys and applies positional embeddings dynamically at attention time.<n>MemRoPE outperforms existing methods in temporal coherence, visual fidelity, and subject consistency across minute- to hour-scale generation.
  arXiv  Detail & Related papers  (2026-03-12T23:14:16Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)
• ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models [66.84478240757038]
  A majority of video diffusion models (VDMs) generate long videos in an autoregressive manner, i.e., generating subsequent clips conditioned on last frames of previous clip. We introduce causal (i.e., unidirectional) generation into VDMs, and use past frames as prompt to generate future frames. Our ViD-GPT achieves state-of-the-art performance both quantitatively and qualitatively on long video generation.
  arXiv  Detail & Related papers  (2024-06-16T15:37:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.