EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation

• URL: http://arxiv.org/abs/2605.23610v1
• Date: Fri, 22 May 2026 13:20:29 GMT
• Title: EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation
• Abstract summary: Multi-shot video generation requires maintaining a consistent appearance of recurring entities across shots.<n>Recent autoregressive methods reuse previously generated frames as memory.<n>We propose an entity-centric memory in the form of an entity-indexed bank of latent patches.
• Score: 18.052867824594347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-shot video generation requires maintaining a consistent appearance of recurring entities across shots while remaining faithful to shot-specific text prompts. Recent autoregressive methods reuse previously generated frames as memory. However, full-frame storage entangles persistent entity information with transient scene context, leading to irrelevant information leakage and high computational cost. We propose an entity-centric memory in the form of an entity-indexed bank of latent patches. We introduce sparse token conditioning compatible with pretrained models, restricting self-attention to entity-relevant tokens and reducing computational cost. To support this, we introduce a structured multi-shot script format. We additionally propose a budgeted memory update strategy to maintain a compact, evolving memory. Finally, we equip the entity representation with a noise-injection mechanism that enables fine-grained appearance control, preventing leakage of irrelevant information. Our method improves prompt adherence and efficiency while preserving subject consistency.

Related papers

• Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding [64.34948094316685]
  We introduce LatentStream, a progressive latent working memory framework that shifts streaming memory from store-and-retrieve to retrieve-and-internalize.<n>LatentStream achieves new state-of-the-art results on existing online and offline video benchmarks.
  arXiv  Detail & Related papers  (2026-09-03T17:28:14Z)
• RECAP-Forcing: Retaining Content Appearances for Long Video Generation [37.756464872090596]
  We propose RECAP-Forcing, organizing memory by appearance novelty.<n>A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes.<n>As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content.
  arXiv  Detail & Related papers  (2026-08-27T06:24:59Z)
• PermaVid: Consistent Video Generation Across Edits via Disentangled Context Memory [80.58558679163197]
  PermaVid is a novel framework built upon a multi-modal context memory that disentangles spatial context into semantic appearance and geometric structure.<n>Our method maintains strong long-term semantic and structural consistency after edits, significantly outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-15T09:20:32Z)
• FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion [59.207505503284715]
  FadeMem organizes historical KV blocks into a temporal hierarchy under a fixed cache budget.<n>New history is inserted as fine-grained entries, while older adjacent entries are progressively merged.<n>Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies.
  arXiv  Detail & Related papers  (2026-06-09T10:22:18Z)
• Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution [25.63670341165374]
  Video models should maintain evolving states when evidence is unobserved, yet current generators often freeze hidden states upon interruption.<n>We introduce ReMind, a framework eliciting dynamic memory behavior via memory-oriented data, event-aware training, and cacheated adaptation.
  arXiv  Detail & Related papers  (2026-05-25T01:30:41Z)
• Tiny-Engram: Trigger-Indexed Concept Tables for Generative Vision [9.538470992436883]
  Small, explicitly addressed concept tables are a practical route to modular visual personalization.<n>Tiny-Engram parameterizes each concept as a small set of memory entries indexed by registered n-gram matches.<n>Results suggest that small, explicitly addressed concept tables are a practical route to modular visual personalization.
  arXiv  Detail & Related papers  (2026-05-19T16:27:07Z)
• SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation [31.105007908298003]
  Streaming long-video generation faces a central challenge in continuous semantic switching.<n>Current approaches rely on cache rebuilding at prompt boundaries or fixed memory budgets.<n>We present SWIFT, a training-free framework for multi-prompt long-video generation.
  arXiv  Detail & Related papers  (2026-05-10T09:37:56Z)
• Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation [22.26820693283945]
  We propose a decoupled framework that separates memory conditioning from generation.<n>We employ a lightweight, independent memory branch to learn precise spatial consistency from historical observation.<n>Our approach achieves state-of-the-art performance in terms of both visual quality and spatial consistency.
  arXiv  Detail & Related papers  (2026-04-20T13:00:17Z)
• Memory-Efficient Fine-Tuning Diffusion Transformers via Dynamic Patch Sampling and Block Skipping [95.91466630940084]
  Diffusion Transformers (DiTs) have significantly enhanced text-to-image (T2I) generation quality, enabling high-quality personalized content creation.<n>We propose a memory-efficient fine-tuning framework called DiT-BlockSkip, integrating timestep-aware dynamic patch sampling and block skipping.<n>Our approach achieves competitive personalization performance qualitatively and quantitatively, while reducing memory usage substantially.
  arXiv  Detail & Related papers  (2026-03-21T11:02:18Z)
• NextMem: Towards Latent Factual Memory for LLM-based Agents [58.35585202907478]
  NextMem is a latent factual memory framework that utilizes an autoregressive autoencoder to efficiently construct latent memory.<n>Extensive experiments demonstrate that NextMem achieves superior performance.
  arXiv  Detail & Related papers  (2026-02-26T14:35:27Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse [4.210760734549566]
  FlashMem is a framework that distills intrinsic memory directly from transient reasoning states via computation reuse.<n>Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times.
  arXiv  Detail & Related papers  (2026-01-09T03:27:43Z)
• VideoMemory: Toward Consistent Video Generation via Memory Integration [28.605816634949814]
  VideoMemory integrates narrative planning with visual generation through a Dynamic Memory Bank.<n>The Dynamic Memory Bank stores explicit visual and semantic descriptors for characters, props, and backgrounds.<n>This retrieval-update mechanism enables consistent portrayal of entities across distant shots and supports coherent long-form generation.
  arXiv  Detail & Related papers  (2026-01-07T07:10:32Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• Mixture of Contexts for Long Video Generation [72.96361488755986]
  We recast long-context video generation as an internal information retrieval task.<n>We propose a simple, learnable sparse attention routing module, Mixture of Contexts (MoC), as an effective long-term memory retrieval engine.<n>As we scale the data and gradually sparsify the routing, the model allocates compute to salient history, preserving identities, actions, and scenes over minutes of content.
  arXiv  Detail & Related papers  (2025-08-28T17:57:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.