ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation

• URL: http://arxiv.org/abs/2605.26525v1
• Date: Tue, 26 May 2026 04:16:56 GMT
• Title: ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation
• Abstract summary: Multi-Shot Video Extrapolation (MSVE) is a task that extends an observed frame or clip into a sequence of cinematically structured shots.<n>MSVE reveals that long-video failure is not merely a limitation of context length, but a failure of context allocation.<n>We propose Recursive Context Allocation (ReCA), an inference-time framework that allocates context hierarchically across planning and generation.
• Score: 69.45329019089041
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Minute-scale cinematic video generation is a central challenge for generative video models. Existing paradigms address only fragments of this challenge: single-shot extrapolation preserves an anchor but lacks cinematic structure, while multi-shot storytelling imposes structure yet remains free to invent its visual states rather than continue an observed one. We define Multi-Shot Video Extrapolation (MSVE), a task that extends an observed frame or clip into a sequence of cinematically structured shots while preserving anchor state and advancing narrative intent. This setting operates under the finite per-call generation budget of short-video models. We identify three coupled bottlenecks: (1) global planners over-specify unsupported details from full screenplays; (2) shot-level prompts dilute task-relevant state when carrying the complete story; and (3) temporal chaining turns generated frames into a lossy memory in which identity, scene, object, and action state decay. MSVE reveals that long-video failure is not merely a limitation of context length, but a failure of context allocation. We propose Recursive Context Allocation (ReCA), an inference-time framework that allocates context hierarchically across planning and generation. ReCA recursively decomposes MSVE into context-bounded subproblems, invokes frozen generators at leaf nodes, and propagates structured state updates across time. To evaluate this setting, we further propose MSVE-Bench and NB-Q, a source-grounded protocol with prompts purpose-built for 3 to 5 minute long-video generation, a regime not addressed by existing short-clip benchmarks. Compared to previous methods, ReCA improves average normalized score by 8 to 16 percent over the strongest competing controller and improves multi-shot consistency metrics by 28 to 43 percent. View the project page at https://reca.vmv.re.

Related papers

• Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction [10.383154504465232]
  ABot-Recon is a simple streaming model that caches KV features from only the preceding 11 frames.<n>It predicts a point map in the current camera coordinate system together with an adjacent-frame relative pose.<n>To reduce accumulated drift, a lightweight temporal refiner improves relative rotations using recent visual and motion context.
  arXiv  Detail & Related papers  (2026-08-27T15:07:38Z)
• Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning [56.17582599802973]
  We introduce the Multi-Instruction Multi-Shot Long-Video Editing (MMLVE) task, which is structured around three core objectives.<n>To tackle these three unique challenges, we introduce an agentic editing framework that leverages the Large Language Models (LLMs) and Vision-Language Models (VLMs)<n>Extensive experiments demonstrate that our MMLVE-Agent outperforms existing closed-source SOTA approaches.
  arXiv  Detail & Related papers  (2026-08-27T08:48:13Z)
• Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration [61.98029663481308]
  Soap2Soap is a multi-agent framework that enforces long-term visual consistency through a DualBridge mechanism.<n>A closed-loop verification agent audits identity, stability, and alignment to trigger selective regeneration.
  arXiv  Detail & Related papers  (2026-05-17T12:38:21Z)
• Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning [82.97398529552166]
  MAGIC-Video is a framework built around a multimodal memory graph with interleaved narrative chain.<n>On EgoLifeQA, Ego-R1 and MM-Lifelong, MAGIC-Video consistently outperforms strong general-purpose, long-video, and agentic baselines.
  arXiv  Detail & Related papers  (2026-05-08T03:21:47Z)
• HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling [52.10845971383909]
  HieraMamba is a hierarchical architecture that preserves temporal structure and semantic richness across scales.<n>It sets a new state-of-the-art on Ego4D-NLQ, MAD, and TACoS, demonstrating precise, temporally faithful localization in long, untrimmed videos.
  arXiv  Detail & Related papers  (2025-10-27T06:13:07Z)
• Enhancing Long Video Question Answering with Scene-Localized Frame Grouping [19.83545369186771]
  Current Multimodal Large Language Models (MLLMs) often perform poorly in long video understanding.<n>We propose a new scenario under the video question-answering task, SceneQA.<n>We introduce a novel method called SLFG to combine individual frames into semantically coherent scene frames.
  arXiv  Detail & Related papers  (2025-08-05T02:28:58Z)
• HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding [14.464718780172582]
  We introduce HierarQ, a task-aware hierarchical Q-Former based framework that sequentially processes frames to bypass the need for frame sampling.<n>We introduce a lightweight two-stream language-guided feature modulator to incorporate task awareness in video understanding.<n>Extensive evaluations on 10 video benchmarks across video understanding, question answering, and captioning tasks demonstrate HierarQ's state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T16:21:23Z)
• VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention [76.3175166538482]
  VideoGen-of-Thought (VGoT) is a step-by-step framework that automates multi-shot video synthesis from a single sentence.<n>VGoT addresses three core challenges: Narrative fragmentation, visual inconsistency, and transition artifacts.<n>Combined in a training-free pipeline, VGoT surpasses strong baselines by 20.4% in within-shot face consistency and 17.4% in style consistency.
  arXiv  Detail & Related papers  (2024-12-03T08:33:50Z)
• Reconstructive Sequence-Graph Network for Video Summarization [107.0328985865372]
  Exploiting the inner-shot and inter-shot dependencies is essential for key-shot based video summarization. We propose a Reconstructive Sequence-Graph Network (RSGN) to encode the frames and shots as sequence and graph hierarchically. A reconstructor is developed to reward the summary generator, so that the generator can be optimized in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-05-10T01:47:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.