ShotPlan: Cinematic Video Generation with Learnable Planning Token

• URL: http://arxiv.org/abs/2607.17675v1
• Date: Mon, 20 Jul 2026 08:27:18 GMT
• Title: ShotPlan: Cinematic Video Generation with Learnable Planning Token
• Abstract summary: ShotPlan is a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model.<n>Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps.
• Score: 58.24475130458831
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current video generation models achieve impressive results in single-shot generation, yet remain limited in cinematic video generation, where coherent narratives and effective multi-shot composition require explicit shot planning. To address this challenge, we propose ShotPlan, a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model. Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps. Unlike standard video generation tokens, the proposed planning tokens are equipped with Fractional Temporal Rotary Position Embedding (FRoPE), enabling shot transitions to be modeled at the frame level. Experiments demonstrate that ShotPlan significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter-shot consistency.

Related papers

• CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling [111.28808966759613]
  We propose a unified framework named CineWeaver to achieve reference-controllable multi-shot long-video generation without retraining.<n> Experimental results demonstrate that CineWeaver produces high-quality cinematic videos of long durations with consistent identities, stable global appearance, and clear shot transitions.
  arXiv  Detail & Related papers  (2026-07-29T06:48:11Z)
• ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation [33.41430446859291]
  We present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents.<n>We design an automated multi-shot camera calibration pipeline that aligns disjoint single-shot trajectories into a unified global coordinate system.<n>Experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting.
  arXiv  Detail & Related papers  (2026-03-12T01:27:08Z)
• STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative [55.05324155854762]
  We introduce a SToryboard-Anchored GEneration workflow to reformulate the STAGE-based video generation task.<n>Instead of using sparses, we propose STEP2 to predict a structural storyboard composed of start-end frame pairs for each shot.<n>We also contribute the large-scale ConStoryBoard dataset, including high-quality movie clips with fine-grained narratives for story progression, cinematic attributes, and human preferences.
  arXiv  Detail & Related papers  (2025-12-13T15:57:29Z)
• ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions [46.3918771233715]
  ShotDirector is an efficient framework that integrates parameter-level camera control and hierarchical editing-pattern-aware prompting.<n>Our framework effectively combines parameter-level conditions with high-level semantic guidance, achieving film-like controllable shot transitions.
  arXiv  Detail & Related papers  (2025-12-11T05:05:07Z)
• MultiShotMaster: A Controllable Multi-Shot Video Generation Framework [67.38203939500157]
  Current generation techniques excel at single-shot clips but struggle to produce narrative multi-shot videos.<n>We propose MultiShotMaster, a framework for highly controllable multi-shot video generation.
  arXiv  Detail & Related papers  (2025-12-02T18:59:48Z)
• ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Models [37.70850513700251]
  Current diffusion-based text-to-video methods are limited to producing short video clips of a single shot.<n>We propose a framework that includes a dataset collection pipeline and architectural extensions to video diffusion models to enable text-to-multi-shot video generation.<n>Our approach enables generation of multi-shot videos as a single video with full attention across all frames of all shots.
  arXiv  Detail & Related papers  (2025-05-12T15:22:28Z)
• Long Context Tuning for Video Generation [63.060794860098795]
  Long Context Tuning (LCT) is a training paradigm that expands the context window of pre-trained single-shot video diffusion models.<n>Our method expands full attention mechanisms from individual shots to encompass all shots within a scene.<n>Experiments demonstrate coherent multi-shot scenes and exhibit emerging capabilities, including compositional generation and interactive shot extension.
  arXiv  Detail & Related papers  (2025-03-13T17:40:07Z)
• VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention [76.3175166538482]
  VideoGen-of-Thought (VGoT) is a step-by-step framework that automates multi-shot video synthesis from a single sentence.<n>VGoT addresses three core challenges: Narrative fragmentation, visual inconsistency, and transition artifacts.<n>Combined in a training-free pipeline, VGoT surpasses strong baselines by 20.4% in within-shot face consistency and 17.4% in style consistency.
  arXiv  Detail & Related papers  (2024-12-03T08:33:50Z)
• SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction [93.26613503521664]
  This paper presents a short-to-long video diffusion model, SEINE, that focuses on generative transition and prediction. We propose a random-mask video diffusion model to automatically generate transitions based on textual descriptions. Our model generates transition videos that ensure coherence and visual quality.
  arXiv  Detail & Related papers  (2023-10-31T17:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.