ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation

• URL: http://arxiv.org/abs/2603.11421v1
• Date: Thu, 12 Mar 2026 01:27:08 GMT
• Title: ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
• Abstract summary: We present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents.<n>We design an automated multi-shot camera calibration pipeline that aligns disjoint single-shot trajectories into a unified global coordinate system.<n>Experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting.
• Score: 33.41430446859291
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent.

Related papers

• ShotPlan: Cinematic Video Generation with Learnable Planning Token [58.24475130458831]
  ShotPlan is a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model.<n>Our method introduces learnable planning tokens that capture shot-level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps.
  arXiv  Detail & Related papers  (2026-07-20T08:27:18Z)
• Geo-Align: Video Generation Alignment via Metric Geometry Reward [53.65904111864641]
  Geo-Align is the first Reinforcement Learning framework specifically designed for camera-controlled video re-rendering.<n>Built upon a pretrained model, we optimize the model through a scale-aware perceptual reward mechanism.<n>Experiments demonstrate that Geo-Align consistently outperforms existing supervised learning baselines in both precise camera controllability and visual fidelity.
  arXiv  Detail & Related papers  (2026-05-22T17:59:43Z)
• Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting [3.1328424544428852]
  We create a framework capable of leveraging internet-scale monocular videos.<n>Our core contribution is the generation of pseudo multi-view training triplets, consisting of a source video, a geometric anchor, and a target video.<n>At inference, our diffusion transformer utilizes a 4D point-cloud derived anchor to achieve state-of-the-art temporal consistency.
  arXiv  Detail & Related papers  (2026-04-23T15:32:56Z)
• CT-1: Vision-Language-Camera Models Transfer Spatial Reasoning Knowledge to Camera-Controllable Video Generation [84.82778089348285]
  Camera Transformer 1 is a specialized model designed to transfer spatial reasoning knowledge to video generation by accurately estimating camera trajectories.<n>Our framework successfully bridges the gap between spatial reasoning and video synthesis, yielding faithful and high-quality camera-controllable videos.
  arXiv  Detail & Related papers  (2026-04-10T10:43:18Z)
• Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation [49.12018869332346]
  InfCam is a camera-controlled video-to-video generation framework with high pose fidelity.<n>The framework integrates two key components: (1) infinite homography warping, which encodes 3D camera rotations directly within the 2D latent space of a video diffusion model.
  arXiv  Detail & Related papers  (2025-12-18T20:03:05Z)
• STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative [55.05324155854762]
  We introduce a SToryboard-Anchored GEneration workflow to reformulate the STAGE-based video generation task.<n>Instead of using sparses, we propose STEP2 to predict a structural storyboard composed of start-end frame pairs for each shot.<n>We also contribute the large-scale ConStoryBoard dataset, including high-quality movie clips with fine-grained narratives for story progression, cinematic attributes, and human preferences.
  arXiv  Detail & Related papers  (2025-12-13T15:57:29Z)
• CineLOG: A Training Free Approach for Cinematic Long Video Generation [19.97092710696699]
  We introduce CineLOG, a new dataset of 5,000 high quality, balanced video clips.<n>Each entry is annotated with a detailed scene description, explicit camera instructions based on a standard cinematic taxonomy.<n>We present our novel pipeline designed to create this dataset, which decouples the complex text to genre video (T2V) task generation into four easier stages with more mature technology.
  arXiv  Detail & Related papers  (2025-12-13T06:44:09Z)
• ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions [46.3918771233715]
  ShotDirector is an efficient framework that integrates parameter-level camera control and hierarchical editing-pattern-aware prompting.<n>Our framework effectively combines parameter-level conditions with high-level semantic guidance, achieving film-like controllable shot transitions.
  arXiv  Detail & Related papers  (2025-12-11T05:05:07Z)
• GimbalDiffusion: Gravity-Aware Camera Control for Video Generation [30.697985626973665]
  We introduce a framework that enables camera control grounded in physical-world coordinates, using gravity as a global reference.<n>We leverage panoramic 360-degree videos to construct a wide variety of camera trajectories, well beyond the predominantly straight, forward-facing trajectories seen in conventional video data.<n>We establish a benchmark for camera-aware video generation by rebalancing SpatialVID-HQ for comprehensive evaluation under wide camera pitch variation.
  arXiv  Detail & Related papers  (2025-12-09T20:54:35Z)
• MultiShotMaster: A Controllable Multi-Shot Video Generation Framework [67.38203939500157]
  Current generation techniques excel at single-shot clips but struggle to produce narrative multi-shot videos.<n>We propose MultiShotMaster, a framework for highly controllable multi-shot video generation.
  arXiv  Detail & Related papers  (2025-12-02T18:59:48Z)
• CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models [28.224969852134606]
  We introduce CineTrans, a framework for generating coherent multi-shot videos with cinematic, film-style transitions.<n>CineTrans produces cinematic multi-shot sequences while adhering to the film editing style, avoiding unstable transitions or naive concatenations.
  arXiv  Detail & Related papers  (2025-08-15T13:58:22Z)
• CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation [76.72787726497343]
  We present CineMaster, a framework for 3D-aware and controllable text-to-video generation.<n>Our goal is to empower users with comparable controllability as professional film directors.
  arXiv  Detail & Related papers  (2025-02-12T18:55:36Z)
• VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention [76.3175166538482]
  VideoGen-of-Thought (VGoT) is a step-by-step framework that automates multi-shot video synthesis from a single sentence.<n>VGoT addresses three core challenges: Narrative fragmentation, visual inconsistency, and transition artifacts.<n>Combined in a training-free pipeline, VGoT surpasses strong baselines by 20.4% in within-shot face consistency and 17.4% in style consistency.
  arXiv  Detail & Related papers  (2024-12-03T08:33:50Z)
• Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control [70.17137528953953]
  Collaborative video diffusion (CVD) is trained on top of a state-of-the-art camera-control module for video generation. CVD generates multiple videos rendered from different camera trajectories with significantly better consistency than baselines.
  arXiv  Detail & Related papers  (2024-05-27T17:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.