OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data

• URL: http://arxiv.org/abs/2606.13432v1
• Date: Thu, 11 Jun 2026 14:58:28 GMT
• Title: OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data
• Abstract summary: We introduce a general camera motion representation that encodes cameras as grid motion videos.<n>This camera grid represents the camera parameters visually and supports the integration of diverse trajectories for multi-shot video generation.<n>We propose OmniDirector, a unified framework trained on a million-scale camera grid-video pairs.
• Score: 47.59993927255196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cloning camera motion from reference videos is an important task in video generation, as videos provide intuitive and precise control. Existing methods either directly use parametric representations that fail to handle multi-shot generation or synthesize cross-paired data, which suffer from data scarcity, resulting in poor performance in complicated camera motion cloning. To address these issues, we introduce a general camera motion representation that encodes cameras as grid motion videos. This camera grid represents the camera parameters visually and supports the integration of diverse trajectories for multi-shot video generation. Building upon this, we propose OmniDirector, a unified framework trained on a million-scale camera grid-video pairs that coordinates characters, actions, and cameras to provide director-level control for multimodal diffusion transformers. Furthermore, we design a novel hierarchical prompt expansion agent that harmoniously integrates different control signals by systematically describing camera motion and visual content through understanding signal relationships. Extensive experiments demonstrate the superior performance and outstanding controllability of our framework. Project page: https://ymlinfeng.github.io/OmniDirector.github.io/

Related papers

• UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation [50.391925716274926]
  textbfUniMoCa is a representation-driven framework that unifies motion and camera controls in visual space.<n>We show that UniMoCa achieves substantial gains in human motion control, camera control, temporal consistency, and camera-aware robustness with minimal additional complexity.
  arXiv  Detail & Related papers  (2026-08-03T09:18:13Z)
• TriMotion: Modality-Agnostic Camera Control for Video Generation [65.5913929253637]
  TriMotion is a framework for camera-controlled video generation that maps video, pose, and text inputs, describing the same camera trajectory into a shared motion embedding space.<n>We show that TriMotion generates high-quality videos that accurately follow the target camera trajectories across all three modalities.
  arXiv  Detail & Related papers  (2026-06-18T16:07:05Z)
• Unified Camera Positional Encoding for Controlled Video Generation [48.5789182990001]
  Transformers have emerged as a universal backbone across 3D perception, video generation, and world models for autonomous driving and embodied AI.<n>We introduce Relative Ray, a geometry-consistent representation that unifies complete camera information, including 6-DoF poses, intrinsics, and lens distortions.<n>To facilitate systematic training and evaluation, we construct a large video dataset covering a wide range of camera motions and lens types.
  arXiv  Detail & Related papers  (2025-12-08T07:34:01Z)
• Modular-Cam: Modular Dynamic Camera-view Video Generation with LLM [43.889033468684445]
  We propose a novel text-to-video generation method, i.e., Modular-Cam.<n>To better understand a given complex prompt, we utilize a large language model to analyze user instructions.<n>To generate a video containing dynamic scenes that match the given camera-views, we incorporate the widely-used temporal transformer.
  arXiv  Detail & Related papers  (2025-04-16T13:04:01Z)
• OmniCam: Unified Multimodal Video Generation via Camera Control [42.94206239207397]
  Camera control which achieves diverse visual effects by changing camera position and pose has attracted widespread attention.<n>Existing methods face challenges such as complex interaction and limited control capabilities.<n>We present OmniCam, a unified camera framework that generates guidance-temporally consistent videos.
  arXiv  Detail & Related papers  (2025-04-03T06:38:30Z)
• Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention [62.2447324481159]
  Cavia is a novel framework for camera-controllable, multi-view video generation. Our framework extends the spatial and temporal attention modules, improving both viewpoint and temporal consistency. Cavia is the first of its kind that allows the user to specify distinct camera motion while obtaining object motion.
  arXiv  Detail & Related papers  (2024-10-14T17:46:32Z)
• VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control [74.5434726968562]
  We show how to tame transformers video for 3D camera control using a ControlNet-like conditioning mechanism.<n>Our work is the first to enable camera control for transformer-based video diffusion models.
  arXiv  Detail & Related papers  (2024-07-17T17:59:05Z)
• Training-free Camera Control for Video Generation [15.79168688275606]
  We propose a training-free and robust solution to offer camera movement control for off-the-shelf video diffusion models.<n>Our method does not require any supervised finetuning on camera-annotated datasets or self-supervised training via data augmentation.<n>It can be plug-and-play with most pretrained video diffusion models and generate camera-controllable videos with a single image or text prompt as input.
  arXiv  Detail & Related papers  (2024-06-14T15:33:00Z)
• Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control [70.17137528953953]
  Collaborative video diffusion (CVD) is trained on top of a state-of-the-art camera-control module for video generation. CVD generates multiple videos rendered from different camera trajectories with significantly better consistency than baselines.
  arXiv  Detail & Related papers  (2024-05-27T17:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.