OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control

• URL: http://arxiv.org/abs/2604.06010v1
• Date: Tue, 07 Apr 2026 16:06:02 GMT
• Title: OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control
• Abstract summary: Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed.<n>Existing generation models often entangle these factors, limiting independent control.<n>We introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions.
• Score: 49.41924736941193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed. However, existing generation models often entangle these factors, limiting independent control. In this work, we introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions. This compositional approach enables flexible video generation by allowing arbitrary pairings of camera and content conditions, unlocking unprecedented creative control. To overcome the fundamental challenges of modality conflict and data scarcity inherent in such a system, we present two key innovations. First, we construct OmniCAM, a novel hybrid dataset combining curated real-world videos with synthetic data that provides diverse paired examples for robust multi-task learning. Second, we propose a Dual-level Curriculum Co-Training strategy that mitigates modality interference and synergistically learns from diverse data sources. This strategy operates on two levels: first, it progressively introduces control modalities by difficulties (condition-level), and second, trains for precise control on synthetic data before adapting to real data for photorealism (data-level). As a result, OmniCamera achieves state-of-the-art performance, enabling flexible control for complex camera movements while maintaining superior visual quality.

Related papers

• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning [82.28800081483224]
  We present DreamVideo- Omni, a unified framework enabling multisubject customization with omni-motion control.<n>To resolve multi-subject ambiguity, we introduce group and role embeddings to explicitly anchor motion signals to specific identities.<n>This provides motion-aware identity rewards in the latent space, prioritizing identity preservation aligned with human preferences.
  arXiv  Detail & Related papers  (2026-03-12T17:59:12Z)
• DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation [77.89090846233906]
  We propose a system-level framework, termed the Divide-and-Conquer Diffusion Model (DCDM)<n>DCDM decomposes video consistency modeling into three dedicated components while sharing a unified video generation backbone.<n>We validate our framework on the test set of the CVM Competition at AAAI'26, and the results demonstrate that the proposed strategies effectively address these challenges.
  arXiv  Detail & Related papers  (2026-02-14T07:02:36Z)
• EchoMotion: Unified Human Video and Motion Generation via Dual-Modality Diffusion Transformer [64.69014756863331]
  We introduce EchoMotion, a framework designed to model the joint distribution of appearance and human motion.<n>We also propose MVS-RoPE, which offers unified 3D positional encoding for both video and motion tokens.<n>Our findings reveal that explicitly representing human motion is to appearance, significantly boosting the coherence and plausibility of human-centric video generation.
  arXiv  Detail & Related papers  (2025-12-21T17:08:14Z)
• Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation [49.12018869332346]
  InfCam is a camera-controlled video-to-video generation framework with high pose fidelity.<n>The framework integrates two key components: (1) infinite homography warping, which encodes 3D camera rotations directly within the 2D latent space of a video diffusion model.
  arXiv  Detail & Related papers  (2025-12-18T20:03:05Z)
• MultiCOIN: Multi-Modal COntrollable Video INbetweening [46.37499813275259]
  We introduce MultiCOIN, a video inbetweening framework that allows multi-modal controls.<n>To ensure compatibility between DiT and our multi-modal controls, we map all motion controls into a common sparse representation.<n>We also propose a stage-wise training strategy to ensure that our model learns the multi-modal controls smoothly.
  arXiv  Detail & Related papers  (2025-10-09T17:59:27Z)
• Pulp Motion: Framing-aware multimodal camera and human motion generation [23.011172300168642]
  We are the first to cast this task as a text-conditioned joint generation, aiming to maintain consistent on-screen framing.<n>We propose a simple, model-agnostic framework that enforces multimodal coherence via an auxiliary modality.<n>Our experiments show the generality and effectiveness of our method in generating on-frame coherent human-camera motions.
  arXiv  Detail & Related papers  (2025-10-06T17:58:34Z)
• OmniCam: Unified Multimodal Video Generation via Camera Control [42.94206239207397]
  Camera control which achieves diverse visual effects by changing camera position and pose has attracted widespread attention.<n>Existing methods face challenges such as complex interaction and limited control capabilities.<n>We present OmniCam, a unified camera framework that generates guidance-temporally consistent videos.
  arXiv  Detail & Related papers  (2025-04-03T06:38:30Z)
• Free-Form Motion Control: Controlling the 6D Poses of Camera and Objects in Video Generation [78.65431951506152]
  We introduce a Synthetic dataset for Free-Form Motion Control (SynFMC)<n>The proposed SynFMC dataset includes diverse object and environment categories.<n>It covers various motion patterns according to specific rules, simulating common and complex real-world scenarios.<n>The complete 6D pose information facilitates models learning to disentangle the motion effects from objects and the camera in a video.
  arXiv  Detail & Related papers  (2025-01-02T18:59:45Z)
• MotionBridge: Dynamic Video Inbetweening with Flexible Controls [29.029643539300434]
  We introduce MotionBridge, a unified video inbetweening framework.<n>It allows flexible controls, including trajectory strokes, video editing masks, guide pixels, and text video.<n>We show that such multi-modal controls enable a more dynamic, customizable, and contextually accurate visual narrative.
  arXiv  Detail & Related papers  (2024-12-17T18:59:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.