UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation
- URL: http://arxiv.org/abs/2608.01944v1
- Date: Mon, 03 Aug 2026 09:18:13 GMT
- Title: UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation
- Abstract summary: textbfUniMoCa is a representation-driven framework that unifies motion and camera controls in visual space.<n>We show that UniMoCa achieves substantial gains in human motion control, camera control, temporal consistency, and camera-aware robustness with minimal additional complexity.
- Score: 50.391925716274926
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Controlling human motion and camera movement is essential for faithful human-oriented video generation, yet remains challenging in multi-person scenes with large body motions, occlusions, and dynamic cameras. Existing pipelines typically rely on visual motion sequences, such as skeleton maps, pose maps, or rendered body representations, for motion control, while using camera embeddings for camera control. Such heterogeneous control interfaces force video generation models to reconcile pixel-aligned visual cues with non-visual geometric embeddings, making motion-camera attribution difficult and sensitive to camera estimation errors. We propose \textbf{UniMoCa}, a representation-driven framework that unifies motion and camera controls in visual space. At the core of UniMoCa is \textbf{Motion-Camera Visual Proxy} (\textbf{MCVP}), a mutually-sharable novel representation that converts 3D human motion and camera trajectories extracted from driving videos into an identity-neutral visual proxy. MCVP renders temporally aligned human geometry under the recovered camera trajectory and augments it with explicit camera trajectory markers, replacing heterogeneous visual-parametric controls with distinguishable visual cues. As both control factors are represented in the same visual space, they become mutually compatible rather than heterogeneous, enabling consistent joint reasoning and editing during video generation. We further curate a \textbf{MCVP-Video} dataset covering complex actions, multi-person interactions, and diverse camera trajectories. Experiments based on the Wan2.2 I2V show that UniMoCa achieves substantial gains in human motion control, camera control, temporal consistency, and camera-aware robustness with minimal additional complexity. More details are shown in our Project page: https://tanliming-daniel.github.io/UniMoCa/.
Related papers
- Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control [56.828569670698975]
"Directing the World" is a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control.<n>Our key idea is to decouple control learning while preserving a unified autoregressive video prior.<n>For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy.
arXiv Detail & Related papers (2026-06-26T11:08:09Z) - TriMotion: Modality-Agnostic Camera Control for Video Generation [65.5913929253637]
TriMotion is a framework for camera-controlled video generation that maps video, pose, and text inputs, describing the same camera trajectory into a shared motion embedding space.<n>We show that TriMotion generates high-quality videos that accurately follow the target camera trajectories across all three modalities.
arXiv Detail & Related papers (2026-06-18T16:07:05Z) - OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data [47.59993927255196]
We introduce a general camera motion representation that encodes cameras as grid motion videos.<n>This camera grid represents the camera parameters visually and supports the integration of diverse trajectories for multi-shot video generation.<n>We propose OmniDirector, a unified framework trained on a million-scale camera grid-video pairs.
arXiv Detail & Related papers (2026-06-11T14:58:28Z) - SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation [11.470401259593944]
SymphoMotion is a unified motion-control framework that governs camera trajectories and object dynamics within a single model.<n>We show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy.
arXiv Detail & Related papers (2026-04-04T12:59:24Z) - ATI: Any Trajectory Instruction for Controllable Video Generation [25.249489701215467]
We propose a unified framework for motion control in video generation that seamlessly integrates camera movement, object-level translation, and fine-grained local motion.<n>Our approach offers a cohesive solution by projecting user-defined trajectories into the latent space of pre-trained image-to-video generation models.
arXiv Detail & Related papers (2025-05-28T23:49:18Z) - Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation [73.73984727616198]
We present Uni3C, a unified framework for precise control of both camera and human motion in video generation.<n>First, we propose a plug-and-play control module trained with a frozen video generative backbone, PCDController.<n>Second, we propose a jointly aligned 3D world guidance for the inference phase that seamlessly integrates both scenic point clouds and SMPL-X characters.
arXiv Detail & Related papers (2025-04-21T07:10:41Z) - Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion [34.404342332033636]
We introduce Direct-a-Video, a system that allows users to independently specify motions for multiple objects as well as camera's pan and zoom movements.
For camera movement, we introduce new temporal cross-attention layers to interpret quantitative camera movement parameters.
Both components operate independently, allowing individual or combined control, and can generalize to open-domain scenarios.
arXiv Detail & Related papers (2024-02-05T16:30:57Z) - MotionCtrl: A Unified and Flexible Motion Controller for Video Generation [77.09621778348733]
Motions in a video primarily consist of camera motion, induced by camera movement, and object motion, resulting from object movement.
This paper presents MotionCtrl, a unified motion controller for video generation designed to effectively and independently control camera and object motion.
arXiv Detail & Related papers (2023-12-06T17:49:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.