Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation

• URL: http://arxiv.org/abs/2607.02798v1
• Date: Thu, 02 Jul 2026 22:17:20 GMT
• Title: Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation
• Abstract summary: We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints.<n>UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames.<n>It achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.
• Score: 14.852131707037655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern image-and-text-to-video diffusion models can synthesize highly realistic videos by iteratively denoising an initial Gaussian noise tensor conditioned on reference image and text inputs. However, existing approaches still lack precise and unified controllability over both object motion and camera motion within a single generation process. We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints by directly constructing the input noise of the diffusion model. Specifically, UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames. Sparse 3D point tracks are used to warp the Gaussian noise of the reference frame along desired object trajectories, while a virtual spherical noise representation provides globally consistent noise values for newly revealed scene regions under camera motion. By combining local track-guided noise warping with global sphere-based noise sampling, UniCaMo maintains geometric and temporal consistency under both object movement and viewpoint changes. Because UniCaMo modifies only the input noise, it requires no auxiliary adapters, control branches, or architectural changes to the underlying video diffusion model. With lightweight LoRA fine-tuning on large pretrained video diffusion models, including Wan 2.1 (14B), UniCaMo achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.

Related papers

• CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warping [81.58057192696843]
  Existing methods that directly inject numerical camera parameters into the diffusion backbone often fail to bridge the gap between abstract coordinates and visual content.<n>We propose CameraNoise, a flow-to-noise warping method that encodes camera motion into a temporally coherent representation.<n>By integrating CameraNoise into the diffusion process, our framework delivers stable, high-fidelity videos.
  arXiv  Detail & Related papers  (2026-05-29T03:02:50Z)
• HumANDiff: Articulated Noise Diffusion for Motion-Consistent Human Video Generation [47.59225421284659]
  HumANDiff enables human video generation by fine-tuning video diffusion models with articulated noise sampling.<n>It achieves motion-consistent, high-fidelity humans with diverse clothing styles.
  arXiv  Detail & Related papers  (2026-04-07T14:55:10Z)
• M2DAO-Talker: Harmonizing Multi-granular Motion Decoupling and Alternating Optimization for Talking-head Generation [65.48046909056468]
  We reformulate talking head generation into a unified framework comprising video preprocessing, motion representation, and rendering reconstruction.<n>M2DAO-Talker achieves state-of-the-art performance, with the 2.43 dB PSNR improvement in generation quality and 0.64 gain in user-evaluated video realness.
  arXiv  Detail & Related papers  (2025-07-11T04:48:12Z)
• DreamJourney: Perpetual View Generation with Video Diffusion Models [91.88716097573206]
  Perpetual view generation aims to synthesize a long-term video corresponding to an arbitrary camera trajectory solely from a single input image.<n>Recent methods commonly utilize a pre-trained text-to-image diffusion model to synthesize new content of previously unseen regions along camera movement.<n>We present DreamJourney, a two-stage framework that leverages the world simulation capacity of video diffusion models to trigger a new perpetual scene view generation task.
  arXiv  Detail & Related papers  (2025-06-21T12:51:34Z)
• Motion-Aware Concept Alignment for Consistent Video Editing [57.08108545219043]
  We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a training-free framework bridging the gap between image-domain semantic mixing and video.<n>Given a generated video and a user-provided reference image, MoCA-Video injects the semantic features of the reference image into a specific object within the video.<n>We evaluate MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS, and introduce a novel metric CASS (Conceptual Alignment Shift Score) to evaluate the consistency and effectiveness of the visual shifts between the source prompt and the modified video frames
  arXiv  Detail & Related papers  (2025-06-01T13:28:04Z)
• MoBGS: Motion Deblurring Dynamic 3D Gaussian Splatting for Blurry Monocular Video [26.468480933928458]
  MoBGS reconstructs sharp and high-quality views from blurry monocular videos in an end-to-end manner.<n>We introduce a Blur-adaptive Latent Camera Estimation (BLCE) method for effective latent camera trajectory estimation.<n>We also propose a physically-inspired Latent Camera-inspired Exposure Estimation (LCEE) method to ensure consistent deblurring of both global camera and local object motion.
  arXiv  Detail & Related papers  (2025-04-21T14:19:19Z)
• LightMotion: A Light and Tuning-free Method for Simulating Camera Motion in Video Generation [56.64004196498026]
  LightMotion is a light and tuning-free method for simulating camera motion in video generation.<n> operating in the latent space, it eliminates additional fine-tuning, inpainting, and depth estimation.
  arXiv  Detail & Related papers  (2025-03-09T08:28:40Z)
• Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise [19.422355461775343]
  We enhance video diffusion models by allowing motion control via structured latent noise sampling.<n>We propose a novel noise warping algorithm, fast enough to run in real time, that replaces random temporal Gaussianity with correlated warped noise.<n>The efficiency of our algorithm enables us to fine-tune modern video diffusion base models using warped noise with minimal overhead.
  arXiv  Detail & Related papers  (2025-01-14T18:59:10Z)
• MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation [19.999239668765885]
  MotionMix is a weakly-supervised diffusion model that leverages both noisy and unannotated motion sequences. Our framework consistently achieves state-of-the-art performances on text-to-motion, action-to-motion, and music-to-dance tasks.
  arXiv  Detail & Related papers  (2024-01-20T04:58:06Z)
• Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising [137.5080784570804]
  Video denoising aims at removing noise from videos to recover clean ones. Some existing works show that optical flow can help the denoising by exploiting the additional spatial-temporal clues from nearby frames. We propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels.
  arXiv  Detail & Related papers  (2022-08-25T00:09:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.