Taming Camera-Controlled Video Generation with Verifiable Geometry Reward

• URL: http://arxiv.org/abs/2512.02870v1
• Date: Tue, 02 Dec 2025 15:33:19 GMT
• Title: Taming Camera-Controlled Video Generation with Verifiable Geometry Reward
• Abstract summary: We introduce an online reinforcement learning framework that optimize a pretrained video generator for precise camera control.<n>We estimate the 3D camera trajectories for both generated and reference videos, divide each trajectory into short segments, and compute segment-wise relative poses.<n>We construct a comprehensive dataset featuring diverse large-amplitude camera motions and scenes with varied subject dynamics.
• Score: 36.31658788083449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in video diffusion models have remarkably improved camera-controlled video generation, but most methods rely solely on supervised fine-tuning (SFT), leaving online reinforcement learning (RL) post-training largely underexplored. In this work, we introduce an online RL post-training framework that optimizes a pretrained video generator for precise camera control. To make RL effective in this setting, we design a verifiable geometry reward that delivers dense segment-level feedback to guide model optimization. Specifically, we estimate the 3D camera trajectories for both generated and reference videos, divide each trajectory into short segments, and compute segment-wise relative poses. The reward function then compares each generated-reference segment pair and assigns an alignment score as the reward signal, which helps alleviate reward sparsity and improve optimization efficiency. Moreover, we construct a comprehensive dataset featuring diverse large-amplitude camera motions and scenes with varied subject dynamics. Extensive experiments show that our online RL post-training clearly outperforms SFT baselines across multiple aspects, including camera-control accuracy, geometric consistency, and visual quality, demonstrating its superiority in advancing camera-controlled video generation.

Related papers

• GeoStream: Toward Precise Camera Controlled Streaming Video Generation [85.01823220801256]
  Existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories.<n>Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive.<n>We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation.
  arXiv  Detail & Related papers  (2026-06-13T07:18:04Z)
• Geo-Align: Video Generation Alignment via Metric Geometry Reward [53.65904111864641]
  Geo-Align is the first Reinforcement Learning framework specifically designed for camera-controlled video re-rendering.<n>Built upon a pretrained model, we optimize the model through a scale-aware perceptual reward mechanism.<n>Experiments demonstrate that Geo-Align consistently outperforms existing supervised learning baselines in both precise camera controllability and visual fidelity.
  arXiv  Detail & Related papers  (2026-05-22T17:59:43Z)
• Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video [19.675672131137382]
  We propose a simple interface that turns camera-induced warps into camera-warped pseudo-history.<n>We align its positional encoding with the target frames being denoised and remove warped-history tokens without valid source observations.<n>Our method improves camera adherence, visual quality, and motion dynamics without test-time optimization or target-video adaptation.
  arXiv  Detail & Related papers  (2026-05-14T17:58:26Z)
• WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memories [36.79437857022868]
  WorldStereo is a novel framework that bridges camera-guided video generation and 3D reconstruction.<n>We show that WorldStereo acts as a powerful world model, tackling diverse scene generation tasks with high-fidelity 3D results.
  arXiv  Detail & Related papers  (2026-03-02T16:36:56Z)
• Beyond Inpainting: Unleash 3D Understanding for Precise Camera-Controlled Video Generation [21.084121261693365]
  We propose DepthDirector, a video re-rendering framework with precise camera controllability.<n>By leveraging the depth video from explicit 3D representation as camera-control guidance, our method can faithfully reproduce the dynamic scene of an input video under novel camera trajectories.
  arXiv  Detail & Related papers  (2026-01-15T09:26:45Z)
• Unified Camera Positional Encoding for Controlled Video Generation [48.5789182990001]
  Transformers have emerged as a universal backbone across 3D perception, video generation, and world models for autonomous driving and embodied AI.<n>We introduce Relative Ray, a geometry-consistent representation that unifies complete camera information, including 6-DoF poses, intrinsics, and lens distortions.<n>To facilitate systematic training and evaluation, we construct a large video dataset covering a wide range of camera motions and lens types.
  arXiv  Detail & Related papers  (2025-12-08T07:34:01Z)
• ReCamDriving: LiDAR-Free Camera-Controlled Novel Trajectory Video Generation [38.23100905961028]
  ReCamDriving is a vision-based, camera-controlled novel-trajectory video generation framework.<n>We present a 3DGS-based cross-trajectory data curation strategy to eliminate the train-test gap in camera transformation patterns.
  arXiv  Detail & Related papers  (2025-12-03T09:55:25Z)
• EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance [69.40274699401473]
  We introduce EPiC, an efficient and precise camera control learning framework.<n>It constructs high-quality anchor videos without expensive camera trajectory annotations.<n>EPiC achieves SOTA performance on RealEstate10K and MiraData for I2V camera control task.
  arXiv  Detail & Related papers  (2025-05-28T01:45:26Z)
• ACT-R: Adaptive Camera Trajectories for Single View 3D Reconstruction [16.03389355810877]
  We introduce the simple idea of adaptive view planning to multi-view synthesis.<n>We generate a sequence of views, leveraging temporal consistency to enhance 3D coherence.
  arXiv  Detail & Related papers  (2025-05-13T05:31:59Z)
• Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation [73.73984727616198]
  We present Uni3C, a unified framework for precise control of both camera and human motion in video generation.<n>First, we propose a plug-and-play control module trained with a frozen video generative backbone, PCDController.<n>Second, we propose a jointly aligned 3D world guidance for the inference phase that seamlessly integrates both scenic point clouds and SMPL-X characters.
  arXiv  Detail & Related papers  (2025-04-21T07:10:41Z)
• CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models [89.63787060844409]
  CameraCtrl II is a framework that enables large-scale dynamic scene exploration through a camera-controlled video diffusion model.<n>We take an approach that progressively expands the generation of dynamic scenes.
  arXiv  Detail & Related papers  (2025-03-13T17:42:01Z)
• Improving Video Generation with Human Feedback [105.81833319891537]
  We develop a systematic pipeline that harnesses human feedback to mitigate video generation problems.<n>We introduce VideoReward, a multi-dimensional video reward model, and examine how annotations and various design choices impact its rewarding efficacy.
  arXiv  Detail & Related papers  (2025-01-23T18:55:41Z)
• CRAYM: Neural Field Optimization via Camera RAY Matching [48.25100687172752]
  We introduce camera ray matching (CRAYM) into the joint optimization of camera poses and neural fields from multi-view images.<n>We formulate our per-ray optimization and matched ray coherence by focusing on camera rays passing through keypoints in the input images.
  arXiv  Detail & Related papers  (2024-12-02T15:39:09Z)
• AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers [66.29824750770389]
  We analyze camera motion from a first principles perspective, uncovering insights that enable precise 3D camera manipulation.<n>We compound these findings to design the Advanced 3D Camera Control (AC3D) architecture.
  arXiv  Detail & Related papers  (2024-11-27T18:49:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.