CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation

• URL: http://arxiv.org/abs/2605.12938v1
• Date: Wed, 13 May 2026 03:18:26 GMT
• Title: CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation
• Abstract summary: We propose Ray Expectation Positional Controlled (CRePE) for camera-conditioned video generation.<n>CRePE represents each image token as a depth-aware positional distribution along its source ray, capturing the projected-path geometry induced by wide-angle and fisheye cameras.<n>Design leads to more stable camera control and improves several geometry-aware and perceptual-quality metrics.
• Score: 48.95338161181985
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Camera-conditioned video generation requires positional encoding that remains reliable under changes in camera motion, lens configuration, and scene structure. However, existing attention-level camera encodings either provide ray-only camera signals or rely on pinhole camera geometry, limiting their applicability to general camera control under the Unified Camera Model, including wide-angle and fisheye lenses. To address this limitation, we propose Curved Ray Expectation Positional Encoding (CRePE). CRePE represents each image token as a depth-aware positional distribution along its source ray, providing a Unified Camera Model-compatible positional encoding that captures the projected-path geometry induced by wide-angle and fisheye cameras. CRePE is implemented through a Geometric Attention Adapter added to frozen video DiTs, injecting token-wise scene-distance information into selected attention layers and stabilizing it with pseudo supervision from a monocular geometry foundation model. This design leads to more stable camera control and improves several geometry-aware and perceptual-quality metrics, while remaining competitive on video-quality metrics. Controlled positional-encoding ablations show a better overall average rank than a RayRoPE-style endpoint PE baseline, demonstrating the effectiveness of UCM-aware projected-path integration across diverse camera models. Furthermore, by extending the same positional-encoding pathway to external geometry control through Radial MixForcing, CRePE supports external radial-map control for scene-geometry-conditioned generation and source-video motion transfer beyond camera control.

Related papers

• MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation [29.69634930299998]
  MeRoPE (Metric Rotary Position Embedding) is a norm-preserving relative camera encoding for attention.<n>Across nuScenes and PanShot, MeRoPE achieves stronger camera control than prior encodings.
  arXiv  Detail & Related papers  (2026-09-01T13:48:31Z)
• CameraAnything: Refilming Videos with Arbitrary Camera Control [104.67227614978822]
  We introduce CameraAnything, the first unified framework for camera controlled video editing.<n>It enables joint control of both intrinsic and extrinsic camera parameters.<n>With a tailored training strategy, CameraAnything enables expressive video reshooting with arbitrary viewpoint control, focal length adjustment, resolution adaptation, and multi-shot transitions within a single generation process.
  arXiv  Detail & Related papers  (2026-07-27T15:55:08Z)
• PE-Field 4D: Video Generation Models as Canvas [44.0217294710719]
  We show that positional encoding in video diffusion transformers provides a useful spatial bias for geometry-aware control.<n>We introduce a geometry-aware cross-attention mechanism that enables target video latent tokens to attend to structured context tokens.<n>The resulting framework provides a simple and effective approach for controllable video generation.
  arXiv  Detail & Related papers  (2026-07-17T06:22:14Z)
• RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation [43.253222907352146]
  RayPE is a positional-encoding extension that injects per-token 6D Plucker coordinates additively into the queries and keys of self-attention.<n>The resulting attention score decomposes into a content term, a geometry term, and two content and geometry cross-terms.
  arXiv  Detail & Related papers  (2026-06-25T17:51:02Z)
• URoPE: Universal Relative Position Embedding across Geometric Spaces [55.651792747815854]
  URoPE is a universal extension of Rotary Position Embedding (RoPE) to cross-view or cross-dimensional geometric spaces.<n>For each key/value image patch, URoPE samples 3D points along the corresponding camera ray at predefined depth anchors.<n>Standard 2D RoPE can then be applied using the projected pixel coordinates.
  arXiv  Detail & Related papers  (2026-04-20T18:52:03Z)
• CETCAM: Camera-Controllable Video Generation via Consistent and Extensible Tokenization [32.42754288735215]
  CETCAM is a camera-controllable video generation framework.<n>It eliminates the need for camera annotations through a consistent and tokenization scheme.<n>It learns robust camera controllability from diverse raw video data and refines fine-grained visual quality using high-fidelity datasets.
  arXiv  Detail & Related papers  (2025-12-22T04:21:39Z)
• Unified Camera Positional Encoding for Controlled Video Generation [48.5789182990001]
  Transformers have emerged as a universal backbone across 3D perception, video generation, and world models for autonomous driving and embodied AI.<n>We introduce Relative Ray, a geometry-consistent representation that unifies complete camera information, including 6-DoF poses, intrinsics, and lens distortions.<n>To facilitate systematic training and evaluation, we construct a large video dataset covering a wide range of camera motions and lens types.
  arXiv  Detail & Related papers  (2025-12-08T07:34:01Z)
• DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation [51.66285725139235]
  We present DualCamCtrl, a novel end-to-end diffusion model for camera-controlled video generation.<n>We propose a dual-branch framework that mutually generates camera-consistent RGB and depth sequences.<n>Experiments demonstrate that DualCamCtrl achieves more consistent camera-controlled video generation.
  arXiv  Detail & Related papers  (2025-11-28T12:19:57Z)
• ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding [60.574308105414026]
  ReDirector is a camera-controlled video generation method for dynamically captured variable-length.<n>We introduce Rotary Camera.<n>RoCE, a camera that integrates RoPE and retake-conditioned cameras.<n>Our method generalizes to out-of-distribution camera trajectories and video lengths, yielding improved dynamic object localization and static background preservation.
  arXiv  Detail & Related papers  (2025-11-25T01:38:56Z)
• CamPVG: Camera-Controlled Panoramic Video Generation with Epipolar-Aware Diffusion [31.032317079295762]
  CamPVG is the first diffusion-based framework for panoramic video generation guided by precise camera poses.<n>We achieve camera position encoding for panoramic images and cross-view feature aggregation based on spherical projection.<n>Our method generates high-quality panoramic videos consistent with camera trajectories, far surpassing existing methods in panoramic video generation.
  arXiv  Detail & Related papers  (2025-09-24T10:34:24Z)
• IDCNet: Guided Video Diffusion for Metric-Consistent RGBD Scene Generation with Precise Camera Control [11.830304371371968]
  IDC-Net is a novel framework designed to generate RGB-D video sequences under explicit camera trajectory control.<n>We show that IDC-Net achieves improvements over state-of-the-art approaches in both visual quality and geometric consistency of generated scene sequences.
  arXiv  Detail & Related papers  (2025-08-06T07:19:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.