GeoStream: Toward Precise Camera Controlled Streaming Video Generation
Abstract Overview
GeoStream is a framework for camera-controlled streaming video generation that targets precise metric-scale viewpoint control in an autoregressive setting. The method uses an explicit 3D geometric conditioning signal built from a self-refreshed 3D cache: depth is estimated from recently generated frames, unprojected into 3D, and reprojected into target views to guide subsequent synthesis. To address the mismatch between training and inference when both frames and geometry are self-generated, the authors combine off-policy warmup with fully on-policy causal distillation. The paper argues that this design improves temporal alignment of geometry under large camera motions while reducing both standard autoregressive drift and geometry-induced feedback errors.
Novelty
The paper's main novelty is bringing explicit, online-refreshed geometric conditioning into autoregressive streaming video generation rather than relying on implicit camera control or a static cache from the initial frame. It also introduces a fully on-policy distillation scheme in which the student is trained using geometric conditioning rendered from its own generated frames, directly targeting the second-order feedback loop caused by self-generated geometry.
Results
On RealEstate10K, GeoStream achieves the best overall numbers in the main comparison table, including FVD 98.1, PSNR 17.59, SSIM 0.651, LPIPS 0.336, ATE 0.816, RTE 0.032, and RRE 0.162. Ablations show that on-policy training outperforms off-policy variants, and analysis indicates stronger robustness to motion magnitude changes and trajectory perturbations compared to CameraCtrl II. The paper also reports faster inference than CameraCtrl II and Spatia on the same Wan2.2-5B backbone.
Key Points
- GeoStream refreshes its 3D cache online from recently generated frames, which keeps geometric conditioning aligned during large camera motions without using long-term fused scene memory.
- The training strategy is explicitly on-policy for both generated frames and geometric conditioning, aiming to reduce exposure bias that arises when cache updates depend on model outputs.
- Experiments on RealEstate10K report improved camera control accuracy and visual quality over recent baselines, with additional analyses showing better scale preservation and robustness under perturbed trajectories.