World-R1: Reinforcing 3D Constraints for Text-to-Video Generation

• URL: http://arxiv.org/abs/2604.24764v1
• Date: Mon, 27 Apr 2026 17:59:56 GMT
• Title: World-R1: Reinforcing 3D Constraints for Text-to-Video Generation
• Abstract summary: We propose World-R1, a framework that aligns video generation with 3D constraints through reinforcement learning.<n>We optimize the model using feedback from pre-trained 3D foundation models and vision-language models.<n>Our approach significantly enhances 3D consistency while preserving the original visual quality of the foundation model.
• Score: 36.18760315762835
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent video foundation models demonstrate impressive visual synthesis but frequently suffer from geometric inconsistencies. While existing methods attempt to inject 3D priors via architectural modifications, they often incur high computational costs and limit scalability. We propose World-R1, a framework that aligns video generation with 3D constraints through reinforcement learning. To facilitate this alignment, we introduce a specialized pure text dataset tailored for world simulation. Utilizing Flow-GRPO, we optimize the model using feedback from pre-trained 3D foundation models and vision-language models to enforce structural coherence without altering the underlying architecture. We further employ a periodic decoupled training strategy to balance rigid geometric consistency with dynamic scene fluidity. Extensive evaluations reveal that our approach significantly enhances 3D consistency while preserving the original visual quality of the foundation model, effectively bridging the gap between video generation and scalable world simulation.

Related papers

• Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video [69.94904173314505]
  We propose Ground4D, a geometry-grounded framework built on two stages.<n>First, we reconstruct multi-view-consistent 3D geometry and camera poses from monocular video.<n>Second, we conduct geometry-consistency-aware refinement via dynamic Gaussian Splatting.
  arXiv  Detail & Related papers  (2026-06-27T09:27:42Z)
• FlowObject: Flow Steering for Bridging Generative Priors and Reconstruction Fidelity [34.142288038459704]
  FlowObject is a framework that reformulates sparse-view 3D reconstruction as a training-free, guided inverse problem.<n>Our method significantly outperforms state-of-the-art generative models and optimization-based frameworks in both geometric completeness and view-dependent appearance fidelity.
  arXiv  Detail & Related papers  (2026-06-17T12:42:09Z)
• GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion [29.999238067855245]
  We propose GTA, a novel image-to-3D world generation method following a Geometry-Then-Appearance paradigm.<n>Specifically, given a single input image, GTA adopts a two-stage framework with two dedicated video diffusion models.<n>Extensive experiments have demonstrated that our proposed method consistently outperforms existing approaches in terms of fidelity, visual quality, and geometric accuracy.
  arXiv  Detail & Related papers  (2026-05-13T03:43:02Z)
• Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors [61.34273238077091]
  We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-14T05:35:46Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction [57.46712611558817]
  3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass.<n>Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry.<n>We propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies.
  arXiv  Detail & Related papers  (2025-12-02T02:22:20Z)
• MoRE: 3D Visual Geometry Reconstruction Meets Mixture-of-Experts [50.37005070020306]
  MoRE is a dense 3D visual foundation model based on a Mixture-of-Experts (MoE) architecture.<n>MoRE incorporates a confidence-based depth refinement module that stabilizes and refines geometric estimation.<n>It integrates dense semantic features with globally aligned 3D backbone representations for high-fidelity surface normal prediction.
  arXiv  Detail & Related papers  (2025-10-31T06:54:27Z)
• Epipolar Geometry Improves Video Generation Models [73.44978239787501]
  3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks.<n>We explore how epipolar geometry constraints improve modern video diffusion models.<n>By bridging data-driven deep learning with classical geometric computer vision, we present a practical method for generating spatially consistent videos.
  arXiv  Detail & Related papers  (2025-10-24T16:21:37Z)
• End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards [8.953379216683732]
  We propose an end-to-end differentiable, reinforcement-learning-free framework that embeds human feedback, expressed as differentiable reward functions, directly into the 3D texture pipeline.<n>By back-propagating preference signals through both geometric and appearance modules, our method generates textures that respect the 3D geometry structure and align with desired criteria.
  arXiv  Detail & Related papers  (2025-06-23T06:24:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.