3DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation

• URL: http://arxiv.org/abs/2605.16795v1
• Date: Sat, 16 May 2026 03:56:52 GMT
• Title: 3DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation
• Abstract summary: This work introduces 3DPhysVideo, a novel training-free pipeline that generates physically realistic videos from a single image.<n>We use it as a novel view synthesizer to reconstruct complete 360-degree 3D scene geometry by guiding the image-to-video (I2V) flow model with rendered point clouds.<n>In the diverse experiments including multi-objects, and fluid interaction scenes, our method successfully bridges the gap from single-images to physically plausible videos.
• Score: 13.662206166615098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video generative models have made remarkable progress, yet they often yield visual artifacts that violate grounding in physical dynamics. Recent works such as PhysGen3D tackle single image-to-3D physics through mesh reconstruction and Physically-Based Rendering, but challenges remain in modeling fluid dynamics, multi-object interactions and photorealism. This work introduces 3DPhysVideo, a novel training-free pipeline that generates physically realistic videos from a single image. We repurpose an off-the-shelf video model for two stages. First, we use it as a novel view synthesizer to reconstruct complete 360-degree 3D scene geometry by guiding the image-to-video (I2V) flow model with rendered point clouds. Second, after applying physics solvers to this geometry, the physically simulated point cloud is used to guide the same I2V flow model to synthesize final, high-quality videos. Consistency-Guided Flow SDE, which decomposes the predicted velocity of the I2V flow model into denoising and consistency bias, enforces consistency to the conditional inputs, allowing us to effectively repurpose the model for both 3D reconstruction and simulation-guided video generation. In the diverse experiments including multi-objects, and fluid interaction scenes, our method successfully bridges the gap from single-images to physically plausible videos, while remaining efficient to run on a single consumer GPU. It outperforms state-of-the-art baselines on GPT-based scores, VideoPhy benchmark and human evaluation.

Related papers

• World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video [62.16750462940772]
  We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos.<n>Our approach conditions a video model on dense, pixel-aligned renderings that encode appearance, geometry, and 3D scene motion.<n>At test time, we distill the model's generations back into a single consistent, high-quality dynamic 3DGS, improving both novel-view synthesis and the underlying 3D motion.
  arXiv  Detail & Related papers  (2026-07-01T17:41:00Z)
• Neural Voxel Dynamics: Learning Implicit 3D Physics via Volumetric Feature Advection [0.14323566945483496]
  We present a self-dimensional framework for learning implicit 3D physical dynamics directly from video-derived supervisory signals.<n>We address this by shifting the predictive bottleneck from 2D image space to a lifted' 3D Volumetric Latent Space.<n>Unlike state-of-the-art hybrid models that rely on explicit classical simulators for training and/or inference, our architecture tracks material states implicitly within high-supervised V-JEPA features.
  arXiv  Detail & Related papers  (2026-06-24T22:06:35Z)
• TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction [51.01447538245441]
  We propose TelePhysics, a training-free framework that converts a single image into a physically consistent and controllable video.<n>By representing the full scene geometry in a unified spatial coordinate system, TelePhysics resolves object penetration and alignment ambiguity.<n> Experimental results demonstrate that TelePhysics substantially outperforms prior methods in physical fidelity, spatial coherence, and controllability.
  arXiv  Detail & Related papers  (2026-05-19T08:16:44Z)
• PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment [23.066204478030627]
  Video Models (VDMs) offer a promising approach for simulating dynamic scenes and environments.<n>Existing models often generate temporally incoherent content that violates basic physical intuition.<n>We propose PhysAlign, an efficient framework for physics-coherent image-to-videoI (2V) generation.
  arXiv  Detail & Related papers  (2026-03-14T05:44:18Z)
• Physical Simulator In-the-Loop Video Generation [96.87054314612142]
  Physical Simulator In-the-loop Video Generation (PSIVG) is a novel framework that integrates a physical simulator into the video diffusion process.<n>PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity.
  arXiv  Detail & Related papers  (2026-03-06T15:48:25Z)
• RealWonder: Real-Time Physical Action-Conditioned Video Generation [31.747349682347167]
  We present RealWonder, the first real-time system for action-conditioned video generation from a single image.<n>RealWonder integrates 3D reconstruction from single images, physics simulation, and a distilled video generator requiring only 4 diffusion steps.<n>Our system achieves 13.2 FPS at 480x832 resolution, enabling interactive exploration of forces, robot actions, and camera controls on rigid objects.
  arXiv  Detail & Related papers  (2026-03-05T18:22:54Z)
• ViSA: 3D-Aware Video Shading for Real-Time Upper-Body Avatar Creation [62.86900540547787]
  Current 3D avatar generation methods often suffer from artifacts such as blurry textures and stiff, unnatural motion.<n>We propose a novel approach that combines the strengths of both paradigms.<n>By uniting the geometric stability of 3D reconstruction with the generative capabilities of video models, our method produces high-fidelity digital avatars.
  arXiv  Detail & Related papers  (2025-12-08T17:10:29Z)
• Restage4D: Reanimating Deformable 3D Reconstruction from a Single Video [56.781766315691854]
  We introduce textbfRestage4D, a geometry-preserving pipeline for video-conditioned 4D restaging.<n>We validate Restage4D on DAVIS and PointOdyssey, demonstrating improved geometry consistency, motion quality, and 3D tracking performance.
  arXiv  Detail & Related papers  (2025-08-08T21:31:51Z)
• PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation [29.831214435147583]
  We present PhysGen, a novel image-to-video generation method. It produces a realistic, physically plausible, and temporally consistent video. Our key insight is to integrate model-based physical simulation with a data-driven video generation process.
  arXiv  Detail & Related papers  (2024-09-27T17:59:57Z)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors.<n>We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.