Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervision

• URL: http://arxiv.org/abs/2607.25321v1
• Date: Tue, 28 Jul 2026 06:13:18 GMT
• Title: Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervision
• Abstract summary: Video diffusion models generate visually compelling content but routinely violate elementary physics when the subject involves fluids.<n>We build a physics-simulation fluid dataset combining 1,638 MPM-simulated pouring/sloshing videos with 2,320 keyword-filtered real pouring videos.<n>We introduce a dual-stream image-to-video architecture built on a pretrained diffusion-filtereder video generator.
• Score: 78.04385989413382
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video diffusion models generate visually compelling content but routinely violate elementary physics when the subject involves fluids: liquid columns break apart in mid-air, container water levels fail to rise as liquid is poured in, and splashes disperse without regard to momentum or gravity. We attribute this gap to the fact that large-scale video-text corpora contain almost no explicit motion supervision, so models learn to imitate fluid appearance rather than dynamics. We address this with two contributions. First, we build a physics-simulation fluid dataset combining 1,638 MPM-simulated pouring/sloshing videos with 2,320 keyword-filtered real pouring videos mined from stock footage, plus two held-out test sets: a 1,515-video real-video benchmark and an 18-prompt text-to-first-frame generalization benchmark. Second, we introduce a dual-stream image-to-video architecture built on a pretrained diffusion-transformer video generator. It augments the standard RGB decoder with a lightweight Optical-Flow Decoder branch trained with explicit end-point-error and smoothness losses, fused into the RGB stream via zero-initialized convolutions so the pretrained backbone starts undisturbed. Only the two decoders are updated; the encoder, temporal transformer, and text encoder remain frozen. Across two model scales (1.3B and 14B) and two test sets, our method improves VideoPhy-2 Physical-Commonsense and Video-Quality scores over the frozen backbone by up to 8.75 and 4.65 points, outperforms a leading open competitor, and is preferred by human raters in a blind study. A direct optical-flow read-out evaluation further shows an end-point error as low as 0.54 pixels in-distribution, confirming the model has internalized a coherent motion prior rather than merely improving surface appearance.

Related papers

• PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control [85.54736105647055]
  PhysStream is an autoregressive model for physics-grounded image-to-video synthesis.<n>It supports fine-grained motion control via sparse velocity-increment signals.<n>PhysStream is preferred by human evaluators in over 85% of in-the-wild comparisons.
  arXiv  Detail & Related papers  (2026-09-15T17:55:13Z)
• Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation [78.41956766048308]
  Proprio is a training-free framework that enables a frozen generator to assess and improve the physical plausibility of its own outputs.<n>Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual as a self-scoring signal.<n>Proprio consistently improves physical plausibility across text-to-video and image-to-video benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T09:44:18Z)
• 3DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation [13.662206166615098]
  This work introduces 3DPhysVideo, a novel training-free pipeline that generates physically realistic videos from a single image.<n>We use it as a novel view synthesizer to reconstruct complete 360-degree 3D scene geometry by guiding the image-to-video (I2V) flow model with rendered point clouds.<n>In the diverse experiments including multi-objects, and fluid interaction scenes, our method successfully bridges the gap from single-images to physically plausible videos.
  arXiv  Detail & Related papers  (2026-05-16T03:56:52Z)
• MMPhysVideo: Scaling Physical Plausibility in Video Generation via Joint Multimodal Modeling [55.72785604682579]
  MMPhysVideo is a framework to scale physical plausibility in video generation in joint multimodal modeling.<n>MMPhysVideo consistently improves physical plausibility and visual quality over advanced models across various benchmarks.
  arXiv  Detail & Related papers  (2026-04-03T07:32:24Z)
• Physical Simulator In-the-Loop Video Generation [96.87054314612142]
  Physical Simulator In-the-loop Video Generation (PSIVG) is a novel framework that integrates a physical simulator into the video diffusion process.<n>PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity.
  arXiv  Detail & Related papers  (2026-03-06T15:48:25Z)
• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models [48.35054927704544]
  Current video diffusion models achieve impressive generation quality but struggle in interactive applications due to bidirectional attention dependencies.<n>We address this limitation by adapting a pretrained bidirectional diffusion transformer to an autoregressive transformer that generates frames on-the-fly.<n>Our model achieves a total score of 84.27 on the VBench-Long benchmark, surpassing all previous video generation models.
  arXiv  Detail & Related papers  (2024-12-10T18:59:50Z)
• AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction [88.70116693750452]
  Text-guided video prediction (TVP) involves predicting the motion of future frames from the initial frame according to an instruction. Previous TVP methods make significant breakthroughs by adapting Stable Diffusion for this task. We introduce the Multi-Modal Large Language Model (MLLM) to predict future video states based on initial frames and text instructions.
  arXiv  Detail & Related papers  (2024-06-10T17:02:08Z)
• All in One: Exploring Unified Video-Language Pre-training [44.22059872694995]
  We introduce an end-to-end video-language model, namely textitall-in-one Transformer, that embeds raw video and textual signals into joint representations. The code and pretrained model have been released in https://github.com/showlab/all-in-one.
  arXiv  Detail & Related papers  (2022-03-14T17:06:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.