Inferring Fluid Dynamics via Inverse Rendering
- URL: http://arxiv.org/abs/2304.04446v1
- Date: Mon, 10 Apr 2023 08:23:17 GMT
- Title: Inferring Fluid Dynamics via Inverse Rendering
- Authors: Jinxian Liu, Ye Chen, Bingbing Ni, Jiyao Mao, Zhenbo Yu
- Abstract summary: Humans have a strong intuitive understanding of physical processes such as fluid falling by just a glimpse of such a scene picture.
This work achieves such a photo-to-fluid reconstruction functionality learned from unannotated videos.
- Score: 37.87293082992423
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Humans have a strong intuitive understanding of physical processes such as
fluid falling by just a glimpse of such a scene picture, i.e., quickly derived
from our immersive visual experiences in memory. This work achieves such a
photo-to-fluid-dynamics reconstruction functionality learned from unannotated
videos, without any supervision of ground-truth fluid dynamics. In a nutshell,
a differentiable Euler simulator modeled with a ConvNet-based pressure
projection solver, is integrated with a volumetric renderer, supporting
end-to-end/coherent differentiable dynamic simulation and rendering. By
endowing each sampled point with a fluid volume value, we derive a NeRF-like
differentiable renderer dedicated from fluid data; and thanks to this
volume-augmented representation, fluid dynamics could be inversely inferred
from the error signal between the rendered result and ground-truth video frame
(i.e., inverse rendering). Experiments on our generated Fluid Fall datasets and
DPI Dam Break dataset are conducted to demonstrate both effectiveness and
generalization ability of our method.
Related papers
- DiffFluid: Plain Diffusion Models are Effective Predictors of Flow Dynamics [16.660107496540146]
We showcase the plain diffusion models with Transformers as effective predictors of fluid dynamics under various working conditions.
Our approach formulates the prediction of flow dynamics as the image translation problem and accordingly leverage the plain diffusion model to tackle the problem.
arXiv Detail & Related papers (2024-09-20T17:19:03Z) - Inpainting Computational Fluid Dynamics with Deep Learning [8.397730500554047]
An effective fluid data completion method reduces the required number of sensors in a fluid dynamics experiment.
The ill-posed nature of the fluid data completion problem makes it prohibitively difficult to obtain a theoretical solution.
We employ the vector quantization technique to map both complete and incomplete fluid data spaces onto discrete-valued lower-dimensional representations.
arXiv Detail & Related papers (2024-02-27T03:44:55Z) - Vision-Informed Flow Image Super-Resolution with Quaternion Spatial
Modeling and Dynamic Flow Convolution [49.45309818782329]
Flow image super-resolution (FISR) aims at recovering high-resolution turbulent velocity fields from low-resolution flow images.
Existing FISR methods mainly process the flow images in natural image patterns.
We propose the first flow visual property-informed FISR algorithm.
arXiv Detail & Related papers (2024-01-29T06:48:16Z) - Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering [41.589093951039814]
We integrate physics-based animations of solids and fluids with 3D Gaussian Splatting (3DGS) to create novel effects in virtual scenes reconstructed using 3DGS.
Our framework is capable of realistically reproducing surface highlights on dynamic fluids and facilitating interactions between scene objects and fluids from new views.
arXiv Detail & Related papers (2024-01-27T06:45:22Z) - FluidLab: A Differentiable Environment for Benchmarking Complex Fluid
Manipulation [80.63838153351804]
We introduce FluidLab, a simulation environment with a diverse set of manipulation tasks involving complex fluid dynamics.
At the heart of our platform is a fully differentiable physics simulator, providing GPU-accelerated simulations and gradient calculations.
We propose several domain-specific optimization schemes coupled with differentiable physics.
arXiv Detail & Related papers (2023-03-04T07:24:22Z) - RISP: Rendering-Invariant State Predictor with Differentiable Simulation
and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
Existing solutions require massive training data or lack generalizability to unknown rendering configurations.
We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem.
Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
arXiv Detail & Related papers (2022-05-11T17:59:51Z) - Simulating Fluids in Real-World Still Images [39.93838010016248]
In this work, we tackle the problem of real-world fluid animation from a still image.
The key of our system is a surface-based layered representation deriving from video decomposition.
In addition, we introduce surface-only fluid simulation, a $2.5D$ fluid calculation version, as a replacement for motion estimation.
arXiv Detail & Related papers (2022-04-24T18:47:15Z) - NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural
Radiance Fields [65.07940731309856]
Deep learning has shown great potential for modeling the physical dynamics of complex particle systems such as fluids.
In this paper, we consider a partially observable scenario known as fluid dynamics grounding.
We propose a differentiable two-stage network named NeuroFluid.
It is shown to reasonably estimate the underlying physics of fluids with different initial shapes, viscosity, and densities.
arXiv Detail & Related papers (2022-03-03T15:13:29Z) - Controllable Animation of Fluid Elements in Still Images [9.194534529360691]
We propose a method to interactively control the animation of fluid elements in still images to generate cinemagraphs.
We represent the motion of such fluid elements in the image in the form of a constant 2D optical flow map.
We devise a novel UNet based architecture to autoregressively generate future frames using the refined optical flow map.
arXiv Detail & Related papers (2021-12-06T13:53:08Z) - gradSim: Differentiable simulation for system identification and
visuomotor control [66.37288629125996]
We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering.
Our unified graph enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision.
arXiv Detail & Related papers (2021-04-06T16:32:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.