Simulating Fluids in Real-World Still Images

• URL: http://arxiv.org/abs/2204.11335v1
• Date: Sun, 24 Apr 2022 18:47:15 GMT
• Title: Simulating Fluids in Real-World Still Images
• Authors: Siming Fan, Jingtan Piao, Chen Qian, Kwan-Yee Lin, Hongsheng Li
• Abstract summary: In this work, we tackle the problem of real-world fluid animation from a still image. The key of our system is a surface-based layered representation deriving from video decomposition. In addition, we introduce surface-only fluid simulation, a $2.5D$ fluid calculation version, as a replacement for motion estimation.
• Score: 39.93838010016248
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we tackle the problem of real-world fluid animation from a still image. The key of our system is a surface-based layered representation deriving from video decomposition, where the scene is decoupled into a surface fluid layer and an impervious background layer with corresponding transparencies to characterize the composition of the two layers. The animated video can be produced by warping only the surface fluid layer according to the estimation of fluid motions and recombining it with the background. In addition, we introduce surface-only fluid simulation, a $2.5D$ fluid calculation version, as a replacement for motion estimation. Specifically, we leverage the triangular mesh based on a monocular depth estimator to represent the fluid surface layer and simulate the motion in the physics-based framework with the inspiration of the classic theory of the hybrid Lagrangian-Eulerian method, along with a learnable network so as to adapt to complex real-world image textures. We demonstrate the effectiveness of the proposed system through comparison with existing methods in both standard objective metrics and subjective ranking scores. Extensive experiments not only indicate our method's competitive performance for common fluid scenes but also better robustness and reasonability under complex transparent fluid scenarios. Moreover, as the proposed surface-based layer representation and surface-only fluid simulation naturally disentangle the scene, interactive editing such as adding objects to the river and texture replacing could be easily achieved with realistic results.

Related papers

• A GPU-based Hydrodynamic Simulator with Boid Interactions [6.356750384481682]
  We present a hydrodynamic simulation system using the GPU compute shaders of DirectX for simulating virtual agent behaviors and navigation inside a smoothed particle hydrodynamical (SPH) fluid environment with real-time water mesh surface reconstruction. Our system is versatile enough for reinforced robotic agents instead of boid agents to interact with the fluid environment for underwater navigation and remote control engineering purposes.
  arXiv  Detail & Related papers  (2023-11-25T17:59:25Z)
• RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation [50.10282876199739]
  This paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects. It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow. A real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.
  arXiv  Detail & Related papers  (2023-11-21T07:19:47Z)
• Ghost on the Shell: An Expressive Representation of General 3D Shapes [97.76840585617907]
  Meshes are appealing since they enable fast physics-based rendering with realistic material and lighting. Recent work on reconstructing and statistically modeling 3D shapes has critiqued meshes as being topologically inflexible. We parameterize open surfaces by defining a manifold signed distance field on watertight surfaces. G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2023-10-23T17:59:52Z)
• SURFSUP: Learning Fluid Simulation for Novel Surfaces [28.90974131540538]
  We introduce SURFSUP, a framework that represents objects implicitly using signed distance functions (SDFs) This continuous representation of geometry enables more accurate simulation of fluid-object interactions over long time periods. We show we can invert our model to design simple objects to manipulate fluid flow.
  arXiv  Detail & Related papers  (2023-04-13T00:17:38Z)
• Inferring Fluid Dynamics via Inverse Rendering [37.87293082992423]
  Humans have a strong intuitive understanding of physical processes such as fluid falling by just a glimpse of such a scene picture. This work achieves such a photo-to-fluid reconstruction functionality learned from unannotated videos.
  arXiv  Detail & Related papers  (2023-04-10T08:23:17Z)
• FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation [80.63838153351804]
  We introduce FluidLab, a simulation environment with a diverse set of manipulation tasks involving complex fluid dynamics. At the heart of our platform is a fully differentiable physics simulator, providing GPU-accelerated simulations and gradient calculations. We propose several domain-specific optimization schemes coupled with differentiable physics.
  arXiv  Detail & Related papers  (2023-03-04T07:24:22Z)
• Water Simulation and Rendering from a Still Photograph [20.631819299595527]
  We propose an approach to simulate and render realistic water animation from a single still input photograph. Our approach creates realistic results with no user intervention for a wide variety of natural scenes.
  arXiv  Detail & Related papers  (2022-10-05T20:47:44Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)
• GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition [81.24107630746508]
  We present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes. We first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases.
  arXiv  Detail & Related papers  (2021-01-16T23:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.