ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field

• URL: http://arxiv.org/abs/2211.13226v3
• Date: Thu, 8 Jun 2023 06:14:30 GMT
• Title: ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field
• Authors: Yuan Li, Zhi-Hao Lin, David Forsyth, Jia-Bin Huang, Shenlong Wang
• Abstract summary: We describe a novel NeRF-editing procedure that can fuse physical simulations with NeRF models of scenes. Results are significantly more realistic than those from SOTA 2D image editing and SOTA 3D NeRF stylization.
• Score: 57.859851662796316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physical simulations produce excellent predictions of weather effects. Neural radiance fields produce SOTA scene models. We describe a novel NeRF-editing procedure that can fuse physical simulations with NeRF models of scenes, producing realistic movies of physical phenomena in those scenes. Our application -- Climate NeRF -- allows people to visualize what climate change outcomes will do to them. ClimateNeRF allows us to render realistic weather effects, including smog, snow, and flood. Results can be controlled with physically meaningful variables like water level. Qualitative and quantitative studies show that our simulated results are significantly more realistic than those from SOTA 2D image editing and SOTA 3D NeRF stylization.

Related papers

• Controllable Weather Synthesis and Removal with Video Diffusion Models [61.56193902622901]
  WeatherWeaver is a video diffusion model that synthesizes diverse weather effects directly into any input video. Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability.
  arXiv  Detail & Related papers  (2025-05-01T17:59:57Z)
• RainyGS: Efficient Rain Synthesis with Physically-Based Gaussian Splatting [28.60412760466588]
  We introduce RainyGS, a novel approach to generate dynamic rain effects in open-world scenes with physical accuracy. At the core of our method is the integration of physically-based raindrop and shallow water simulation techniques within the fast 3DGS rendering framework. Our method supports synthesizing rain effects at over 30 fps, offering users flexible control over rain intensity.
  arXiv  Detail & Related papers  (2025-03-27T12:35:03Z)
• ClimateGS: Real-Time Climate Simulation with 3D Gaussian Style Transfer [11.17376076195671]
  ClimateGS is a novel framework integrating 3D Gaussian representations with physical simulation to enable real-time climate effects rendering. We evaluate ClimateGS on MipNeRF360 and Tanks and Temples, demonstrating real-time rendering with comparable or superior visual quality to SOTA 2D/3D methods.
  arXiv  Detail & Related papers  (2025-03-19T03:01:35Z)
• Taming Latent Diffusion Model for Neural Radiance Field Inpainting [63.297262813285265]
  Neural Radiance Field (NeRF) is a representation for 3D reconstruction from multi-view images. We propose tempering the diffusion model'sity with per-scene customization and mitigating the textural shift with masked training. Our framework yields state-of-the-art NeRF inpainting results on various real-world scenes.
  arXiv  Detail & Related papers  (2024-04-15T17:59:57Z)
• FaIRGP: A Bayesian Energy Balance Model for Surface Temperatures Emulation [13.745581787463962]
  We introduce FaIRGP, a data-driven emulator that satisfies the physical temperature response equations of an energy balance model. We show how FaIRGP can be used to obtain estimates of top-of-atmosphere radiative forcing. We hope that this work will contribute to widening the adoption of data-driven methods in climate emulation.
  arXiv  Detail & Related papers  (2023-07-14T08:43:36Z)
• Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields [2.5432277893532116]
  Neural Radiance Fields (NeRF) can generate realistic images from any viewpoint. NeRF-based models are susceptible to interference issues caused by colored "fog" noise. Our approach, coined Enhance-NeRF, adopts joint color to balance low and high reflectivity objects display.
  arXiv  Detail & Related papers  (2023-06-08T15:49:30Z)
• BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields [9.744593647024253]
  We present a novel bundle adjusted deblur Neural Radiance Fields (BAD-NeRF) BAD-NeRF can be robust to severe motion blurred images and inaccurate camera poses. Our approach models the physical image formation process of a motion blurred image, and jointly learns the parameters of NeRF.
  arXiv  Detail & Related papers  (2022-11-23T10:53:37Z)
• FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields [64.17946473855382]
  This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs) Our 3D coarse-to-fine Face Landmarks NeRF (FLNeRF) model efficiently samples from a given face NeRF with individual facial features for accurate landmarks detection.
  arXiv  Detail & Related papers  (2022-11-21T06:26:01Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• BARF: Bundle-Adjusting Neural Radiance Fields [104.97810696435766]
  We propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect camera poses. BARF can effectively optimize the neural scene representations and resolve large camera pose misalignment at the same time. This enables view synthesis and localization of video sequences from unknown camera poses, opening up new avenues for visual localization systems.
  arXiv  Detail & Related papers  (2021-04-13T17:59:51Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.