WeatherEdit: Controllable Weather Editing with 4D Gaussian Field

• URL: http://arxiv.org/abs/2505.20471v3
• Date: Thu, 07 Aug 2025 13:07:35 GMT
• Title: WeatherEdit: Controllable Weather Editing with 4D Gaussian Field
• Authors: Chenghao Qian, Wenjing Li, Yuhu Guo, Gustav Markkula,
• Abstract summary: We present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects in 3D scenes.<n>Our approach is structured into two key components: weather background editing and weather particle construction.<n>Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity.
• Score: 5.240297013713328
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects with controllable types and severity in 3D scenes. Our approach is structured into two key components: weather background editing and weather particle construction. For weather background editing, we introduce an all-in-one adapter that integrates multiple weather styles into a single pretrained diffusion model, enabling the generation of diverse weather effects in 2D image backgrounds. During inference, we design a Temporal-View (TV-) attention mechanism that follows a specific order to aggregate temporal and spatial information, ensuring consistent editing across multi-frame and multi-view images. To construct the weather particles, we first reconstruct a 3D scene using the edited images and then introduce a dynamic 4D Gaussian field to generate snowflakes, raindrops and fog in the scene. The attributes and dynamics of these particles are precisely controlled through physical-based modelling and simulation, ensuring realistic weather representation and flexible severity adjustments. Finally, we integrate the 4D Gaussian field with the 3D scene to render consistent and highly realistic weather effects. Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity, highlighting its potential for autonomous driving simulation in adverse weather. See project page: https://jumponthemoon.github.io/w-edit

Related papers

• SceneCrafter: Controllable Multi-View Driving Scene Editing [44.91248700043744]
  We propose SceneCrafter, a versatile editor for realistic 3D-consistent manipulation of driving scenes captured from multiple cameras.<n>SceneCrafter achieves state-of-the-art realism, controllability, 3D consistency, and scene editing quality compared to existing baselines.
  arXiv  Detail & Related papers  (2025-06-24T10:23:47Z)
• Weather-Magician: Reconstruction and Rendering Framework for 4D Weather Synthesis In Real Time [28.860317925222954]
  We propose a framework based on gaussian splatting to reconstruct real scenes and render them under synthesized 4D weather effects.<n>Our work supports continuous dynamic weather changes and can easily control the details of the effects.
  arXiv  Detail & Related papers  (2025-05-26T12:44:53Z)
• Controllable Weather Synthesis and Removal with Video Diffusion Models [61.56193902622901]
  WeatherWeaver is a video diffusion model that synthesizes diverse weather effects directly into any input video.<n>Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability.
  arXiv  Detail & Related papers  (2025-05-01T17:59:57Z)
• WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting [5.240297013713328]
  WeatherGS is a 3DGS-based framework for reconstructing clear scenes from multi-view images under different weather conditions.<n>We propose a dense-to-sparse preprocess strategy, which sequentially removes the dense particles by an Atmospheric Effect Filter.<n>Finally, we train a set of 3D Gaussians by the processed images and generated masks for excluding occluded areas.
  arXiv  Detail & Related papers  (2024-12-25T10:16:57Z)
• WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning [69.82211470647349]
  We introduce the first generalist weather foundation model (WeatherGFM)<n>It addresses a wide spectrum of weather understanding tasks in a unified manner.<n>Our model can effectively handle up to ten weather understanding tasks, including weather forecasting, super-resolution, weather image translation, and post-processing.
  arXiv  Detail & Related papers  (2024-11-08T09:14:19Z)
• MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detection [22.277210748714378]
  Existing methods mainly focus on performing 3D detection in ideal weather conditions, characterized by scenarios with clear and optimal visibility. We introduce MonoWAD, a novel weather-robust monocular 3D object detector with a weather-adaptive diffusion model. Experiments under various weather conditions demonstrate that MonoWAD achieves weather-robust monocular 3D object detection.
  arXiv  Detail & Related papers  (2024-07-23T12:58:49Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field [57.859851662796316]
  We describe a novel NeRF-editing procedure that can fuse physical simulations with NeRF models of scenes. Results are significantly more realistic than those from SOTA 2D image editing and SOTA 3D NeRF stylization.
  arXiv  Detail & Related papers  (2022-11-23T18:59:13Z)
• Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather [92.84066576636914]
  This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. We tackle this problem by simulating physically accurate fog into clear-weather scenes. We are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset.
  arXiv  Detail & Related papers  (2021-08-11T14:37:54Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.