WeatherEdit: Controllable Weather Editing with 4D Gaussian Field

• URL: http://arxiv.org/abs/2505.20471v3
• Date: Thu, 07 Aug 2025 13:07:35 GMT
• Title: WeatherEdit: Controllable Weather Editing with 4D Gaussian Field
• Abstract summary: We present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects in 3D scenes.<n>Our approach is structured into two key components: weather background editing and weather particle construction.<n>Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity.
• Score: 5.240297013713328
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects with controllable types and severity in 3D scenes. Our approach is structured into two key components: weather background editing and weather particle construction. For weather background editing, we introduce an all-in-one adapter that integrates multiple weather styles into a single pretrained diffusion model, enabling the generation of diverse weather effects in 2D image backgrounds. During inference, we design a Temporal-View (TV-) attention mechanism that follows a specific order to aggregate temporal and spatial information, ensuring consistent editing across multi-frame and multi-view images. To construct the weather particles, we first reconstruct a 3D scene using the edited images and then introduce a dynamic 4D Gaussian field to generate snowflakes, raindrops and fog in the scene. The attributes and dynamics of these particles are precisely controlled through physical-based modelling and simulation, ensuring realistic weather representation and flexible severity adjustments. Finally, we integrate the 4D Gaussian field with the 3D scene to render consistent and highly realistic weather effects. Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity, highlighting its potential for autonomous driving simulation in adverse weather. See project page: https://jumponthemoon.github.io/w-edit

Related papers

• Multi-Sensor Alignment for Weather Simulations [1.9021908527898033]
  Perception tasks for autonomous vehicles need to work satisfactorily in adverse weather conditions.<n>To ensure simulations closely mirror real-world weather data, it's crucial that they represent the same weather characteristics, including severity and particle positioning, across different sensors.<n>We propose the Reference dataset Alignment Method (ReDAM) for weather intensity alignment in fog and Unified-weather-edit (inspired by Weather-edit) for particle positioning alignment in rain and snow.
  arXiv  Detail & Related papers  (2026-07-28T11:43:17Z)
• Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis [81.61856788730198]
  Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion.<n>Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics.<n>We propose a Semantic-Aware, Physics-Informed, and Geometry-Grounded framework that steers an off-the-shelf video editor to synthesize diverse global appearances and detailed particle dynamics.
  arXiv  Detail & Related papers  (2026-06-27T17:38:21Z)
• CityRAG: Stepping Into a City via Spatially-Grounded Video Generation [61.843558834675065]
  We present CityRAG, a video generative model that generates a 3D-consistent, navigable environment that is spatially grounded.<n>CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes.<n>Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
  arXiv  Detail & Related papers  (2026-04-21T17:59:03Z)
• NimbusGS: Unified 3D Scene Reconstruction under Hybrid Weather [83.14198482357388]
  We present NimbusGS, a unified framework for reconstructing high-quality 3D scenes from degraded multi-view inputs.<n>Unlike existing methods that target specific weather types, NimbusGS addresses the broader challenge of generalization.
  arXiv  Detail & Related papers  (2026-03-28T10:46:29Z)
• WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation [19.48387412274678]
  We propose WeatherCity, a novel framework for 4D urban scene reconstruction and weather editing.<n> Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds.<n>Our framework achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing.
  arXiv  Detail & Related papers  (2026-02-25T16:44:12Z)
• Free-Form Scene Editor: Enabling Multi-Round Object Manipulation like in a 3D Engine [83.0145525456509]
  We present FFSE, a 3D-aware framework designed to enable intuitive, physically-consistent object editing on real-world images.<n>Unlike previous approaches that either operate in image space or require slow and error-prone 3D reconstruction, FFSE models editing as a sequence of learned 3D transformations.<n>To support learning of multi-round 3D-aware object manipulation, we introduce 3DObjectEditor.
  arXiv  Detail & Related papers  (2025-11-17T18:57:39Z)
• SceneCrafter: Controllable Multi-View Driving Scene Editing [44.91248700043744]
  We propose SceneCrafter, a versatile editor for realistic 3D-consistent manipulation of driving scenes captured from multiple cameras.<n>SceneCrafter achieves state-of-the-art realism, controllability, 3D consistency, and scene editing quality compared to existing baselines.
  arXiv  Detail & Related papers  (2025-06-24T10:23:47Z)
• Weather-Magician: Reconstruction and Rendering Framework for 4D Weather Synthesis In Real Time [28.860317925222954]
  We propose a framework based on gaussian splatting to reconstruct real scenes and render them under synthesized 4D weather effects.<n>Our work supports continuous dynamic weather changes and can easily control the details of the effects.
  arXiv  Detail & Related papers  (2025-05-26T12:44:53Z)
• Controllable Weather Synthesis and Removal with Video Diffusion Models [61.56193902622901]
  WeatherWeaver is a video diffusion model that synthesizes diverse weather effects directly into any input video.<n>Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability.
  arXiv  Detail & Related papers  (2025-05-01T17:59:57Z)
• WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting [5.240297013713328]
  WeatherGS is a 3DGS-based framework for reconstructing clear scenes from multi-view images under different weather conditions.<n>We propose a dense-to-sparse preprocess strategy, which sequentially removes the dense particles by an Atmospheric Effect Filter.<n>Finally, we train a set of 3D Gaussians by the processed images and generated masks for excluding occluded areas.
  arXiv  Detail & Related papers  (2024-12-25T10:16:57Z)
• WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning [69.82211470647349]
  We introduce the first generalist weather foundation model (WeatherGFM)<n>It addresses a wide spectrum of weather understanding tasks in a unified manner.<n>Our model can effectively handle up to ten weather understanding tasks, including weather forecasting, super-resolution, weather image translation, and post-processing.
  arXiv  Detail & Related papers  (2024-11-08T09:14:19Z)
• MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detection [22.277210748714378]
  Existing methods mainly focus on performing 3D detection in ideal weather conditions, characterized by scenarios with clear and optimal visibility. We introduce MonoWAD, a novel weather-robust monocular 3D object detector with a weather-adaptive diffusion model. Experiments under various weather conditions demonstrate that MonoWAD achieves weather-robust monocular 3D object detection.
  arXiv  Detail & Related papers  (2024-07-23T12:58:49Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field [57.859851662796316]
  We describe a novel NeRF-editing procedure that can fuse physical simulations with NeRF models of scenes. Results are significantly more realistic than those from SOTA 2D image editing and SOTA 3D NeRF stylization.
  arXiv  Detail & Related papers  (2022-11-23T18:59:13Z)
• Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather [92.84066576636914]
  This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. We tackle this problem by simulating physically accurate fog into clear-weather scenes. We are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset.
  arXiv  Detail & Related papers  (2021-08-11T14:37:54Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.