Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis

• URL: http://arxiv.org/abs/2606.29020v1
• Date: Sat, 27 Jun 2026 17:38:21 GMT
• Title: Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis
• Abstract summary: Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion.<n>Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics.<n>We propose a Semantic-Aware, Physics-Informed, and Geometry-Grounded framework that steers an off-the-shelf video editor to synthesize diverse global appearances and detailed particle dynamics.
• Score: 81.61856788730198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion. The key limitation of existing methods is the lack of diversity in weather appearance and effective control over weather dynamics (e.g., temporal evolution and particle motion). Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics. Additionally, general-purpose video editors optimized for clean and aesthetic outputs tend to suppress heavy weather phenomena, making dense particle effects difficult to generate. To address these, we propose a Semantic-Aware, Physics-Informed, and Geometry-Grounded framework that steers an off-the-shelf video editor to synthesize diverse global appearances and detailed particle dynamics. We factorize the synthesis into three conditional signals, so that each provides a distinct and stable source of guidance: semantics specifies what the weather should look like, dynamics governs how it evolves over time, and geometry determines where it should appear in the scene. Specifically, we introduce (1) semantic-aware appearance anchoring to establish the target appearance from scene semantics and user input; (2) physics-informed dynamic simulation to generate particle effects by simulating a Gaussian-represented particle field under gravity, wind, and turbulence; and (3) geometry-grounded video synthesis to align the simulated particles with target scene geometry and synthesize the final video. Experiments demonstrate that our method produces diverse, physically and visually realistic weather effects. Furthermore, we show that our synthesized data significantly improves the robustness of autonomous driving semantic segmentation under adverse weather conditions. Project page: https://jumponthemoon.github.io/w-crafter/.

Related papers

• RealWeather: Realistic and Scene-Faithful Weather Translation with Driving World Models [42.84298567152532]
  RealWeather is a driving world model for both realistic and scene-faithful weather translation.<n>Our key idea is to learn authentic weather dynamics directly from real-world videos.<n>Experiments demonstrate that RealWeather significantly outperforms existing methods in visual realism and structural preservation.
  arXiv  Detail & Related papers  (2026-08-03T23:33:49Z)
• Physical Simulator In-the-Loop Video Generation [96.87054314612142]
  Physical Simulator In-the-loop Video Generation (PSIVG) is a novel framework that integrates a physical simulator into the video diffusion process.<n>PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity.
  arXiv  Detail & Related papers  (2026-03-06T15:48:25Z)
• WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation [19.48387412274678]
  We propose WeatherCity, a novel framework for 4D urban scene reconstruction and weather editing.<n> Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds.<n>Our framework achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing.
  arXiv  Detail & Related papers  (2026-02-25T16:44:12Z)
• What Happens Next? Anticipating Future Motion by Generating Point Trajectories [76.16266402727643]
  We consider the problem of forecasting motion from a single image, predicting how objects in the world are likely to move.<n>We formulate this task as conditional generation of dense trajectory grids with a model that closely follows the architecture of modern video generators.<n>This approach captures scene-wide dynamics and uncertainty, yielding more accurate and diverse predictions than prior regressors and generators.
  arXiv  Detail & Related papers  (2025-09-25T21:03:56Z)
• WeatherEdit: Controllable Weather Editing with 4D Gaussian Field [5.240297013713328]
  We present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects in 3D scenes.<n>Our approach is structured into two key components: weather background editing and weather particle construction.<n>Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity.
  arXiv  Detail & Related papers  (2025-05-26T19:10:47Z)
• Controllable Weather Synthesis and Removal with Video Diffusion Models [61.56193902622901]
  WeatherWeaver is a video diffusion model that synthesizes diverse weather effects directly into any input video.<n>Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability.
  arXiv  Detail & Related papers  (2025-05-01T17:59:57Z)
• Let it Snow! Animating Static Gaussian Scenes With Dynamic Weather Effects [25.126055173812187]
  3D Gaussian Splatting has recently enabled fast and photorealistic reconstruction of static 3D scenes.<n>We present a novel framework that combines Gaussian-particle representations for incorporating physically-based global weather effects into static scenes.<n>Our approach supports a variety of weather effects, including snowfall, rainfall, fog, and sandstorms, and can also support falling objects.
  arXiv  Detail & Related papers  (2025-04-07T17:51:21Z)
• High-Fidelity Neural Human Motion Transfer from Monocular Video [71.75576402562247]
  Video-based human motion transfer creates video animations of humans following a source motion. We present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations. In the experimental results, we significantly outperform the state-of-the-art in terms of video realism.
  arXiv  Detail & Related papers  (2020-12-20T16:54:38Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.