WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion

• URL: http://arxiv.org/abs/2504.13561v1
• Date: Fri, 18 Apr 2025 09:01:07 GMT
• Title: WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion
• Authors: Yang Wu, Yun Zhu, Kaihua Zhang, Jianjun Qian, Jin Xie, Jian Yang,
• Abstract summary: 3D scene perception demands a large amount of adverse-weather LiDAR data.<n>Yet, the cost of LiDAR data collection presents a significant scaling-up challenge.<n>This paper presents WeatherGen, the first unified diverse-weather LiDAR data diffusion generation framework.
• Score: 39.36578688743474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D scene perception demands a large amount of adverse-weather LiDAR data, yet the cost of LiDAR data collection presents a significant scaling-up challenge. To this end, a series of LiDAR simulators have been proposed. Yet, they can only simulate a single adverse weather with a single physical model, and the fidelity of the generated data is quite limited. This paper presents WeatherGen, the first unified diverse-weather LiDAR data diffusion generation framework, significantly improving fidelity. Specifically, we first design a map-based data producer, which can provide a vast amount of high-quality diverse-weather data for training purposes. Then, we utilize the diffusion-denoising paradigm to construct a diffusion model. Among them, we propose a spider mamba generator to restore the disturbed diverse weather data gradually. The spider mamba models the feature interactions by scanning the LiDAR beam circle or central ray, excellently maintaining the physical structure of the LiDAR data. Subsequently, following the generator to transfer real-world knowledge, we design a latent feature aligner. Afterward, we devise a contrastive learning-based controller, which equips weather control signals with compact semantic knowledge through language supervision, guiding the diffusion model to generate more discriminative data. Extensive evaluations demonstrate the high generation quality of WeatherGen. Through WeatherGen, we construct the mini-weather dataset, promoting the performance of the downstream task under adverse weather conditions. Code is available: https://github.com/wuyang98/weathergen

Related papers

• SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal [57.52777076116241]
  Adverse weather removal aims to restore clear vision under adverse weather conditions. This paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network.
  arXiv  Detail & Related papers  (2024-09-29T12:12:22Z)
• TripleMixer: A 3D Point Cloud Denoising Model for Adverse Weather [6.752848431431841]
  Real-world adverse weather conditions, such as rain, fog, and snow, introduce significant noise and interference. Existing datasets often suffer from limited weather diversity and small dataset sizes. We propose a novel point cloud denoising model, TripleMixer, comprising three mixer layers.
  arXiv  Detail & Related papers  (2024-08-25T10:45:52Z)
• RangeLDM: Fast Realistic LiDAR Point Cloud Generation [12.868053836790194]
  We introduce RangeLDM, a novel approach for rapidly generating high-quality range-view LiDAR point clouds. We achieve this by correcting range-view data distribution for accurate projection from point clouds to range images via Hough voting. We instruct the model to preserve 3D structural fidelity by devising a range-guided discriminator.
  arXiv  Detail & Related papers  (2024-03-15T08:19:57Z)
• Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator [17.605152926277707]
  We propose a simulator-based physical modeling approach to augment LiDAR data in rainy weather. We complete the modeling task of the rainy weather in the CARLA simulator and establish a pipeline for LiDAR data collection. In the experiment, we observe that the model augmented by the synthetic data improves the object detection task's performance.
  arXiv  Detail & Related papers  (2023-12-20T05:16:04Z)
• UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation [51.443788294845845]
  We present UltraLiDAR, a data-driven framework for scene-level LiDAR completion, LiDAR generation, and LiDAR manipulation. We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds. By learning a prior over the discrete codebook, we can generate diverse, realistic LiDAR point clouds for self-driving.
  arXiv  Detail & Related papers  (2023-11-02T17:57:03Z)
• Transformer-based nowcasting of radar composites from satellite images for severe weather [45.0983299269404]
  We present a Transformer-based model for nowcasting ground-based radar image sequences using satellite data up to two hours lead time. Trained on a dataset reflecting severe weather conditions, the model predicts radar fields occurring under different weather phenomena. The model can support precipitation nowcasting across large domains without an explicit need for radar towers.
  arXiv  Detail & Related papers  (2023-10-30T13:17:38Z)
• WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting [38.95768804529958]
  We augment the Open Motion dataset with large-scale, high-quality, diverse LiDAR data for the motion forecasting task. The new augmented dataset WOMD-LiDAR consists of over 100,000 scenes that each spans 20 seconds, consisting of well-synchronized and high quality LiDAR point clouds captured across a range of urban and suburban geographies. Experiments show that the LiDAR data brings improvement in the motion forecasting task.
  arXiv  Detail & Related papers  (2023-04-07T20:23:15Z)
• LiDAR Snowfall Simulation for Robust 3D Object Detection [116.10039516404743]
  We propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam. We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall.
  arXiv  Detail & Related papers  (2022-03-28T21:48:26Z)
• Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather [92.84066576636914]
  This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. We tackle this problem by simulating physically accurate fog into clear-weather scenes. We are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset.
  arXiv  Detail & Related papers  (2021-08-11T14:37:54Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.