AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

• URL: http://arxiv.org/abs/2603.16261v1
• Date: Tue, 17 Mar 2026 08:48:48 GMT
• Title: AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection
• Abstract summary: AW-MoE is a framework that integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches.<n>IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies.<n>We propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation.
• Score: 28.2299577324803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.

Related papers

• ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness [65.3988208273595]
  We introduce textbfObsDriveBench, a real-world multi-modal benchmark for adverse-weather autonomous driving.<n>Our benchmark is designed with three capability dimensions: textbfobservability awareness, textbfspatial reliability, and textbfrisk-aware decision-making.<n>Experiments reveal consistent performance degradation of existing vision-language models.
  arXiv  Detail & Related papers  (2026-07-26T08:13:46Z)
• CAMF-Det: Closure-Aware Multimodal Fusion for LiDAR-Camera 3D Object Detection on UAV Platforms [9.923404615359223]
  Multimodal 3D object detection based on LiDAR and cameras has demonstrated excellent performance in ground-vehicle scenarios.<n>We propose CAMF-Det, a closure-aware multimodal fusion framework for LiDAR-camera 3D object detection on UAV platforms.<n>We show that CAMF-Det achieves the best performance across all difficulty levels, with hard-level mAP$_mathrmBEV$ improvements of 9.43% and 4.88% over the best competing methods.
  arXiv  Detail & Related papers  (2026-06-08T07:40:17Z)
• LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving [0.055923945039144884]
  LRC-WeatherNet is a novel multi-sensor fusion framework that integrates LiDAR, RADAR, and camera data for real-time classification of weather conditions.<n>Our approach adapts to the varying reliability of each sensor under changing weather.<n>This work is the first to combine all three modalities for robust, real-time weather classification in autonomous driving.
  arXiv  Detail & Related papers  (2026-03-23T13:49:33Z)
• Diffusion-Based Restoration for Multi-Modal 3D Object Detection in Adverse Weather [15.57759675028067]
  DiffFusion is a novel framework designed to enhance robustness in challenging weather.<n>Our key insight is that diffusion models possess strong capabilities for denoising and generating data.<n> implementation of our DiffFusion will be released as open-source.
  arXiv  Detail & Related papers  (2025-12-15T09:03:46Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection [64.93675471780209]
  We present V2X-R, the first simulated V2X dataset incorporating LiDAR, camera, and 4D radar.<n>V2X-R contains 12,079 scenarios with 37,727 frames of LiDAR and 4D radar point clouds, 150,908 images, and 170,859 annotated 3D vehicle bounding boxes.<n>We propose a novel cooperative LiDAR-4D radar fusion pipeline for 3D object detection and implement it with various fusion strategies.
  arXiv  Detail & Related papers  (2024-11-13T07:41:47Z)
• Robust Modality-incomplete Anomaly Detection: A Modality-instructive Framework with Benchmark [69.02666229531322]
  We introduce a pioneering study that investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD)<n>We find that most existing MIAD methods perform poorly on the MIIAD Bench, leading to significant performance degradation.<n>We propose a novel two-stage Robust modAlity-aware fusing and Detecting framewoRk, abbreviated as RADAR.
  arXiv  Detail & Related papers  (2024-10-02T16:47:55Z)
• UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection [2.123197540438989]
  Many radar-vision fusion models treat radar as a sparse LiDAR, underutilizing radar-specific information. We propose the Radar Depth Lift-Splat-Shoot (RDL) module, which integrates radar-specific data into the depth prediction process. We also introduce a Unified Feature Fusion (UFF) approach that extracts BEV features across different modalities.
  arXiv  Detail & Related papers  (2024-09-23T06:57:27Z)
• MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detection [22.277210748714378]
  Existing methods mainly focus on performing 3D detection in ideal weather conditions, characterized by scenarios with clear and optimal visibility. We introduce MonoWAD, a novel weather-robust monocular 3D object detector with a weather-adaptive diffusion model. Experiments under various weather conditions demonstrate that MonoWAD achieves weather-robust monocular 3D object detection.
  arXiv  Detail & Related papers  (2024-07-23T12:58:49Z)
• WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models [51.61662672912017]
  We introduce WEDGE: a synthetic dataset generated with a vision-language generative model via prompting. WEDGE consists of 3360 images in 16 extreme weather conditions manually annotated with 16513 bounding boxes. We establish baseline performance for classification and detection with 53.87% test accuracy and 45.41 mAP.
  arXiv  Detail & Related papers  (2023-05-12T14:42:47Z)
• WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal [38.257012295118145]
  Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. We propose Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset.
  arXiv  Detail & Related papers  (2023-03-24T01:46:25Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.