RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions

• URL: http://arxiv.org/abs/2504.07603v1
• Date: Thu, 10 Apr 2025 09:54:57 GMT
• Title: RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions
• Authors: Youngwan Jin, Michal Kovac, Yagiz Nalcakan, Hyeongjin Ju, Hanbin Song, Sanghyeop Yeo, Shiho Kim,
• Abstract summary: Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR.<n>Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions.<n>We introduce the RGB and SWIR Multispectral Driving dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs.
• Score: 0.3141085922386211
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions like fog, rain, snow, glare, and high contrast. Although other spectral bands like near-infrared (NIR) and long-wave infrared (LWIR) can enhance vision perception in such situations, they have limitations and lack large-scale datasets and benchmarks. Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR. However, no publicly available large-scale datasets currently incorporate SWIR data for autonomous driving. To address this gap, we introduce the RGB and SWIR Multispectral Driving (RASMD) dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs collected across diverse locations, lighting, and weather conditions. In addition, we provide a subset for RGB-SWIR translation and object detection annotations for a subset of challenging traffic scenarios to demonstrate the utility of SWIR imaging through experiments on both object detection and RGB-to-SWIR image translation. Our experiments show that combining RGB and SWIR data in an ensemble framework significantly improves detection accuracy compared to RGB-only approaches, particularly in conditions where visible-spectrum sensors struggle. We anticipate that the RASMD dataset will advance research in multispectral imaging for autonomous driving and robust perception systems.

Related papers

• SAGA: Semantic-Aware Gray color Augmentation for Visible-to-Thermal Domain Adaptation across Multi-View Drone and Ground-Based Vision Systems [1.891522135443594]
  Domain-adaptive thermal object detection plays a key role in facilitating visible (RGB)-to-thermal (IR) adaptation. In inherent limitations of IR images, such as the lack of color and texture cues, pose challenges for RGB-trained models. We propose Semantic-Aware Gray color Augmentation (SAGA), a novel strategy for mitigating color bias and bridging the domain gap.
  arXiv  Detail & Related papers  (2025-04-22T09:22:11Z)
• Multi-modal Multi-platform Person Re-Identification: Benchmark and Method [58.59888754340054]
  MP-ReID is a novel dataset designed specifically for multi-modality and multi-platform ReID.<n>This benchmark compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging.<n>We introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios.
  arXiv  Detail & Related papers  (2025-03-21T12:27:49Z)
• Multi-Domain Biometric Recognition using Body Embeddings [51.36007967653781]
  We show that body embeddings perform better than face embeddings in medium-wave infrared (MWIR) and long-wave infrared (LWIR) domains.<n>We leverage a vision transformer architecture to establish benchmark results on the IJB-MDF dataset.<n>We also show that finetuning a body model, pretrained exclusively on VIS data, with a simple combination of cross-entropy and triplet losses achieves state-of-the-art mAP scores.
  arXiv  Detail & Related papers  (2025-03-13T22:38:18Z)
• Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection [67.02804741856512]
  We propose a novel Hierarchical Multi-Modal Enhancement Network (HMMEN) that integrates RGB and IR data for robust and accurate TL detection.<n>Our method introduces two key components: (1) a Mutual Multi-Modal Enhanced Block (MMEB), which fuses and enhances hierarchical RGB and IR feature maps in a coarse-to-fine manner, and (2) a Feature Alignment Block (FAB) that corrects misalignments between decoder outputs and IR feature maps by leveraging deformable convolutions.
  arXiv  Detail & Related papers  (2025-01-25T06:21:06Z)
• CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain [7.007302908953179]
  Infrared (IR) imaging offers advantages in several fields due to its unique ability of capturing content in extreme light conditions. As an alternative, visible light can be used to synthesize IR images but this causes a loss of fidelity in image details and introduces inconsistencies due to lack of contextual awareness of the scene.
  arXiv  Detail & Related papers  (2024-11-25T12:23:14Z)
• Visibility Constrained Wide-band Illumination Spectrum Design for Seeing-in-the-Dark [38.11468156313255]
  Seeing-in-the-dark is one of the most important and challenging computer vision tasks. In this paper, we try to robustify NIR2RGB translation by designing the optimal spectrum of auxiliary illumination in the wide-band VIS-NIR range.
  arXiv  Detail & Related papers  (2023-03-21T07:27:37Z)
• Self-Aligning Depth-regularized Radiance Fields for Asynchronous RGB-D Sequences [12.799443250845224]
  We propose a novel time-pose function, which is an implicit network that maps timestamps to $rm SE(3)$ elements. Our algorithm consists of three steps: (1) time-pose function fitting, (2) radiance field bootstrapping, (3) joint pose error compensation and radiance field refinement. We also show qualitatively improved results on a real-world asynchronous RGB-D sequence captured by drone.
  arXiv  Detail & Related papers  (2022-11-14T15:37:27Z)
• Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection [10.460296317901662]
  We find detection in aerial RGB-IR images suffers from cross-modal weakly misalignment problems. We propose a Translation-Scale-Rotation Alignment (TSRA) module to address the problem. A two-stream feature alignment detector (TSFADet) based on the TSRA module is constructed for RGB-IR object detection in aerial images.
  arXiv  Detail & Related papers  (2022-09-28T03:06:18Z)
• Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images [89.81919625224103]
  Training deep models for RGB-D salient object detection (SOD) often requires a large number of labeled RGB-D images. We present a Dual-Semi RGB-D Salient Object Detection Network (DS-Net) to leverage unlabeled RGB images for boosting RGB-D saliency detection.
  arXiv  Detail & Related papers  (2022-01-01T03:02:27Z)
• Synergistic saliency and depth prediction for RGB-D saliency detection [76.27406945671379]
  Existing RGB-D saliency datasets are small, which may lead to overfitting and limited generalization for diverse scenarios. We propose a semi-supervised system for RGB-D saliency detection that can be trained on smaller RGB-D saliency datasets without saliency ground truth.
  arXiv  Detail & Related papers  (2020-07-03T14:24:41Z)
• Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning [59.19469551774703]
  Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
  arXiv  Detail & Related papers  (2020-03-05T05:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.