Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark

• URL: http://arxiv.org/abs/2604.27499v1
• Date: Thu, 30 Apr 2026 06:50:25 GMT
• Title: Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark
• Abstract summary: IRON is the first large-scale infrared dataset for temporal freespace detection under all-day conditions.<n>IRONet is a novel flow-free framework for temporal freespace detection.
• Score: 14.038383477789603
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Off-road nighttime autonomous driving suffers from unreliable visible-light perception, making infrared modality crucial for accurate freespace detection. However, progress remains limited due to the scarcity of annotated infrared off-road datasets and the inter-frame inconsistencies inherent to current single-frame methods. To address these gaps, we present the IRON dataset, which, to our knowledge, is the first large-scale infrared dataset for off-road temporal freespace detection under all-day conditions, with strong support for nighttime perception. The dataset comprises 24,314 densely annotated infrared images with synchronized RGB images in diverse scenes and different light conditions. Building upon this dataset, we propose IRONet, a novel flow-free framework for temporal freespace detection that addresses inter-frame inconsistencies by aggregating historical context via a memory-attention mechanism and a carefully designed mask decoder. On our IRON dataset, IRONet achieves state-of-the-art performance, reaching 82.93%(+1.19%) IoU and 90.66%(+0.71%) F1 score at real-time inference. Remarkably, IRONet also exhibits robust generalization to RGB modalities on ORFD and Rellis datasets. Overall, our work establishes a foundation for reliable all-day off-road autonomous driving and future research in infrared temporal perception. The code and IRON dataset are available at https://github.com/wsnbws/IRON.

Related papers

• Event-RGB Adaptive Tracking for Nighttime Highway Perception [7.388485566907394]
  Intelligent Transportation Systems predominantly rely on conventional RGB cameras for traffic perception and vehicle tracking.<n>Nighttime scenarios suffer from motion blur, insufficient exposure, and poor signal-to-noise ratios, which catastrophically impair the reliability of RGB-based sensing systems.<n>We propose a novel Joint Event-RGB Adaptive Tracking (JEAT) framework to address these limitations.
  arXiv  Detail & Related papers  (2026-07-13T14:59:46Z)
• Descriptor: LYNRED Mobility Dataset Multimodal Detection Subset (LYNRED-MDS) [21.053811864320302]
  We introduce the LYNRED-MDS: Multimodal Detection Subset, comprised of 4000 RGB-infrared image pairs captured in Grenoble, France.<n>Our dataset supports the development of more reliable and deployable vision systems for advanced driver-assistance systems.
  arXiv  Detail & Related papers  (2026-07-02T08:26:39Z)
• OctoSense: Self-Supervised Learning for Multimodal Robot Perception [50.20346079739425]
  We present OctoSense, an open-source sensor platform with stereo and event cameras, LiDAR, a thermal camera, an inertial measurement unit, and proprioception (CAN bus data from a car, and joint angles for a quadruped robot)<n>The eponymous OctoSense dataset contains 59 hours of time-synchronized driving data across different types of environments at different times of the day.<n>We demonstrate multi-modal self-supervised learning using such real-world data, where sensors have different representations, frequencies, latencies and noise.
  arXiv  Detail & Related papers  (2026-06-25T17:30:49Z)
• FEMOT: Multi-Object Tracking using Frame and Event Cameras [57.75021868843091]
  Bio-inspired event cameras offer high temporal resolution and high dynamic range, providing complementary cues under extreme scenarios.<n>FEMOT is a large-scale RGB-event multi-object tracking dataset that covers diverse real-world scenarios and 14 challenging attributes.<n>FEMOTR is a multimodal tracking framework that decouples RGB and event features and fuses them in the frequency domain.
  arXiv  Detail & Related papers  (2026-06-12T04:17:41Z)
• Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking [51.31378940976401]
  Existing RGB-Event tracking approaches fail to fully exploit the unique advantages of event cameras.<n>We propose a novel tracking framework that performs early fusion in the frequency domain, enabling effective aggregation of high-frequency information from the event modality.<n>Experiments on three widely used RGB-Event tracking benchmark datasets, including FE108, FELT, and COESOT, demonstrate the high performance and efficiency of our method.
  arXiv  Detail & Related papers  (2026-01-03T01:10:17Z)
• IrisNet: Infrared Image Status Awareness Meta Decoder for Infrared Small Targets Detection [92.56025546608699]
  IrisNet is a novel meta-learned framework that adapts detection strategies to the input infrared image status.<n>Our approach establishes a dynamic mapping between infrared image features and entire decoder parameters.<n> Experiments on NUDT-SIRST, NUAA-SIRST, and IRSTD-1K datasets demonstrate the superiority of our IrisNet.
  arXiv  Detail & Related papers  (2025-11-25T13:53:54Z)
• RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions [0.3141085922386211]
  Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR.<n>Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions.<n>We introduce the RGB and SWIR Multispectral Driving dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs.
  arXiv  Detail & Related papers  (2025-04-10T09:54:57Z)
• NIR-Assisted Image Denoising: A Selective Fusion Approach and A Real-World Benchmark Dataset [53.79524776100983]
  Leveraging near-infrared (NIR) images to assist visible RGB image denoising shows the potential to address this issue.<n>Existing works still struggle with taking advantage of NIR information effectively for real-world image denoising.<n>We propose an efficient Selective Fusion Module (SFM), which can be plug-and-played into the advanced denoising networks.
  arXiv  Detail & Related papers  (2024-04-12T14:54:26Z)
• Caltech Aerial RGB-Thermal Dataset in the Wild [14.699908177967181]
  We present the first publicly-available RGB-thermal dataset designed for aerial robotics operating in natural environments. Our dataset captures a variety of terrain across the United States, including rivers, lakes, coastlines, deserts, and forests. We provide semantic segmentation annotations for 10 classes commonly encountered in natural settings.
  arXiv  Detail & Related papers  (2024-03-13T23:31:04Z)
• Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera [8.673063170884591]
  EOLO is a novel object detection framework that achieves robust and efficient all-day detection by fusing both RGB and event modalities. Our EOLO framework is built based on a lightweight spiking neural network (SNN) to efficiently leverage the asynchronous property of events.
  arXiv  Detail & Related papers  (2023-09-17T15:14:01Z)
• Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning [59.19469551774703]
  Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
  arXiv  Detail & Related papers  (2020-03-05T05:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.