IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability

• URL: http://arxiv.org/abs/2412.03173v2
• Date: Sun, 02 Mar 2025 06:24:05 GMT
• Title: IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability
• Authors: Saksham Sharma, Akshit Raizada, Suresh Sundaram,
• Abstract summary: Traditional on-road autonomous methods struggle with dynamic terrains, leading to poor vehicle control in off-road conditions.<n>Recent deep-learning models have used perception sensors along with kinesthetic feedback for navigation on such terrains.<n>We propose a multi modal fusion network "IRisPath" capable of using Thermal and RGB images to provide robustness against dynamic weather and light conditions.
• Score: 2.21687743334279
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous off-road navigation is required for applications in agriculture, construction, search and rescue and defence. Traditional on-road autonomous methods struggle with dynamic terrains, leading to poor vehicle control in off-road conditions. Recent deep-learning models have used perception sensors along with kinesthetic feedback for navigation on such terrains. However, this approach has out-of-domain uncertainty. Factors like change in time of day and weather impacts the performance of the model. We propose a multi modal fusion network "IRisPath" capable of using Thermal and RGB images to provide robustness against dynamic weather and light conditions. To aid further works in this domain, we also open-source a day-night dataset with Thermal and RGB images along with pseudo-labels for traversability. In order to co-register for fusion model we also develop a novel method for targetless extrinsic calibration of Thermal, LiDAR and RGB cameras with translation accuracy of +/-1.7cm and rotation accuracy of +/-0.827degrees.

Related papers

• UNIV: Unified Foundation Model for Infrared and Visible Modalities [12.0490466425884]
  We propose a biologically inspired UNified foundation model for Infrared and Visible modalities (UNIV)<n>PCCL is an attention-guided distillation framework that mimics retinal horizontal cells' lateral inhibition.<n>Our dual-knowledge preservation mechanism emulates the retina's bipolar cell signal routing.
  arXiv  Detail & Related papers  (2025-09-19T06:07:53Z)
• SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather [33.428611256223256]
  We introduce a novel multi-sensor fusion approach tailored to adverse weather conditions.<n>In addition to fusing RGB and LiDAR sensors, our sensor fusion stack is also capable of learning from NIR camera and radar modalities.<n>Our method improves the reliability of multimodal sensor fusion in autonomous vehicles under challenging weather conditions.
  arXiv  Detail & Related papers  (2025-08-22T14:20:46Z)
• ROD: RGB-Only Fast and Efficient Off-road Freespace Detection [15.62982857392494]
  Off-road freespace detection is more challenging than on-road scenarios because of the blurred boundaries of traversable areas.<n>Previous state-of-the-art (SOTA) methods employ multi-modal fusion of RGB images and LiDAR data.<n>This paper presents a novel RGB-only approach for off-road freespace detection, named ROD, eliminating the reliance on LiDAR data.
  arXiv  Detail & Related papers  (2025-08-12T07:41:20Z)
• CARE: Enhancing Safety of Visual Navigation through Collision Avoidance via Repulsive Estimation [6.216878556851609]
  We propose CARE (Collision Avoidance via Repulsive Estimation) to improve the robustness of learning-based visual navigation methods.<n> CARE can be integrated seamlessly into any RGB-based navigation model that generates local robot trajectories.<n>We evaluate CARE by integrating it with state-of-the-art visual navigation models across diverse robot platforms.
  arXiv  Detail & Related papers  (2025-06-04T11:02:15Z)
• Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset [65.76480665062363]
  Human Activity Recognition primarily relied on traditional RGB cameras to achieve high-performance activity recognition. Challenges in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras. In this work, we rethink human activity recognition by combining the RGB and event cameras.
  arXiv  Detail & Related papers  (2025-04-08T09:14:24Z)
• Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection [67.02804741856512]
  We propose a novel Hierarchical Multi-Modal Enhancement Network (HMMEN) that integrates RGB and IR data for robust and accurate TL detection. Our method introduces two key components: (1) a Mutual Multi-Modal Enhanced Block (MMEB), which fuses and enhances hierarchical RGB and IR feature maps in a coarse-to-fine manner, and (2) a Feature Alignment Block (FAB) that corrects misalignments between decoder outputs and IR feature maps by leveraging deformable convolutions.
  arXiv  Detail & Related papers  (2025-01-25T06:21:06Z)
• TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection [2.8038082486377114]
  A novel Convolutional Neural Network model is proposed for the accurate estimation of the roadway surface. Our model is based on the use of a Twin-Decoder Neural Network (TEDNet) for independent camera and LiDAR feature extraction. Bird's Eye View projections of the camera and LiDAR data are used in this model to perform semantic segmentation on whether each pixel belongs to the road surface.
  arXiv  Detail & Related papers  (2024-05-14T08:45:34Z)
• DIDLM: A SLAM Dataset for Difficult Scenarios Featuring Infrared, Depth Cameras, LIDAR, 4D Radar, and Others under Adverse Weather, Low Light Conditions, and Rough Roads [20.600516423425688]
  We introduce a multi-sensor dataset covering challenging scenarios such as snowy weather, rainy weather, nighttime conditions, speed bumps, and rough terrains. The dataset includes rarely utilized sensors for extreme conditions, such as 4D millimeter-wave radar, infrared cameras, and depth cameras, alongside 3D LiDAR, RGB cameras, GPS, and IMU. It supports both autonomous driving and ground robot applications and provides reliable GPS/INS ground truth data, covering structured and semi-structured terrains.
  arXiv  Detail & Related papers  (2024-04-15T09:49:33Z)
• Caltech Aerial RGB-Thermal Dataset in the Wild [14.699908177967181]
  We present the first publicly-available RGB-thermal dataset designed for aerial robotics operating in natural environments. Our dataset captures a variety of terrain across the United States, including rivers, lakes, coastlines, deserts, and forests. We provide semantic segmentation annotations for 10 classes commonly encountered in natural settings.
  arXiv  Detail & Related papers  (2024-03-13T23:31:04Z)
• RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving [13.101416329887755]
  We present RoadRunner, a framework capable of predicting terrain traversability and an elevation map directly from camera and LiDAR sensor inputs. RoadRunner enables reliable autonomous navigation, by fusing sensory information, handling of uncertainty, and generation of contextually informed predictions. We demonstrate the effectiveness of RoadRunner in enabling safe and reliable off-road navigation at high speeds in multiple real-world driving scenarios through unstructured desert environments.
  arXiv  Detail & Related papers  (2024-02-29T16:47:54Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learning [64.92447072894055]
  Infrared (IR) cameras are robust under adverse illumination and lighting conditions. We propose an algorithm meta-learning framework to improve existing UDA methods. We produce a state-of-the-art thermal detector for the KAIST and DSIAC datasets.
  arXiv  Detail & Related papers  (2021-10-07T02:28:18Z)
• Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking [85.333260415532]
  We develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. When the appearance cue is unreliable, we take motion cues into account to make the tracker robust. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2020-07-04T08:11:33Z)
• Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning [59.19469551774703]
  Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
  arXiv  Detail & Related papers  (2020-03-05T05:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.