On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving

• URL: http://arxiv.org/abs/2411.19274v1
• Date: Thu, 28 Nov 2024 17:10:50 GMT
• Title: On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving
• Authors: Jon Gutiérrez-Zaballa, Koldo Basterretxea, Javier Echanobe, M. Victoria Martínez, Unai Martínez-Corral, Óscar Mata Carballeira, Inés del Campo,
• Abstract summary: spectral reflectance of different objects in a driving scene beyond the visible spectrum can offer additional information. In this work we explore the use of snapshot, video-rate hyperspectral imaging (HSI) cameras in advanced driver assistance systems (ADAS) We analyze to what extent the spatial features codified by standard, tiny fully convolutional network (FCN) models can improve the performance of HSI segmentation systems.
• Score: 1.696186398088554
• License:
• Abstract: Most of current computer vision-based advanced driver assistance systems (ADAS) perform detection and tracking of objects quite successfully under regular conditions. However, under adverse weather and changing lighting conditions, and in complex situations with many overlapping objects, these systems are not completely reliable. The spectral reflectance of the different objects in a driving scene beyond the visible spectrum can offer additional information to increase the reliability of these systems, especially under challenging driving conditions. Furthermore, this information may be significant enough to develop vision systems that allow for a better understanding and interpretation of the whole driving scene. In this work we explore the use of snapshot, video-rate hyperspectral imaging (HSI) cameras in ADAS on the assumption that the near infrared (NIR) spectral reflectance of different materials can help to better segment the objects in real driving scenarios. To do this, we have used the HSI-Drive 1.1 dataset to perform various experiments on spectral classification algorithms. However, the information retrieval of hyperspectral recordings in natural outdoor scenarios is challenging, mainly because of deficient colour constancy and other inherent shortcomings of current snapshot HSI technology, which poses some limitations to the development of pure spectral classifiers. In consequence, in this work we analyze to what extent the spatial features codified by standard, tiny fully convolutional network (FCN) models can improve the performance of HSI segmentation systems for ADAS applications. The abstract above is truncated due to submission limits. For the full abstract, please refer to the published article.

Related papers

• Multi-Modality Driven LoRA for Adverse Condition Depth Estimation [61.525312117638116]
  We propose Multi-Modality Driven LoRA (MMD-LoRA) for Adverse Condition Depth Estimation. It consists of two core components: Prompt Driven Domain Alignment (PDDA) and Visual-Text Consistent Contrastive Learning (VTCCL) It achieves state-of-the-art performance on the nuScenes and Oxford RobotCar datasets.
  arXiv  Detail & Related papers  (2024-12-28T14:23:58Z)
• Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems [1.8874331450711404]
  We explore the use of hyperspectral imaging (HSI) in Advanced Driver Assistance Systems (ADAS) This paper describes some experimental results of the application of fully convolutional networks (FCN) to the image segmentation of HSI for ADAS applications. We use the HSI-Drive v1.1 dataset, which provides a set of labelled images recorded in real driving conditions with a small-size snapshot NIR-HSI camera.
  arXiv  Detail & Related papers  (2024-12-05T08:58:25Z)
• Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions [48.529493393948435]
  The visible-light camera has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems. The visual imaging quality inevitably suffers from several kinds of degradations under complex weather conditions. We develop a general-purpose multi-scene visibility enhancement method to restore degraded images captured under different weather conditions.
  arXiv  Detail & Related papers  (2024-09-02T23:46:27Z)
• SEVD: Synthetic Event-based Vision Dataset for Ego and Fixed Traffic Perception [22.114089372056238]
  We present SEVD, a first-of-its-kind multi-view ego, and fixed perception synthetic event-based dataset. SEVD spans urban, suburban, rural, and highway scenes featuring various classes of objects. We evaluate the dataset using state-of-the-art event-based (RED, RVT) and frame-based (YOLOv8) methods for traffic participant detection tasks.
  arXiv  Detail & Related papers  (2024-04-12T20:40:12Z)
• Learning to Find Missing Video Frames with Synthetic Data Augmentation: A General Framework and Application in Generating Thermal Images Using RGB Cameras [0.0]
  This paper addresses the issue of missing data due to sensor frame rate mismatches. We propose using conditional generative adversarial networks (cGANs) to create synthetic yet realistic thermal imagery.
  arXiv  Detail & Related papers  (2024-02-29T23:52:15Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• Multi-Attention Fusion Drowsy Driving Detection Model [1.2043574473965317]
  We introduce a novel approach called the Multi-Attention Fusion Drowsy Driving Detection Model (MAF) Our proposed model achieves an impressive driver drowsiness detection accuracy of 96.8%.
  arXiv  Detail & Related papers  (2023-12-28T14:53:32Z)
• A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design [77.34726150561087]
  We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles. Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
  arXiv  Detail & Related papers  (2023-03-08T01:29:55Z)
• Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics [77.34726150561087]
  This work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. We give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.
  arXiv  Detail & Related papers  (2023-03-08T00:48:32Z)
• Attention Guided Network for Salient Object Detection in Optical Remote Sensing Images [16.933770557853077]
  salient object detection in optical remote sensing images (RSI-SOD) is a very difficult task. We propose a novel Attention Guided Network (AGNet) for SOD in optical RSIs, including position enhancement stage and detail refinement stage. AGNet achieves competitive performance compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-07-05T01:01:03Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.