Hyperspectral vs. RGB for Pedestrian Segmentation in Urban Driving Scenes: A Comparative Study

• URL: http://arxiv.org/abs/2508.11301v1
• Date: Fri, 15 Aug 2025 08:10:19 GMT
• Title: Hyperspectral vs. RGB for Pedestrian Segmentation in Urban Driving Scenes: A Comparative Study
• Abstract summary: This study investigates the potential of hyperspectral imaging (HSI) for enhanced pedestrian segmentation in urban driving scenarios.<n>Three semantic segmentation models were evaluated: U-Net, DeepLabV3+, and SegFormer.<n>CSNR-JMIM consistently outperformed RGB with an average improvements of 1.44% in Intersection over Union (IoU) and 2.18% in F1-score for pedestrian segmentation.
• Score: 3.6096086024478775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pedestrian segmentation in automotive perception systems faces critical safety challenges due to metamerism in RGB imaging, where pedestrians and backgrounds appear visually indistinguishable.. This study investigates the potential of hyperspectral imaging (HSI) for enhanced pedestrian segmentation in urban driving scenarios using the Hyperspectral City v2 (H-City) dataset. We compared standard RGB against two dimensionality-reduction approaches by converting 128-channel HSI data into three-channel representations: Principal Component Analysis (PCA) and optimal band selection using Contrast Signal-to-Noise Ratio with Joint Mutual Information Maximization (CSNR-JMIM). Three semantic segmentation models were evaluated: U-Net, DeepLabV3+, and SegFormer. CSNR-JMIM consistently outperformed RGB with an average improvements of 1.44% in Intersection over Union (IoU) and 2.18% in F1-score for pedestrian segmentation. Rider segmentation showed similar gains with 1.43% IoU and 2.25% F1-score improvements. These improved performance results from enhanced spectral discrimination of optimally selected HSI bands effectively reducing false positives. This study demonstrates robust pedestrian segmentation through optimal HSI band selection, showing significant potential for safety-critical automotive applications.

Related papers

• RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection [76.1829298714382]
  This paper focuses on the inconsistency in salient regions between RGB and thermal images.<n>We propose the Region-guided Selective Optimization Network for RGB-T Salient Object Detection.<n>We conduct extensive experiments on the RGB-T dataset, and the results demonstrate that the proposed RSONet achieves competitive performance against 27 state-of-the-art SOD methods.
  arXiv  Detail & Related papers  (2026-03-13T06:01:17Z)
• A Contrastive Fewshot RGBD Traversability Segmentation Framework for Indoor Robotic Navigation [0.7258309433541204]
  Pure vision-based models often fail to detect thin obstacles, such as chair legs, which can pose serious safety risks.<n>We propose a multi-modal segmentation framework that leverages RGB images and sparse 1D laser depth information.<n>Our method achieves up to 9% higher mIoU under both 1-shot and 5-shot settings.
  arXiv  Detail & Related papers  (2026-03-06T22:56:39Z)
• Enhancing Reinforcement Learning in 3D Environments through Semantic Segmentation: A Case Study in ViZDoom [0.0]
  Reinforcement learning (RL) in 3D environments with high-dimensional sensory input poses two major challenges.<n>This project proposes two novel input representations: SS-only and RGB+SS, both employing semantic segmentation on RGB colour images.<n> Experiments were conducted in deathmatches of ViZDoom, utilizing perfect segmentation results for controlled evaluation.
  arXiv  Detail & Related papers  (2025-11-12T14:28:46Z)
• Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing [55.291219073365546]
  Open-Vocabulary Remote Sensing Image (OVRSIS) is an emerging task that adapts Open-Vocabulary (OVS) to the remote sensing (RS) domain.<n>textbfRSKT-Seg is a novel open-vocabulary segmentation framework tailored for remote sensing.<n> RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation.
  arXiv  Detail & Related papers  (2025-09-15T15:24:49Z)
• CSNR and JMIM Based Spectral Band Selection for Reducing Metamerism in Urban Driving [3.2953155328671038]
  metamerism is a phenomenon where distinct materials appear similar in RGB imagery.<n>This work investigates hyperspectral imaging (HSI) to overcome this limitation by capturing unique material signatures beyond the visible spectrum.
  arXiv  Detail & Related papers  (2025-08-14T13:50:51Z)
• Multi-Scale Spectral Attention Module-based Hyperspectral Segmentation in Autonomous Driving Scenarios [3.437245452211197]
  This paper introduces a Multi-scale Spectral Attention Module (MSAM) that enhances spectral feature extraction.<n>By integrating MSAM into UNet's skip connections (UNet-SC), our proposed UNet-MSAM achieves significant improvements in semantic segmentation performance.
  arXiv  Detail & Related papers  (2025-06-23T14:24:20Z)
• Heuristical Comparison of Vision Transformers Against Convolutional Neural Networks for Semantic Segmentation on Remote Sensing Imagery [0.0]
  Vision Transformers (ViT) have brought a new wave of research in the field of computer vision.<n>This paper focuses on the comparison of three key factors of using (or not using) ViT for semantic segmentation of aerial images.<n>We show that the novel combined weighted loss function significantly boosts the CNN model's performance compared to transfer learning with ViT.
  arXiv  Detail & Related papers  (2024-11-14T00:18:04Z)
• HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios [3.7498611358320733]
  There is no standard benchmark available to measure progress on semantic segmentation in driving scenarios. In this paper, we provide the HyperSpectral Semantic benchmark (HS3-Bench) It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols.
  arXiv  Detail & Related papers  (2024-09-17T14:00:49Z)
• Superpixel Graph Contrastive Clustering with Semantic-Invariant Augmentations for Hyperspectral Images [64.72242126879503]
  Hyperspectral images (HSI) clustering is an important but challenging task. We first use 3-D and 2-D hybrid convolutional neural networks to extract the high-order spatial and spectral features of HSI. We then design a superpixel graph contrastive clustering model to learn discriminative superpixel representations.
  arXiv  Detail & Related papers  (2024-03-04T07:40:55Z)
• Sub-token ViT Embedding via Stochastic Resonance Transformers [51.12001699637727]
  Vision Transformer (ViT) architectures represent images as collections of high-dimensional vectorized tokens, each corresponding to a rectangular non-overlapping patch. We propose a training-free method inspired by "stochastic resonance" The resulting "Stochastic Resonance Transformer" (SRT) retains the rich semantic information of the original representation, but grounds it on a finer-scale spatial domain, partly mitigating the coarse effect of spatial tokenization.
  arXiv  Detail & Related papers  (2023-10-06T01:53:27Z)
• Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation [17.546960391700985]
  We propose the first test-time adaptation framework, dubbed Night-TTA, to address the problems for nighttime RGBT semantic segmentation. Our method achieves state-of-the-art (SoTA) performance with a 13.07% boost in mIoU.
  arXiv  Detail & Related papers  (2023-07-10T10:40:44Z)
• ARHNet: Adaptive Region Harmonization for Lesion-aware Augmentation to Improve Segmentation Performance [61.04246102067351]
  We propose a foreground harmonization framework (ARHNet) to tackle intensity disparities and make synthetic images look more realistic. We demonstrate the efficacy of our method in improving the segmentation performance using real and synthetic images.
  arXiv  Detail & Related papers  (2023-07-02T10:39:29Z)
• Generalizing Face Forgery Detection with High-frequency Features [63.33397573649408]
  Current CNN-based detectors tend to overfit to method-specific color textures and thus fail to generalize. We propose to utilize the high-frequency noises for face forgery detection. The first is the multi-scale high-frequency feature extraction module that extracts high-frequency noises at multiple scales. The second is the residual-guided spatial attention module that guides the low-level RGB feature extractor to concentrate more on forgery traces from a new perspective.
  arXiv  Detail & Related papers  (2021-03-23T08:19:21Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.