LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classification

• URL: http://arxiv.org/abs/2404.03883v2
• Date: Mon, 15 Apr 2024 06:34:52 GMT
• Title: LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classification
• Authors: Judy X Yang, Jun Zhou, Jing Wang, Hui Tian, Alan Wee-Chung Liew,
• Abstract summary: This paper introduces a cross-attention mechanism for the selection of HSI bands guided by LiDAR data. The method ensures that the selected HSI bands drastically reduce redundancy and computational requirements. Experiments have been undertaken on three paired HSI and LiDAR data sets: Houston 2013, Trento and MUUFL.
• Score: 16.742768644585684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The fusion of hyperspectral and LiDAR data has been an active research topic. Existing fusion methods have ignored the high-dimensionality and redundancy challenges in hyperspectral images, despite that band selection methods have been intensively studied for hyperspectral image (HSI) processing. This paper addresses this significant gap by introducing a cross-attention mechanism from the transformer architecture for the selection of HSI bands guided by LiDAR data. LiDAR provides high-resolution vertical structural information, which can be useful in distinguishing different types of land cover that may have similar spectral signatures but different structural profiles. In our approach, the LiDAR data are used as the "query" to search and identify the "key" from the HSI to choose the most pertinent bands for LiDAR. This method ensures that the selected HSI bands drastically reduce redundancy and computational requirements while working optimally with the LiDAR data. Extensive experiments have been undertaken on three paired HSI and LiDAR data sets: Houston 2013, Trento and MUUFL. The results highlight the superiority of the cross-attention mechanism, underlining the enhanced classification accuracy of the identified HSI bands when fused with the LiDAR features. The results also show that the use of fewer bands combined with LiDAR surpasses the performance of state-of-the-art fusion models.

Related papers

• Test-time Training for Hyperspectral Image Super-resolution [95.38382633281398]
  Hyperspectral image (HSI) super-resolution (SR) is still lagging behind the research of RGB image SR. In this work, we propose a new test-time training method to tackle this problem. Specifically, a novel self-training framework is developed, where more accurate pseudo-labels and more accurate LR-HR relationships are generated.
  arXiv  Detail & Related papers  (2024-09-13T09:30:19Z)
• Unsupervised Band Selection Using Fused HSI and LiDAR Attention Integrating With Autoencoder [16.742768644585684]
  Band selection in hyperspectral imaging (HSI) is critical for optimising data processing and enhancing analytical accuracy. Traditional approaches have predominantly concentrated on analysing spectral and pixel characteristics within individual bands independently. This paper introduces a novel unsupervised band selection framework that incorporates attention mechanisms and an Autoencoder for reconstruction-based band selection.
  arXiv  Detail & Related papers  (2024-04-08T07:47:28Z)
• Bridging Sensor Gaps via Attention Gated Tuning for Hyperspectral Image Classification [9.82907639745345]
  HSI classification methods require high-quality labeled HSIs, which are often costly to obtain. We propose a novel Attention-Gated Tuning (AGT) strategy and a triplet-structured transformer model, Tri-Former, to address this issue.
  arXiv  Detail & Related papers  (2023-09-22T13:39:24Z)
• Multimodal Hyperspectral Image Classification via Interconnected Fusion [12.41850641917384]
  An Interconnected Fusion (IF) framework is proposed to explore the relationships across HSI and LiDAR modalities comprehensively. Experiments have been conducted on three widely used datasets: Trento, MUUFL, and Houston.
  arXiv  Detail & Related papers  (2023-04-02T09:46:13Z)
• Benchmarking the Robustness of LiDAR Semantic Segmentation Models [78.6597530416523]
  In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. We propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. We design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications.
  arXiv  Detail & Related papers  (2023-01-03T06:47:31Z)
• Hyperspectral Images Classification and Dimensionality Reduction using spectral interaction and SVM classifier [0.0]
  The high dimensionality of the hyperspectral images (HSI) is one of the main challenges for the analysis of the collected data. The existence of noisy, redundant and irrelevant bands increases the computational complexity. We propose a novel filter approach based on the spectral interaction measure and the support vector machines for dimensionality reduction.
  arXiv  Detail & Related papers  (2022-10-27T15:37:57Z)
• Supervised classification methods applied to airborne hyperspectral images: Comparative study using mutual information [0.0]
  This paper investigates the performance of four supervised learning algorithms, namely, Support Vector Machines SVM, Random Forest RF, K-Nearest Neighbors KNN and Linear Discriminant Analysis LDA. The experiments have been performed on three real hyperspectral datasets taken from the NASA's Airborne Visible/Infrared Imaging Spectrometer Sensor AVIRIS and the Reflective Optics System Imaging Spectrometer ROSIS sensors.
  arXiv  Detail & Related papers  (2022-10-27T13:39:08Z)
• Boosting 3D Object Detection by Simulating Multimodality on Point Clouds [51.87740119160152]
  This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector. The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference. Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors.
  arXiv  Detail & Related papers  (2022-06-30T01:44:30Z)
• LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection [96.63947479020631]
  In many real-world applications, the LiDAR points used by mass-produced robots and vehicles usually have fewer beams than that in large-scale public datasets. We propose the LiDAR Distillation to bridge the domain gap induced by different LiDAR beams for 3D object detection.
  arXiv  Detail & Related papers  (2022-03-28T17:59:02Z)
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets [99.92564298432387]
  This work studies Hyperspectral image (HSI) super-resolution (SR) HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
  arXiv  Detail & Related papers  (2021-01-19T12:19:53Z)
• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution [79.97180849505294]
  We propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet, to enhance the spatial resolution of HSI. Experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models.
  arXiv  Detail & Related papers  (2020-07-10T08:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.