Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification

• URL: http://arxiv.org/abs/2503.06945v1
• Date: Mon, 10 Mar 2025 05:50:13 GMT
• Title: Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification
• Authors: Junyan Lin, Feng Gap, Lin Qi, Junyu Dong, Qian Du, Xinbo Gao,
• Abstract summary: Hyperspectral image (HSI) and LiDAR data joint classification is a challenging task.<n>We propose a novel Dynamic Cross-Modal Feature Interaction Network (DCMNet)<n>Our approach introduces three feature interaction blocks: Bilinear Spatial Attention Block (BSAB), Bilinear Channel Attention Block (BCAB), and Integration Convolutional Block (ICB)
• Score: 66.59320112015556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral image (HSI) and LiDAR data joint classification is a challenging task. Existing multi-source remote sensing data classification methods often rely on human-designed frameworks for feature extraction, which heavily depend on expert knowledge. To address these limitations, we propose a novel Dynamic Cross-Modal Feature Interaction Network (DCMNet), the first framework leveraging a dynamic routing mechanism for HSI and LiDAR classification. Specifically, our approach introduces three feature interaction blocks: Bilinear Spatial Attention Block (BSAB), Bilinear Channel Attention Block (BCAB), and Integration Convolutional Block (ICB). These blocks are designed to effectively enhance spatial, spectral, and discriminative feature interactions. A multi-layer routing space with routing gates is designed to determine optimal computational paths, enabling data-dependent feature fusion. Additionally, bilinear attention mechanisms are employed to enhance feature interactions in spatial and channel representations. Extensive experiments on three public HSI and LiDAR datasets demonstrate the superiority of DCMNet over state-of-the-art methods. Our code will be available at https://github.com/oucailab/DCMNet.

Related papers

• Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data [14.104497777255137]
  We introduce Low-rank Efficient Spatial-Spectral Vision Transformer with three key innovations. We pretrain LESS ViT using a Hyperspectral Masked Autoencoder framework with integrated positional and channel masking strategies. Experimental results demonstrate that our proposed method achieves competitive performance against state-of-the-art multi-modal geospatial foundation models.
  arXiv  Detail & Related papers  (2025-03-17T05:42:19Z)
• HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks [7.06787067270941]
  The integration of hyperspectral imaging (HSI) and LiDAR data within new linear feature spaces offers a promising solution to the challenges posed by the high-dimensionality and redundancy inherent in HSIs.<n>This study introduces a dual linear fused space framework that capitalizes on bidirectional reversed convolutional neural network (CNN) pathways, coupled with a specialized spatial analysis block.<n>The proposed method not only enhances data processing and classification accuracy, but also mitigates the computational burden typically associated with advanced models such as Transformers.
  arXiv  Detail & Related papers  (2024-11-30T01:08:08Z)
• Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving [69.20604395205248]
  We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving. We introduce an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector. We conduct extensive experiments on the KITTI, runtime, and nuScenes datasets, and the results demonstrate the state-of-the-art performance of Shift-SSD.
  arXiv  Detail & Related papers  (2024-03-10T10:36:32Z)
• Dual Attention U-Net with Feature Infusion: Pushing the Boundaries of Multiclass Defect Segmentation [1.487252325779766]
  The proposed architecture, Dual Attentive U-Net with Feature Infusion (DAU-FI Net), addresses challenges in semantic segmentation. DAU-FI Net integrates multiscale spatial-channel attention mechanisms and feature injection to enhance precision in object localization. Comprehensive experiments on a challenging sewer pipe and culvert defect dataset and a benchmark dataset validate DAU-FI Net's capabilities.
  arXiv  Detail & Related papers  (2023-12-21T17:23:49Z)
• Multi-Dimensional Refinement Graph Convolutional Network with Robust Decouple Loss for Fine-Grained Skeleton-Based Action Recognition [19.031036881780107]
  We propose a flexible attention block called Channel-Variable Spatial-Temporal Attention (CVSTA) to enhance the discriminative power of spatial-temporal joints. Based on CVSTA, we construct a Multi-Dimensional Refinement Graph Convolutional Network (MDR-GCN), which can improve the discrimination among channel-, joint- and frame-level features. Furthermore, we propose a Robust Decouple Loss (RDL), which significantly boosts the effect of the CVSTA and reduces the impact of noise.
  arXiv  Detail & Related papers  (2023-06-27T09:23:36Z)
• A3CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification [24.006660419933727]
  We propose a new approach to exploit the complement of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. We develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel A3CLNN) for feature extraction and classification.
  arXiv  Detail & Related papers  (2022-04-09T12:43:32Z)
• LC3Net: Ladder context correlation complementary network for salient object detection [0.32116198597240836]
  We propose a novel ladder context correlation complementary network (LC3Net) FCB is a filterable convolution block to assist the automatic collection of information on the diversity of initial features. DCM is a dense cross module to facilitate the intimate aggregation of different levels of features. BCD is a bidirectional compression decoder to help the progressive shrinkage of multi-scale features.
  arXiv  Detail & Related papers  (2021-10-21T03:12:32Z)
• Specificity-preserving RGB-D Saliency Detection [103.3722116992476]
  We propose a specificity-preserving network (SP-Net) for RGB-D saliency detection. Two modality-specific networks and a shared learning network are adopted to generate individual and shared saliency maps. Experiments on six benchmark datasets demonstrate that our SP-Net outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-18T14:14:22Z)
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  LiDAR-based panoptic segmentation aims to parse both objects and scenes in a unified manner. We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-24T08:44:46Z)
• RGB-D Salient Object Detection with Cross-Modality Modulation and Selection [126.4462739820643]
  We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD) The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features.
  arXiv  Detail & Related papers  (2020-07-14T14:22:50Z)
• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution [79.97180849505294]
  We propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet, to enhance the spatial resolution of HSI. Experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models.
  arXiv  Detail & Related papers  (2020-07-10T08:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.