CMTNet: Convolutional Meets Transformer Network for Hyperspectral Images Classification

• URL: http://arxiv.org/abs/2406.14080v2
• Date: Fri, 21 Jun 2024 01:58:18 GMT
• Title: CMTNet: Convolutional Meets Transformer Network for Hyperspectral Images Classification
• Authors: Faxu Guo, Quan Feng, Sen Yang, Wanxia Yang,
• Abstract summary: Current convolutional neural networks (CNNs) focus on local features in hyperspectral data. Transformer framework excels at extracting global features from hyperspectral imagery. This research introduces the Convolutional Meet Transformer Network (CMTNet)
• Score: 3.821081081400729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral remote sensing (HIS) enables the detailed capture of spectral information from the Earth's surface, facilitating precise classification and identification of surface crops due to its superior spectral diagnostic capabilities. However, current convolutional neural networks (CNNs) focus on local features in hyperspectral data, leading to suboptimal performance when classifying intricate crop types and addressing imbalanced sample distributions. In contrast, the Transformer framework excels at extracting global features from hyperspectral imagery. To leverage the strengths of both approaches, this research introduces the Convolutional Meet Transformer Network (CMTNet). This innovative model includes a spectral-spatial feature extraction module for shallow feature capture, a dual-branch structure combining CNN and Transformer branches for local and global feature extraction, and a multi-output constraint module that enhances classification accuracy through multi-output loss calculations and cross constraints across local, international, and joint features. Extensive experiments conducted on three datasets (WHU-Hi-LongKou, WHU-Hi-HanChuan, and WHU-Hi-HongHu) demonstrate that CTDBNet significantly outperforms other state-of-the-art networks in classification performance, validating its effectiveness in hyperspectral crop classification.

Related papers

• PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection [59.355022416218624]
  integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. We propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN) PVAFN uses a multi-pooling strategy to integrate both multi-scale and region-specific information effectively.
  arXiv  Detail & Related papers  (2024-08-26T19:43:01Z)
• Global-Local Progressive Integration Network for Blind Image Quality Assessment [6.095342999639137]
  Vision transformers (ViTs) excel in computer vision for modeling long-term dependencies, yet face two key challenges for image quality assessment (IQA) We propose a Global-Local progressive INTegration network for IQA, called GlintIQA, to address these issues through three key components.
  arXiv  Detail & Related papers  (2024-08-07T16:34:32Z)
• Double-Shot 3D Shape Measurement with a Dual-Branch Network [14.749887303860717]
  We propose a dual-branch Convolutional Neural Network (CNN)-Transformer network (PDCNet) to process different structured light (SL) modalities. Within PDCNet, a Transformer branch is used to capture global perception in the fringe images, while a CNN branch is designed to collect local details in the speckle images. We show that our method can reduce fringe order ambiguity while producing high-accuracy results on a self-made dataset.
  arXiv  Detail & Related papers  (2024-07-19T10:49:26Z)
• DCNN: Dual Cross-current Neural Networks Realized Using An Interactive Deep Learning Discriminator for Fine-grained Objects [48.65846477275723]
  This study proposes novel dual-current neural networks (DCNN) to improve the accuracy of fine-grained image classification. The main novel design features for constructing a weakly supervised learning backbone model DCNN include (a) extracting heterogeneous data, (b) keeping the feature map resolution unchanged, (c) expanding the receptive field, and (d) fusing global representations and local features.
  arXiv  Detail & Related papers  (2024-05-07T07:51:28Z)
• Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
  Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
  arXiv  Detail & Related papers  (2024-03-15T07:18:43Z)
• SENetV2: Aggregated dense layer for channelwise and global representations [0.0]
  We introduce a novel aggregated multilayer perceptron, a multi-branch dense layer, within the Squeeze residual module. This fusion enhances the network's ability to capture channel-wise patterns and have global knowledge. We conduct extensive experiments on benchmark datasets to validate the model and compare them with established architectures.
  arXiv  Detail & Related papers  (2023-11-17T14:10:57Z)
• DiffSpectralNet : Unveiling the Potential of Diffusion Models for Hyperspectral Image Classification [6.521187080027966]
  We propose a new network called DiffSpectralNet, which combines diffusion and transformer techniques. First, we use an unsupervised learning framework based on the diffusion model to extract both high-level and low-level spectral-spatial features. The diffusion method is capable of extracting diverse and meaningful spectral-spatial features, leading to improvement in HSI classification.
  arXiv  Detail & Related papers  (2023-10-29T15:26:37Z)
• DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification [109.09061514799413]
  Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions. We propose a tri-spectral image generation pipeline that transforms HSI into high-quality tri-spectral images. Our proposed method outperforms state-of-the-art methods for HSI classification.
  arXiv  Detail & Related papers  (2023-04-19T18:32:52Z)
• Conformer: Local Features Coupling Global Representations for Visual Recognition [72.9550481476101]
  We propose a hybrid network structure, termed Conformer, to take advantage of convolutional operations and self-attention mechanisms for enhanced representation learning. Experiments show that Conformer, under the comparable parameter complexity, outperforms the visual transformer (DeiT-B) by 2.3% on ImageNet.
  arXiv  Detail & Related papers  (2021-05-09T10:00:03Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.