Shallow Network Based on Depthwise Over-Parameterized Convolution for Hyperspectral Image Classification

• URL: http://arxiv.org/abs/2112.00250v1
• Date: Wed, 1 Dec 2021 03:10:02 GMT
• Title: Shallow Network Based on Depthwise Over-Parameterized Convolution for Hyperspectral Image Classification
• Authors: Hongmin Gao, Member, IEEE, Zhonghao Chen, Student Member, IEEE, and Chenming Li
• Abstract summary: This letter proposes a shallow model for hyperspectral image classification (HSIC) using convolutional neural network (CNN) techniques. The proposed method outperforms other state-of-the-art methods in terms of classification accuracy and computational efficiency.
• Score: 0.7329200485567825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, convolutional neural network (CNN) techniques have gained popularity as a tool for hyperspectral image classification (HSIC). To improve the feature extraction efficiency of HSIC under the condition of limited samples, the current methods generally use deep models with plenty of layers. However, deep network models are prone to overfitting and gradient vanishing problems when samples are limited. In addition, the spatial resolution decreases severely with deeper depth, which is very detrimental to spatial edge feature extraction. Therefore, this letter proposes a shallow model for HSIC, which is called depthwise over-parameterized convolutional neural network (DOCNN). To ensure the effective extraction of the shallow model, the depthwise over-parameterized convolution (DO-Conv) kernel is introduced to extract the discriminative features. The depthwise over-parameterized Convolution kernel is composed of a standard convolution kernel and a depthwise convolution kernel, which can extract the spatial feature of the different channels individually and fuse the spatial features of the whole channels simultaneously. Moreover, to further reduce the loss of spatial edge features due to the convolution operation, a dense residual connection (DRC) structure is proposed to apply to the feature extraction part of the whole network. Experimental results obtained from three benchmark data sets show that the proposed method outperforms other state-of-the-art methods in terms of classification accuracy and computational efficiency.

Related papers

• Variational Inference for Infinitely Deep Neural Networks [0.4061135251278187]
  unbounded depth neural network (UDN) We introduce the unbounded depth neural network (UDN), an infinitely deep probabilistic model that adapts its complexity to the training data. We study the UDN on real and synthetic data.
  arXiv  Detail & Related papers  (2022-09-21T03:54:34Z)
• Dynamic Spatial Propagation Network for Depth Completion [6.3447233767041356]
  This paper introduces an efficient model that learns the affinity among neighboring pixels with an attention-based approach. In practice, our method requires less iteration to match the performance of other SPNs and yields better results overall.
  arXiv  Detail & Related papers  (2022-02-20T09:43:17Z)
• Optimization-Based Separations for Neural Networks [57.875347246373956]
  We show that gradient descent can efficiently learn ball indicator functions using a depth 2 neural network with two layers of sigmoidal activations. This is the first optimization-based separation result where the approximation benefits of the stronger architecture provably manifest in practice.
  arXiv  Detail & Related papers  (2021-12-04T18:07:47Z)
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover [87.28082715343896]
  We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications. We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem. The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
  arXiv  Detail & Related papers  (2021-10-20T06:15:45Z)
• Pyramidal Dense Attention Networks for Lightweight Image Super-Resolution [37.58180059860872]
  Deep convolutional neural network methods have achieved an excellent performance in image superresolution. We propose a pyramidal dense attention network (PDAN) for lightweight image super-resolution. Our method achieves superior performance in comparison with the state-of-the-art lightweight SR methods.
  arXiv  Detail & Related papers  (2021-06-13T13:49:41Z)
• Random Features for the Neural Tangent Kernel [57.132634274795066]
  We propose an efficient feature map construction of the Neural Tangent Kernel (NTK) of fully-connected ReLU network. We show that dimension of the resulting features is much smaller than other baseline feature map constructions to achieve comparable error bounds both in theory and practice.
  arXiv  Detail & Related papers  (2021-04-03T09:08:12Z)
• Compressive spectral image classification using 3D coded convolutional neural network [12.67293744927537]
  This paper develops a novel deep learning HIC approach based on measurements of coded-aperture snapshot spectral imagers (CASSI) A new kind of deep learning strategy, namely 3D coded convolutional neural network (3D-CCNN), is proposed to efficiently solve for the classification problem. The accuracy of classification is effectively improved by exploiting the synergy between the deep learning network and coded apertures.
  arXiv  Detail & Related papers  (2020-09-23T15:05:57Z)
• Tensor decomposition to Compress Convolutional Layers in Deep Learning [5.199454801210509]
  We propose to use CP-decomposition to approximately compress the convolutional layer (CPAC-Conv layer) in deep learning. The contributions of our work could be summarized into three aspects: (1) we adapt CP-decomposition to compress convolutional kernels and derive the expressions of both forward and backward propagations for our proposed CPAC-Conv layer; (2) compared with the original convolutional layer, the proposed CPAC-Conv layer can reduce the number of parameters without decaying prediction performance; and (3) the value of decomposed kernels indicates the significance of the corresponding feature map.
  arXiv  Detail & Related papers  (2020-05-28T02:35:48Z)
• Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks [107.77595511218429]
  In this paper, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks. We propose a feature distortion method (Disout) for addressing the aforementioned problem. The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated.
  arXiv  Detail & Related papers  (2020-02-23T13:59:13Z)
• MSE-Optimal Neural Network Initialization via Layer Fusion [68.72356718879428]
  Deep neural networks achieve state-of-the-art performance for a range of classification and inference tasks. The use of gradient combined nonvolutionity renders learning susceptible to novel problems. We propose fusing neighboring layers of deeper networks that are trained with random variables.
  arXiv  Detail & Related papers  (2020-01-28T18:25:15Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.