Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object
Detection
- URL: http://arxiv.org/abs/2312.01060v1
- Date: Sat, 2 Dec 2023 08:05:45 GMT
- Title: Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object
Detection
- Authors: Peifu Liu, Tingfa Xu, Huan Chen, Shiyun Zhou, Haolin Qin, Jianan Li
- Abstract summary: We propose a novel approach that fully leverages the spectral characteristics by extracting two distinct frequency components from the spectrum.
The Spectral Saliency approximates the region of salient objects, while the Spectral Edge captures edge information of salient objects.
To effectively utilize this dual-frequency information, we introduce a novel lightweight Spectrum-driven Mixed-frequency Network (SMN)
- Score: 14.621504062838731
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Hyperspectral salient object detection (HSOD) aims to detect spectrally
salient objects in hyperspectral images (HSIs). However, existing methods
inadequately utilize spectral information by either converting HSIs into
false-color images or converging neural networks with clustering. We propose a
novel approach that fully leverages the spectral characteristics by extracting
two distinct frequency components from the spectrum: low-frequency Spectral
Saliency and high-frequency Spectral Edge. The Spectral Saliency approximates
the region of salient objects, while the Spectral Edge captures edge
information of salient objects. These two complementary components, crucial for
HSOD, are derived by computing from the inter-layer spectral angular distance
of the Gaussian pyramid and the intra-neighborhood spectral angular gradients,
respectively. To effectively utilize this dual-frequency information, we
introduce a novel lightweight Spectrum-driven Mixed-frequency Network (SMN).
SMN incorporates two parameter-free plug-and-play operators, namely Spectral
Saliency Generator and Spectral Edge Operator, to extract the Spectral Saliency
and Spectral Edge components from the input HSI independently. Subsequently,
the Mixed-frequency Attention module, comprised of two frequency-dependent
heads, intelligently combines the embedded features of edge and saliency
information, resulting in a mixed-frequency feature representation.
Furthermore, a saliency-edge-aware decoder progressively scales up the
mixed-frequency feature while preserving rich detail and saliency information
for accurate salient object prediction. Extensive experiments conducted on the
HS-SOD benchmark and our custom dataset HSOD-BIT demonstrate that our SMN
outperforms state-of-the-art methods regarding HSOD performance. Code and
dataset will be available at https://github.com/laprf/SMN.
Related papers
- Spectral Graph Reasoning Network for Hyperspectral Image Classification [0.43512163406551996]
Convolutional neural networks (CNNs) have achieved remarkable performance in hyperspectral image (HSI) classification.
We propose a spectral graph reasoning network (SGR) learning framework comprising two crucial modules.
Experiments on two HSI datasets demonstrate that the proposed architecture can significantly improve the classification accuracy.
arXiv Detail & Related papers (2024-07-02T20:29:23Z) - DMSSN: Distilled Mixed Spectral-Spatial Network for Hyperspectral Salient Object Detection [12.823338405434244]
Hyperspectral salient object detection (HSOD) has exhibited remarkable promise across various applications.
Previous methods insufficiently harness the inherent distinctive attributes of hyperspectral images (HSIs) during the feature extraction process.
We propose Distilled Mixed Spectral-Spatial Network (DMSSN), comprising a Distilled Spectral-Spatial Transformer (MSST)
We have created a large-scale HSOD dataset, HSOD-BIT, to tackle the issue of data scarcity in this field.
arXiv Detail & Related papers (2024-03-31T14:04:57Z) - SSF-Net: Spatial-Spectral Fusion Network with Spectral Angle Awareness
for Hyperspectral Object Tracking [21.664141982246598]
Hyperspectral video (HSV) offers valuable spatial, spectral, and temporal information simultaneously.
Existing methods primarily focus on band regrouping and rely on RGB trackers for feature extraction.
In this paper, a spatial-spectral fusion network with spectral angle awareness (SST-Net) is proposed for hyperspectral (HS) object tracking.
arXiv Detail & Related papers (2024-03-09T09:37:13Z) - SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance
Field [70.15900280156262]
We propose an end-to-end Neural Radiance Field (NeRF)-based architecture for high-quality physically based rendering from a novel spectral perspective.
SpectralNeRF is superior to recent NeRF-based methods when synthesizing new views on synthetic and real datasets.
arXiv Detail & Related papers (2023-12-14T07:19:31Z) - ESSAformer: Efficient Transformer for Hyperspectral Image
Super-resolution [76.7408734079706]
Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation.
We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
arXiv Detail & Related papers (2023-07-26T07:45:14Z) - Boosting the Generalization Ability for Hyperspectral Image Classification using Spectral-spatial Axial Aggregation Transformer [14.594398447576188]
In the hyperspectral image classification (HSIC) task, the most commonly used model validation paradigm is partitioning the training-test dataset through pixel-wise random sampling.
In our experiments, we found that the high accuracy was reached because the training and test datasets share a lot of information.
We propose a spectral-spatial axial aggregation transformer model, namely SaaFormer, that preserves generalization across dataset partitions.
arXiv Detail & Related papers (2023-06-29T07:55:43Z) - Object Detection in Hyperspectral Image via Unified Spectral-Spatial
Feature Aggregation [55.9217962930169]
We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images.
S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
arXiv Detail & Related papers (2023-06-14T09:01:50Z) - Spectral Enhanced Rectangle Transformer for Hyperspectral Image
Denoising [64.11157141177208]
We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images.
For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain.
For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
arXiv Detail & Related papers (2023-04-03T09:42:13Z) - Deep Spectro-temporal Artifacts for Detecting Synthesized Speech [57.42110898920759]
This paper provides an overall assessment of track 1 (Low-quality Fake Audio Detection) and track 2 (Partially Fake Audio Detection)
In this paper, spectro-temporal artifacts were detected using raw temporal signals, spectral features, as well as deep embedding features.
We ranked 4th and 5th in track 1 and track 2, respectively.
arXiv Detail & Related papers (2022-10-11T08:31:30Z) - Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image
Reconstruction [127.20208645280438]
Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement.
Modeling the inter-spectra interactions is beneficial for HSI reconstruction.
Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
arXiv Detail & Related papers (2021-11-15T16:59:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.