HyperColorization: Propagating spatially sparse noisy spectral clues for reconstructing hyperspectral images

• URL: http://arxiv.org/abs/2403.11935v1
• Date: Mon, 18 Mar 2024 16:33:43 GMT
• Title: HyperColorization: Propagating spatially sparse noisy spectral clues for reconstructing hyperspectral images
• Authors: M. Kerem Aydin, Qi Guo, Emma Alexander,
• Abstract summary: We present a colorization to reconstruct hyperspectral images from a grayscale guide image and spatially sparse spectral clues. Our algorithm generalizes to varying spectral dimensions for hyperspectral images, and show that colorizing in a low-rank space reduces compute time and the impact of shot noise.
• Score: 2.7214317850962106
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Hyperspectral cameras face challenging spatial-spectral resolution trade-offs and are more affected by shot noise than RGB photos taken over the same total exposure time. Here, we present a colorization algorithm to reconstruct hyperspectral images from a grayscale guide image and spatially sparse spectral clues. We demonstrate that our algorithm generalizes to varying spectral dimensions for hyperspectral images, and show that colorizing in a low-rank space reduces compute time and the impact of shot noise. To enhance robustness, we incorporate guided sampling, edge-aware filtering, and dimensionality estimation techniques. Our method surpasses previous algorithms in various performance metrics, including SSIM, PSNR, GFC, and EMD, which we analyze as metrics for characterizing hyperspectral image quality. Collectively, these findings provide a promising avenue for overcoming the time-space-wavelength resolution trade-off by reconstructing a dense hyperspectral image from samples obtained by whisk or push broom scanners, as well as hybrid spatial-spectral computational imaging systems.

Related papers

• SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution [73.46167948298041]
  We propose a neural implicit model that represents an image as a function of both continuous pixel coordinates in the spatial domain and continuous wavelengths in the spectral domain. We show that SSIF generalizes well to both unseen spatial resolutions and spectral resolutions. It can generate high-resolution images that improve the performance of downstream tasks by 1.7%-7%.
  arXiv  Detail & Related papers  (2023-09-30T15:23:30Z)
• Aperture Diffraction for Compact Snapshot Spectral Imaging [27.321750056840706]
  We demonstrate a compact, cost-effective snapshot spectral imaging system named Aperture Diffraction Imaging Spectrometer (ADIS) A new optical design that each point in the object space is multiplexed to discrete encoding locations on the mosaic filter sensor is introduced. The Cascade Shift-Shuffle Spectral Transformer (CSST) with strong perception of the diffraction degeneration is designed to solve a sparsity-constrained inverse problem.
  arXiv  Detail & Related papers  (2023-09-27T16:48:46Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• Blind deblurring of hyperspectral document images [0.0]
  Multispectral (MS) and hyperspectral (HS) images contain much richer spectral information than RGB images. We propose novel blind HS image deblurring methods tailored to document images.
  arXiv  Detail & Related papers  (2023-03-09T09:31:13Z)
• Unsupervised Spatial-spectral Hyperspectral Image Reconstruction and Clustering with Diffusion Geometry [6.279792995020646]
  This work introduces the Spatial-Spectral Image Reconstruction and Clustering with Diffusion Geometry (DSIRC) algorithm for partitioning highly mixed hyperspectral images. DSIRC locates spectrally correlated pixels within a data-adaptive spatial neighborhood and reconstructs that pixel's spectral signature using those of its neighbors. Results indicate that incorporating spatial information through image reconstruction substantially improves the performance of pixel-wise clustering.
  arXiv  Detail & Related papers  (2022-04-28T13:42:12Z)
• Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution [82.59267937569213]
  High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions. In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution. We present a spectral splitting and aggregation network (SSANet) for HFSR with limited training samples.
  arXiv  Detail & Related papers  (2021-08-31T02:13:00Z)
• SASSI -- Super-Pixelated Adaptive Spatio-Spectral Imaging [17.2152544145501]
  We introduce a novel video-rate hyperspectral imager with high spatial, and temporal resolutions. A scene-adaptive spatial sampling of an hyperspectral scene, guided by its super-pixel segmented image, is capable of obtaining high-quality reconstructions. We validate the proposed technique with extensive simulations as well as a lab prototype that measures hyperspectral video at a spatial resolution of $600 times 900$ pixels.
  arXiv  Detail & Related papers  (2020-12-28T21:34:18Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.