Self-Supervised Hyperspectral Inpainting with the Optimisation inspired Deep Neural Network Prior

• URL: http://arxiv.org/abs/2306.07308v3
• Date: Fri, 21 Jul 2023 11:52:28 GMT
• Title: Self-Supervised Hyperspectral Inpainting with the Optimisation inspired Deep Neural Network Prior
• Authors: Shuo Li and Mehrdad Yaghoobi
• Abstract summary: We introduce a novel HSI missing pixel prediction algorithm, called Low Rank Spars Constraint Plug-and-Play (LRS-DIP) LRS-DIP is able to predict missing pixels and even when spectral bands are missing.
• Score: 7.777433987363129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperspectral Image (HSI)s cover hundreds or thousands of narrow spectral bands, conveying a wealth of spatial and spectral information. However, due to the instrumental errors and the atmospheric changes, the HSI obtained in practice are often contaminated by noise and dead pixels(lines), resulting in missing information that may severely compromise the subsequent applications. We introduce here a novel HSI missing pixel prediction algorithm, called Low Rank and Sparsity Constraint Plug-and-Play (LRS-PnP). It is shown that LRS-PnP is able to predict missing pixels and bands even when all spectral bands of the image are missing. The proposed LRS-PnP algorithm is further extended to a self-supervised model by combining the LRS-PnP with the Deep Image Prior (DIP), called LRS-PnP-DIP. In a series of experiments with real data, It is shown that the LRS-PnP-DIP either achieves state-of-the-art inpainting performance compared to other learning-based methods, or outperforms them.

Related papers

• Self-supervised Deep Hyperspectral Inpainting with the Plug and Play and Deep Image Prior Models [6.8557067473167415]
  Hyperspectral images are typically composed of hundreds of narrow and contiguous spectral bands, each containing information regarding the composition of the imaged scene. These images can be affected by various sources of noise, distortions data, or material loss, which can significantly degrade their quality and usefulness. This paper introduces a converge a guaranteed algorithm, LRS-nt-DIP, which successfully addresses the instability issue of DHP.
  arXiv  Detail & Related papers  (2025-01-14T15:18:28Z)
• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Learning Degradation-Independent Representations for Camera ISP Pipelines [14.195578257521934]
  We propose a novel approach to learn degradation-independent representations (DiR) through the refinement of a self-supervised learned baseline representation. The proposed DiR learning technique has remarkable domain generalization capability and it outperforms state-of-the-art methods across various downstream tasks.
  arXiv  Detail & Related papers  (2023-07-03T05:38:28Z)
• Self-supervised Deep Hyperspectral Inpainting with the Sparsity and Low-Rank Considerations [7.777433987363129]
  Hyperspectral images can be affected by various sources of noise, distortions, or data losses. We introduce two novel self-supervised Images (HSI) in Hyperpainting algorithms. We conduct the stability analysis under some mild assumptions which guarantees the algorithm to converge.
  arXiv  Detail & Related papers  (2023-06-13T20:49:02Z)
• Degradation-Noise-Aware Deep Unfolding Transformer for Hyperspectral Image Denoising [9.119226249676501]
  Hyperspectral images (HSIs) are often quite noisy because of narrow band spectral filtering. To reduce the noise in HSI data cubes, both model-driven and learning-based denoising algorithms have been proposed. This paper proposes a Degradation-Noise-Aware Unfolding Network (DNA-Net) that addresses these issues.
  arXiv  Detail & Related papers  (2023-05-06T13:28:20Z)
• Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering [84.37776381343662]
  Mip-NeRF proposes a multiscale representation as a conical frustum to encode scale information. We propose mip voxel grids (Mip-VoG), an explicit multiscale representation for real-time anti-aliasing rendering. Our approach is the first to offer multiscale training and real-time anti-aliasing rendering simultaneously.
  arXiv  Detail & Related papers  (2023-04-20T04:05:22Z)
• Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning [9.391271552098878]
  We propose a practical yet lightweight architectural unit, Pyramid Pixel Context Adaption (PPCA) module. PPCA exploits multi-scale pixel context information to recalibrate pixel position in a pixel-independent manner. We show that PPCANet outperforms state-of-the-art attention-based networks and recent deep neural networks.
  arXiv  Detail & Related papers  (2023-03-03T13:36:55Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction [64.10636296274168]
  Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets) We propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers.
  arXiv  Detail & Related papers  (2021-07-06T14:11:03Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.