Physically Plausible Spectral Reconstruction from RGB Images

• URL: http://arxiv.org/abs/2001.00558v1
• Date: Thu, 2 Jan 2020 18:46:26 GMT
• Title: Physically Plausible Spectral Reconstruction from RGB Images
• Abstract summary: We show how CNN learning can be extended so that physical plausibility is enforced. Our solution improves the state-of-the-art spectral recovery performance under varying exposure conditions.
• Score: 13.46266335798719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learning-based SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies. In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (the recovered spectra reintegrate to the input RGBs exactly).

Related papers

• BRIC-Net: Boundary-Reliable Illumination-Color Interaction for Remote Sensing Image Deshadowing [7.326663156595675]
  Shadows in remote sensing images obscure surface appearance and disrupt radiometric continuity.<n>We propose the Boundary-Reliable Illumination-Color Interaction Network (BRIC-Net), which decouples these failures at different representation levels.<n>BRIC-Net achieves 29.46dB full-image peak signal-to-noise ratio (PSNR) on AeroDS-Syn and 27.96dB on SRGTA.
  arXiv  Detail & Related papers  (2026-08-01T14:07:43Z)
• Toward Robust and 3D-Aware RGB-NIR Imaging in the Dark [68.49637363691325]
  This paper offers a new perspective for RGB-NIR low-light imaging by incorporating 3D-aware neural modeling.<n>A powerful model can be optimized to implicitly fuse extremely noisy RGB observations with NIR cues in 3D space, effectively recovering clean RGB images.
  arXiv  Detail & Related papers  (2026-07-31T17:59:18Z)
• R2H-Diff: Guided Spectral Diffusion Model for RGB-to-Hyperspectral Reconstruction [3.6195340076207447]
  We propose R2H-Diff, an efficient diffusion-based framework tailored for RGB-to-HSI reconstruction.<n>R2H-Diff formulates spectral recovery as a conditional iterative refinement process, enabling progressive reconstruction under RGB guidance.<n>Experiments on NTIRE2022, CAVE, and Harvard demonstrate that R2H-Diff achieves a favorable balance between reconstruction quality and computational efficiency.
  arXiv  Detail & Related papers  (2026-05-07T05:28:12Z)
• Frequency-Decomposed INR for NIR-Assisted Low-Light RGB Image Denoising [3.1407264198113936]
  This paper proposes a Near Infrared (NIR) aided low light image restoration method based on Frequency Decoupled Implicit Neural Representation (FDINR)<n>We decompose images into distinct frequency components via multi-scale wavelet transforms and construct a dual-branch implicit neural representation framework.<n> Experimental results demonstrate that FD-INR not only effectively restores image luminance consistency and structural details but also, benefitting from its implicit continuous representation, outperforms existing methods in arbitrary-resolution reconstruction tasks.
  arXiv  Detail & Related papers  (2026-04-18T03:12:51Z)
• RAW-Flow: Advancing RGB-to-RAW Image Reconstruction with Deterministic Latent Flow Matching [55.03149221192589]
  We introduce a novel framework named RAW-Flow to bridge the gap between RGB and RAW representations.<n>We also introduce a cross-scale context guidance module that injects hierarchical RGB features into the flow estimation process.<n> RAW-Flow outperforms state-of-the-art approaches both quantitatively and visually.
  arXiv  Detail & Related papers  (2026-01-28T08:27:38Z)
• RGB Pre-Training Enhanced Unobservable Feature Latent Diffusion Model for Spectral Reconstruction [16.54284634377436]
  We propose a two-stage pipeline consisting of spectral structure representation learning and spectral-spatial joint distribution learning.<n>In the first stage, a spectral unobservable feature autoencoder (SpeUAE) is trained to extract and compress the unobservable feature into a 3D manifold aligned with RGB space.<n>The ULDM is then acquired to model the distribution of the coded unobservable feature with guidance from the corresponding RGB images.
  arXiv  Detail & Related papers  (2025-07-17T10:07:32Z)
• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• Deep learning-based hyperspectral image reconstruction for quality assessment of agro-product [0.0]
  The goal of this study was to reconstruct hyperspectral images from RGB images through deep learning for agricultural applications. The algorithm accurately reconstructed the hyperspectral images from RGB images, with the resulting spectra closely matching the ground-truth. These findings highlight the potential of deep learning-based hyperspectral image reconstruction as a low-cost, efficient tool for various agricultural uses.
  arXiv  Detail & Related papers  (2024-05-20T18:15:20Z)
• Rethinking RGB Color Representation for Image Restoration Models [55.81013540537963]
  We augment the representation to hold structural information of local neighborhoods at each pixel. Substituting the underlying representation space for the per-pixel losses facilitates the training of image restoration models. Our space consistently improves overall metrics by reconstructing both color and local structures.
  arXiv  Detail & Related papers  (2024-02-05T06:38:39Z)
• Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation [22.07699685165064]
  Recent efforts in data-driven spectral reconstruction aim at extracting spectral information from RGB images captured by cost-effective RGB cameras. We evaluate both the practical limitations with respect to current datasets and overfitting, as well as fundamental limitations with respect to the nature of the information encoded in the RGB images. We propose to exploit the combination of metameric data augmentation and optical lens aberrations to improve the encoding of the metameric information into the RGB image.
  arXiv  Detail & Related papers  (2024-01-08T11:46:45Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• Learning to Recover Spectral Reflectance from RGB Images [20.260831758913902]
  spectral reflectance recovery (SRR) from RGB images is challenging and costly. Most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images. We propose a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information.
  arXiv  Detail & Related papers  (2023-04-04T23:27:02Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB [84.1657998542458]
  It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. This paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery.
  arXiv  Detail & Related papers  (2021-03-26T19:42:21Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.