Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB

• URL: http://arxiv.org/abs/2103.14708v1
• Date: Fri, 26 Mar 2021 19:42:21 GMT
• Title: Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB
• Authors: Bo Sun, Junchi Yan, Xiao Zhou, and Yinqiang Zheng
• Abstract summary: It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. This paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery.
• Score: 84.1657998542458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To reconstruct spectral signals from multi-channel observations, in particular trichromatic RGBs, has recently emerged as a promising alternative to traditional scanning-based spectral imager. It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. To improve accuracy, data-driven algorithms have been proposed to retrieve the best response curves of existing RGB cameras, or even to design brand new three-channel response curves. Instead, this paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery, which stands out as an in-between solution with better trade-off between reconstruction accuracy and implementation complexity. We further propose a deep learning based spectral reconstruction method, which allows to recover the illumination spectrum as well. Experiment results with both synthetic and real images under daylight illumination have shown the benefits of our IR-cut filter tuning method and our illumination-aware spectral reconstruction method.

Related papers

• Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation [22.07699685165064]
  Recent efforts in data-driven spectral reconstruction aim at extracting spectral information from RGB images captured by cost-effective RGB cameras. We evaluate both the practical limitations with respect to current datasets and overfitting, as well as fundamental limitations with respect to the nature of the information encoded in the RGB images. We propose to exploit the combination of metameric data augmentation and optical lens aberrations to improve the encoding of the metameric information into the RGB image.
  arXiv  Detail & Related papers  (2024-01-08T11:46:45Z)
• SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field [70.15900280156262]
  We propose an end-to-end Neural Radiance Field (NeRF)-based architecture for high-quality physically based rendering from a novel spectral perspective. SpectralNeRF is superior to recent NeRF-based methods when synthesizing new views on synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-12-14T07:19:31Z)
• Spec-NeRF: Multi-spectral Neural Radiance Fields [9.242830798112855]
  We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction.
  arXiv  Detail & Related papers  (2023-09-14T16:17:55Z)
• Enhancing Low-Light Images Using Infrared-Encoded Images [81.8710581927427]
  Previous arts mainly focus on the low-light images captured in the visible spectrum using pixel-wise loss. We propose a novel approach to increase the visibility of images captured under low-light environments by removing the in-camera infrared (IR) cut-off filter.
  arXiv  Detail & Related papers  (2023-07-09T08:29:19Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• Learning to Recover Spectral Reflectance from RGB Images [20.260831758913902]
  spectral reflectance recovery (SRR) from RGB images is challenging and costly. Most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images. We propose a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information.
  arXiv  Detail & Related papers  (2023-04-04T23:27:02Z)
• Deep Coding Patterns Design for Compressive Near-Infrared Spectral Classification [80.93625278357229]
  spectral classification can be performed directly in the compressive domain, considering the amount of spectral information embedded in the measurements. This work proposes an end-to-end approach to jointly design the coding patterns used in CSI and the network parameters to perform spectral classification directly from the embedded near-infrared compressive measurements.
  arXiv  Detail & Related papers  (2022-05-27T15:55:53Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• Spectral MVIR: Joint Reconstruction of 3D Shape and Spectral Reflectance [15.600451871592107]
  We present a rendering model that considers both geometric and photometric principles in the image formation. We build a cost-optimization MVIR framework for the joint reconstruction of the 3D shape and the per-vertex spectral reflectance.
  arXiv  Detail & Related papers  (2021-04-15T08:36:23Z)
• Fast Hyperspectral Image Recovery via Non-iterative Fusion of Dual-Camera Compressive Hyperspectral Imaging [22.683482662362337]
  Coded aperture snapshot spectral imaging (CASSI) is a promising technique to capture the three-dimensional hyperspectral image (HSI) Various regularizers have been exploited to reconstruct the 3D data from the 2D measurement. One feasible solution is to utilize additional information such as the RGB measurement in CASSI.
  arXiv  Detail & Related papers  (2020-12-30T10:29:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.