Snapshot multi-spectral imaging through defocusing and a Fourier imager network

• URL: http://arxiv.org/abs/2501.14287v1
• Date: Fri, 24 Jan 2025 07:04:27 GMT
• Title: Snapshot multi-spectral imaging through defocusing and a Fourier imager network
• Authors: Xilin Yang, Michael John Fanous, Hanlong Chen, Ryan Lee, Paloma Casteleiro Costa, Yuhang Li, Luzhe Huang, Yijie Zhang, Aydogan Ozcan,
• Abstract summary: We introduce a snapshot multi-spectral imaging approach employing a standard monochrome image sensor with no additional spectral filters or customized components. This deep learning-powered framework achieves high-quality multi-spectral image reconstruction using snapshot image acquisition with a monochrome image sensor.
• Score: 13.068631760956265
• License:
• Abstract: Multi-spectral imaging, which simultaneously captures the spatial and spectral information of a scene, is widely used across diverse fields, including remote sensing, biomedical imaging, and agricultural monitoring. Here, we introduce a snapshot multi-spectral imaging approach employing a standard monochrome image sensor with no additional spectral filters or customized components. Our system leverages the inherent chromatic aberration of wavelength-dependent defocusing as a natural source of physical encoding of multi-spectral information; this encoded image information is rapidly decoded via a deep learning-based multi-spectral Fourier Imager Network (mFIN). We experimentally tested our method with six illumination bands and demonstrated an overall accuracy of 92.98% for predicting the illumination channels at the input and achieved a robust multi-spectral image reconstruction on various test objects. This deep learning-powered framework achieves high-quality multi-spectral image reconstruction using snapshot image acquisition with a monochrome image sensor and could be useful for applications in biomedicine, industrial quality control, and agriculture, among others.

Related papers

• Multispectral Stereo-Image Fusion for 3D Hyperspectral Scene Reconstruction [4.2056926734482065]
  We present a novel approach combining two calibrated multispectral real-time capable snapshot cameras, covering different spectral ranges, into a stereo-system. The combined use of different multispectral snapshot cameras enables both 3D reconstruction and spectral analysis.
  arXiv  Detail & Related papers  (2023-12-15T13:20:35Z)
• TOP-ReID: Multi-spectral Object Re-Identification with Token Permutation [64.65950381870742]
  We propose a cyclic token permutation framework for multi-spectral object ReID, dubbled TOP-ReID. We also propose a Token Permutation Module (TPM) for cyclic multi-spectral feature aggregation. Our proposed framework can generate more discriminative multi-spectral features for robust object ReID.
  arXiv  Detail & Related papers  (2023-12-15T08:54:15Z)
• Spec-NeRF: Multi-spectral Neural Radiance Fields [9.242830798112855]
  We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction.
  arXiv  Detail & Related papers  (2023-09-14T16:17:55Z)
• Longwave infrared multispectral image sensor system using aluminum-germanium plasmonic filter arrays [0.8081564951955755]
  A multispectral camera records image data in various wavelengths across the electromagnetic spectrum to acquire additional information that a conventional camera fails to capture. We experimentally demonstrate an LWIR multispectral image sensor with three wavelength bands using optical elements made of an aluminum-based plasmonic filter array sandwiched in germanium. Our work demonstrates a versatile spectral thermography technique for detecting target signatures in the LWIR range and other advanced spectral analyses.
  arXiv  Detail & Related papers  (2023-03-03T01:47:32Z)
• Toward deep-learning-assisted spectrally-resolved imaging of magnetic noise [52.77024349608834]
  We implement a deep neural network to efficiently reconstruct the spectral density of the underlying fluctuating magnetic field. These results create opportunities for the application of machine-learning methods to color-center-based nanoscale sensing and imaging.
  arXiv  Detail & Related papers  (2022-08-01T19:18:26Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics [72.9038524082252]
  We propose a compact single-shot monocular hyperspectral-depth (HS-D) imaging method. Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum. To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager.
  arXiv  Detail & Related papers  (2020-09-01T14:19:35Z)
• Advances in Deep Learning for Hyperspectral Image Analysis--Addressing Challenges Arising in Practical Imaging Scenarios [7.41157183358269]
  We will review advances in the community that leverage deep learning for robust hyperspectral image analysis. challenges include limited ground truth and high dimensional nature of the data. Specifically, we will review unsupervised, semi-supervised and active learning approaches to image analysis.
  arXiv  Detail & Related papers  (2020-07-16T19:51:02Z)
• Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array [1.6058099298620423]
  Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring. Traditional hyperspectral imagers are prohibitively slow and expensive for widespread adoption. We propose a compact, compact and inexpensive camera for hyperspectral imaging.
  arXiv  Detail & Related papers  (2020-06-15T17:31:17Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.