Learning deep illumination-robust features from multispectral filter array images

• URL: http://arxiv.org/abs/2407.15472v2
• Date: Tue, 23 Jul 2024 09:27:10 GMT
• Title: Learning deep illumination-robust features from multispectral filter array images
• Authors: Anis Amziane,
• Abstract summary: Multispectral (MS) snapshot cameras capture multiple spectral bands in a single shot, resulting in a raw image where each pixel holds only one channel value. Training on fully-defined MS images can be computationally intensive, particularly with deep neural networks (DNNs) This paper presents an original approach to learn discriminant and illumination-robust features from raw images.
• Score: 0.5439020425819
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multispectral (MS) snapshot cameras equipped with a MS filter array (MSFA), capture multiple spectral bands in a single shot, resulting in a raw mosaic image where each pixel holds only one channel value. The fully-defined MS image is estimated from the raw one through $\textit{demosaicing}$, which inevitably introduces spatio-spectral artifacts. Moreover, training on fully-defined MS images can be computationally intensive, particularly with deep neural networks (DNNs), and may result in features lacking discrimination power due to suboptimal learning of spatio-spectral interactions. Furthermore, outdoor MS image acquisition occurs under varying lighting conditions, leading to illumination-dependent features. This paper presents an original approach to learn discriminant and illumination-robust features directly from raw images. It involves: $\textit{raw spectral constancy}$ to mitigate the impact of illumination, $\textit{MSFA-preserving}$ transformations suited for raw image augmentation to train DNNs on diverse raw textures, and $\textit{raw-mixing}$ to capture discriminant spatio-spectral interactions in raw images. Experiments on MS image classification show that our approach outperforms both handcrafted and recent deep learning-based methods, while also requiring significantly less computational effort.

Related papers

• Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement [71.13353154514418]
  Low-light image enhancement, particularly in cross-domain tasks such as mapping from the raw domain to the sRGB domain, remains a significant challenge. We present a novel Mamba scanning mechanism, called RAWMamba, to effectively handle raw images with different CFAs. We also present a Retinex Decomposition Module (RDM) grounded in Retinex prior, which decouples illumination from reflectance to facilitate more effective denoising and automatic non-linear exposure correction.
  arXiv  Detail & Related papers  (2024-09-11T06:12:03Z)
• Deep Learning Based Speckle Filtering for Polarimetric SAR Images. Application to Sentinel-1 [51.404644401997736]
  We propose a complete framework to remove speckle in polarimetric SAR images using a convolutional neural network. Experiments show that the proposed approach offers exceptional results in both speckle reduction and resolution preservation.
  arXiv  Detail & Related papers  (2024-08-28T10:07:17Z)
• A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss [11.585269110131659]
  Low-light image enhancement aims to improve the perception of images collected in dim environments. Existing methods cannot adaptively extract the differentiated luminance information, which will easily cause over-exposure and under-exposure. We propose a multi-scale attention Transformer named MSATr, which sufficiently extracts local and global features for light balance to improve the visual quality.
  arXiv  Detail & Related papers  (2023-12-27T10:07:11Z)
• Multi-spectral Class Center Network for Face Manipulation Detection and Localization [52.569170436393165]
  We propose a novel Multi-Spectral Class Center Network (MSCCNet) for face manipulation detection and localization. Based on the features of different frequency bands, the MSCC module collects multi-spectral class centers and computes pixel-to-class relations. Applying multi-spectral class-level representations suppresses the semantic information of the visual concepts which is insensitive to manipulated regions of forgery images.
  arXiv  Detail & Related papers  (2023-05-18T08:09:20Z)
• Enhancing Low-Light Images in Real World via Cross-Image Disentanglement [58.754943762945864]
  We propose a new low-light image enhancement dataset consisting of misaligned training images with real-world corruptions. Our model achieves state-of-the-art performances on both the newly proposed dataset and other popular low-light datasets.
  arXiv  Detail & Related papers  (2022-01-10T03:12:52Z)
• Deep Contrastive Patch-Based Subspace Learning for Camera Image Signal Processing [5.678834480723395]
  We present a specific patch-based, local subspace deep neural network that improves Camera ISP to be robust to heterogeneous artifacts. We call our three-fold deep-trained model the Patch Subspace Learning Autoencoder (PSL-AE) PSL-AE encodes patches extracted from noisy a nd clean image pairs, with different artifact types or distortion levels, by contrastive learning.
  arXiv  Detail & Related papers  (2021-04-01T04:40:22Z)
• Learning to Enhance Visual Quality via Hyperspectral Domain Mapping [8.365634649800658]
  SpecNet computes spectral profile to estimate pixel-wise dynamic range adjustment of a given image. We incorporate a self-supervision and a spectral profile regularization network to infer a plausible HSI from an RGB image.
  arXiv  Detail & Related papers  (2021-02-10T13:27:34Z)
• Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning [29.834065415830764]
  spectral superresolution (SSR) of MS imagery is challenging and less investigated due to its high ill-posedness in inverse imaging. We develop a simple but effective method, called joint sparse and low-rank learning (J-SLoL), to spectrally enhance MS images by jointly learning low-rank HS-MS dictionary pairs from overlapped regions.
  arXiv  Detail & Related papers  (2020-07-28T06:08:44Z)
• Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning [48.890709236564945]
  A small ISO and a small exposure time are usually used to capture an image in the back or low light conditions. In this paper, a single image brightening algorithm is introduced to brighten such an image. The proposed algorithm includes a unique hybrid learning framework to generate two virtual images with large exposure times.
  arXiv  Detail & Related papers  (2020-07-04T08:23:07Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.