Training Neural Networks on RAW and HDR Images for Restoration Tasks

• URL: http://arxiv.org/abs/2312.03640v1
• Date: Wed, 6 Dec 2023 17:47:16 GMT
• Title: Training Neural Networks on RAW and HDR Images for Restoration Tasks
• Authors: Lei Luo, Alexandre Chapiro, Xiaoyu Xiang, Yuchen Fan, Rakesh Ranjan, Rafal Mantiuk
• Abstract summary: In this work, we test approaches on three popular image restoration applications: denoising, deblurring, and single-image super-resolution. Our results indicate that neural networks train significantly better on HDR and RAW images represented in display color spaces. This small change to the training strategy can bring a very substantial gain in performance, up to 10-15 dB.
• Score: 59.41340420564656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The vast majority of standard image and video content available online is represented in display-encoded color spaces, in which pixel values are conveniently scaled to a limited range (0-1) and the color distribution is approximately perceptually uniform. In contrast, both camera RAW and high dynamic range (HDR) images are often represented in linear color spaces, in which color values are linearly related to colorimetric quantities of light. While training on commonly available display-encoded images is a well-established practice, there is no consensus on how neural networks should be trained for tasks on RAW and HDR images in linear color spaces. In this work, we test several approaches on three popular image restoration applications: denoising, deblurring, and single-image super-resolution. We examine whether HDR/RAW images need to be display-encoded using popular transfer functions (PQ, PU21, mu-law), or whether it is better to train in linear color spaces, but use loss functions that correct for perceptual non-uniformity. Our results indicate that neural networks train significantly better on HDR and RAW images represented in display-encoded color spaces, which offer better perceptual uniformity than linear spaces. This small change to the training strategy can bring a very substantial gain in performance, up to 10-15 dB.

Related papers

• You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement [50.37253008333166]
  Low-Light Image Enhancement (LLIE) task tends to restore the details and visual information from corrupted low-light images. We propose a novel trainable color space, named Horizontal/Vertical-Intensity (HVI) It not only decouples brightness and color from RGB channels to mitigate the instability during enhancement but also adapts to low-light images in different illumination ranges due to the trainable parameters.
  arXiv  Detail & Related papers  (2024-02-08T16:47:43Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild [159.71025525493354]
  We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
  arXiv  Detail & Related papers  (2022-03-20T20:13:59Z)
• Locally Weighted Mean Phase Angle (LWMPA) Based Tone Mapping Quality Index (TMQI-3) [0.0]
  We propose a metric called the Tone Mapping Quality Index (TMQI-3), which evaluates the quality of the Low dynamic range (LDR) image based on its objective score. TMQI-3 is noise resilient, takes account of structure and naturalness, and works on all three color channels combined into one component.
  arXiv  Detail & Related papers  (2021-09-17T22:17:20Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Learning to Structure an Image with Few Colors [59.34619548026885]
  We propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner. With only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset. For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime.
  arXiv  Detail & Related papers  (2020-03-17T17:56:15Z)
• Adaptive Dithering Using Curved Markov-Gaussian Noise in the Quantized Domain for Mapping SDR to HDR Image [2.913398015606848]
  High Dynamic Range (SDR) imaging is gaining increased attention due to its realistic content, for not only regular displays but also smartphones. We present a technique for noise generation that operates on pixels of a quantized image. We vary the magnitude and structure of the noise pattern adaptively based on the luma of the quantized pixel and the slope of the inverse-tone mapping function.
  arXiv  Detail & Related papers  (2020-01-20T05:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.