Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild

• URL: http://arxiv.org/abs/2203.10636v2
• Date: Tue, 22 Mar 2022 13:16:10 GMT
• Title: Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild
• Authors: Ardhendu Shekhar Tripathi, Martin Danelljan, Samarth Shukla, Radu Timofte, Luc Van Gool
• Abstract summary: We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
• Score: 159.71025525493354
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose a trainable Image Signal Processing (ISP) framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image. During inference, we predict the target color image by designing a color prediction network with efficient Global Context Transformer modules. The latter effectively leverage global information to learn consistent color and tone mappings. We further propose a robust masked aligned loss to identify and discard regions with inaccurate motion estimation during training. Lastly, we introduce the ISP in the Wild (ISPW) dataset, consisting of weakly paired phone RAW and DSLR sRGB images. We extensively evaluate our method, setting a new state-of-the-art on two datasets.

Related papers

• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs [53.68932498994655]
  This paper introduces a novel method for unpaired learning of raw-to-raw translation across diverse cameras. It accurately maps raw images captured by a certain camera to the target camera, facilitating the generalization of learnable ISPs to new unseen cameras. Our method demonstrates superior performance on real camera datasets, achieving higher accuracy compared to previous state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-04-16T16:17:48Z)
• MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition [17.986236212580565]
  We present MetaISP, a model designed to learn how to translate between the color and local contrast characteristics of different devices. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest.
  arXiv  Detail & Related papers  (2024-01-06T14:06:29Z)
• Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain [27.1716081216131]
  Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images. We present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
  arXiv  Detail & Related papers  (2024-01-04T09:18:31Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Model-Based Image Signal Processors via Learnable Dictionaries [6.766416093990318]
  Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP) Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping. We present a novel hybrid model-based and data-driven ISP that is both learnable and interpretable.
  arXiv  Detail & Related papers  (2022-01-10T08:36:10Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Replacing Mobile Camera ISP with a Single Deep Learning Model [171.49776472948957]
  PyNET is a novel pyramidal CNN architecture designed for fine-grained image restoration. The model is trained to convert RAW Bayer data obtained directly from mobile camera sensor into photos captured with a professional high-end DSLR camera.
  arXiv  Detail & Related papers  (2020-02-13T14:22:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.