Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain

• URL: http://arxiv.org/abs/2401.02161v1
• Date: Thu, 4 Jan 2024 09:18:31 GMT
• Title: Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain
• Authors: Xuanhua He, Tao Hu, Guoli Wang, Zejin Wang, Run Wang, Qian Zhang, Keyu Yan, Ziyi Chen, Rui Li, Chenjun Xie, Jie Zhang, Man Zhou
• Abstract summary: Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images. We present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
• Score: 27.1716081216131
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RAW to sRGB mapping, which aims to convert RAW images from smartphones into RGB form equivalent to that of Digital Single-Lens Reflex (DSLR) cameras, has become an important area of research. However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations. Recent methods directly rebuild color mapping and spatial structure via shared deep representation, limiting optimal performance. Inspired by Image Signal Processing (ISP) pipeline, which distinguishes image restoration and enhancement, we present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization. FourierISP is comprised of three subnetworks: Phase Enhance Subnet for structural refinement, Amplitude Refine Subnet for color learning, and Color Adaptation Subnet for blending them in a smooth manner. This approach sharpens both color and structure, and extensive evaluations across varied datasets confirm that our approach realizes state-of-the-art results. Code will be available at ~\url{https://github.com/alexhe101/FourierISP}.

Related papers

• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition [17.986236212580565]
  We present MetaISP, a model designed to learn how to translate between the color and local contrast characteristics of different devices. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest.
  arXiv  Detail & Related papers  (2024-01-06T14:06:29Z)
• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Efficient Visual Computing with Camera RAW Snapshots [41.9863557302409]
  Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP) One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing. We propose a novel $rho$-Vision framework to perform high-level semantic understanding and low-level compression using RAW images.
  arXiv  Detail & Related papers  (2022-12-15T12:54:21Z)
• Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild [159.71025525493354]
  We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
  arXiv  Detail & Related papers  (2022-03-20T20:13:59Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• Deep Burst Super-Resolution [165.90445859851448]
  We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
  arXiv  Detail & Related papers  (2021-01-26T18:57:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.