ParamISP: Learned Forward and Inverse ISPs using Camera Parameters
- URL: http://arxiv.org/abs/2312.13313v2
- Date: Mon, 15 Apr 2024 01:49:23 GMT
- Title: ParamISP: Learned Forward and Inverse ISPs using Camera Parameters
- Authors: Woohyeok Kim, Geonu Kim, Junyong Lee, Seungyong Lee, Seung-Hwan Baek, Sunghyun Cho,
- Abstract summary: ParamISP is a learning-based method for forward and inverse conversion between sRGB and RAW images.
It can be effectively used for a variety of applications such as deblurring dataset synthesis, raw deblurring, HDR reconstruction, and camera-to-camera transfer.
- Score: 27.244062839494
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: RAW images are rarely shared mainly due to its excessive data size compared to their sRGB counterparts obtained by camera ISPs. Learning the forward and inverse processes of camera ISPs has been recently demonstrated, enabling physically-meaningful RAW-level image processing on input sRGB images. However, existing learning-based ISP methods fail to handle the large variations in the ISP processes with respect to camera parameters such as ISO and exposure time, and have limitations when used for various applications. In this paper, we propose ParamISP, a learning-based method for forward and inverse conversion between sRGB and RAW images, that adopts a novel neural-network module to utilize camera parameters, which is dubbed as ParamNet. Given the camera parameters provided in the EXIF data, ParamNet converts them into a feature vector to control the ISP networks. Extensive experiments demonstrate that ParamISP achieve superior RAW and sRGB reconstruction results compared to previous methods and it can be effectively used for a variety of applications such as deblurring dataset synthesis, raw deblurring, HDR reconstruction, and camera-to-camera transfer.
Related papers
- RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images [51.68432586065828]
We introduce RAW-Adapter, a novel approach aimed at adapting sRGB pre-trained models to camera RAW data.
Raw-Adapter comprises input-level adapters that employ learnable ISP stages to adjust RAW inputs, as well as model-level adapters to build connections between ISP stages and subsequent high-level networks.
arXiv Detail & Related papers (2024-08-27T06:14:54Z) - Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks.
We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images.
Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
arXiv Detail & Related papers (2024-04-17T17:11:47Z) - Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs [53.68932498994655]
This paper introduces a novel method for unpaired learning of raw-to-raw translation across diverse cameras.
It accurately maps raw images captured by a certain camera to the target camera, facilitating the generalization of learnable ISPs to new unseen cameras.
Our method demonstrates superior performance on real camera datasets, achieving higher accuracy compared to previous state-of-the-art techniques.
arXiv Detail & Related papers (2024-04-16T16:17:48Z) - Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain [27.1716081216131]
Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images.
We present a novel Neural ISP framework, named FourierISP.
This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
arXiv Detail & Related papers (2024-01-04T09:18:31Z) - Self-Supervised Reversed Image Signal Processing via Reference-Guided
Dynamic Parameter Selection [1.1602089225841632]
We propose a self-supervised reversed ISP method that does not require metadata and paired images.
The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image.
We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
arXiv Detail & Related papers (2023-03-24T11:12:05Z) - Efficient Visual Computing with Camera RAW Snapshots [41.9863557302409]
Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP)
One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing.
We propose a novel $rho$-Vision framework to perform high-level semantic understanding and low-level compression using RAW images.
arXiv Detail & Related papers (2022-12-15T12:54:21Z) - Reversed Image Signal Processing and RAW Reconstruction. AIM 2022
Challenge Report [109.2135194765743]
This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction.
We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
arXiv Detail & Related papers (2022-10-20T10:43:53Z) - Transform your Smartphone into a DSLR Camera: Learning the ISP in the
Wild [159.71025525493354]
We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone.
To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
arXiv Detail & Related papers (2022-03-20T20:13:59Z) - Model-Based Image Signal Processors via Learnable Dictionaries [6.766416093990318]
Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP)
Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping.
We present a novel hybrid model-based and data-driven ISP that is both learnable and interpretable.
arXiv Detail & Related papers (2022-01-10T08:36:10Z) - Replacing Mobile Camera ISP with a Single Deep Learning Model [171.49776472948957]
PyNET is a novel pyramidal CNN architecture designed for fine-grained image restoration.
The model is trained to convert RAW Bayer data obtained directly from mobile camera sensor into photos captured with a professional high-end DSLR camera.
arXiv Detail & Related papers (2020-02-13T14:22:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.