CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

• URL: http://arxiv.org/abs/2006.12709v1
• Date: Tue, 23 Jun 2020 02:59:11 GMT
• Title: CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks
• Authors: Mahmoud Afifi, Abdelrahman Abdelhamed, Abdullah Abuolaim, Abhijith Punnappurath, and Michael S. Brown
• Abstract summary: Cameras currently allow access to two image states: (i.e., raw sensor data) or (ii.) a highly-processed nonlinear image state (e.g., sRGB) There are many computer vision tasks that work best with a linear image state, such as image deblurring and image dehazing. We propose a deep learning framework, CIE XYZ Net, that can unprocess a nonlinear image back to the canonical CIE XYZ image.
• Score: 45.820956016608314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cameras currently allow access to two image states: (i) a minimally processed linear raw-RGB image state (i.e., raw sensor data) or (ii) a highly-processed nonlinear image state (e.g., sRGB). There are many computer vision tasks that work best with a linear image state, such as image deblurring and image dehazing. Unfortunately, the vast majority of images are saved in the nonlinear image state. Because of this, a number of methods have been proposed to "unprocess" nonlinear images back to a raw-RGB state. However, existing unprocessing methods have a drawback because raw-RGB images are sensor-specific. As a result, it is necessary to know which camera produced the sRGB output and use a method or network tailored for that sensor to properly unprocess it. This paper addresses this limitation by exploiting another camera image state that is not available as an output, but it is available inside the camera pipeline. In particular, cameras apply a colorimetric conversion step to convert the raw-RGB image to a device-independent space based on the CIE XYZ color space before they apply the nonlinear photo-finishing. Leveraging this canonical image state, we propose a deep learning framework, CIE XYZ Net, that can unprocess a nonlinear image back to the canonical CIE XYZ image. This image can then be processed by any low-level computer vision operator and re-rendered back to the nonlinear image. We demonstrate the usefulness of the CIE XYZ Net on several low-level vision tasks and show significant gains that can be obtained by this processing framework. Code and dataset are publicly available at https://github.com/mahmoudnafifi/CIE_XYZ_NET.

Related papers

• Toward Efficient Deep Blind RAW Image Restoration [56.41827271721955]
  We design a new realistic degradation pipeline for training deep blind RAW restoration models. Our pipeline considers realistic sensor noise, motion blur, camera shake, and other common degradations. The models trained with our pipeline and data from multiple sensors, can successfully reduce noise and blur, and recover details in RAW images captured from different cameras.
  arXiv  Detail & Related papers  (2024-09-26T18:34:37Z)
• SEL-CIE: Knowledge-Guided Self-Supervised Learning Framework for CIE-XYZ Reconstruction from Non-Linear sRGB Images [7.932206255996779]
  The CIE-XYZ color space is a device-independent linear space used as part of the camera pipeline. Images are usually saved in non-linear states, and achieving CIE-XYZ color images using conventional methods is not always possible. This paper proposes a framework for using SSL methods alongside paired data to reconstruct CIE-XYZ images and re-render sRGB images, outperforming existing approaches.
  arXiv  Detail & Related papers  (2024-05-20T17:20:41Z)
• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• BSRAW: Improving Blind RAW Image Super-Resolution [63.408484584265985]
  We tackle blind image super-resolution in the RAW domain. We design a realistic degradation pipeline tailored specifically for training models with raw sensor data. Our BSRAW models trained with our pipeline can upscale real-scene RAW images and improve their quality.
  arXiv  Detail & Related papers  (2023-12-24T14:17:28Z)
• Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection [1.1602089225841632]
  We propose a self-supervised reversed ISP method that does not require metadata and paired images. The proposed method converts a RGB image into a RAW-like image taken in the same environment with the same sensor as a reference RAW image. We show that the proposed method is able to learn various reversed ISPs with comparable accuracy to other state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-03-24T11:12:05Z)
• Efficient Visual Computing with Camera RAW Snapshots [41.9863557302409]
  Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP) One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing. We propose a novel $rho$-Vision framework to perform high-level semantic understanding and low-level compression using RAW images.
  arXiv  Detail & Related papers  (2022-12-15T12:54:21Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Invertible Image Signal Processing [42.109752151834456]
  Invertible Image Signal Processing (InvISP) pipeline enables rendering visually appealing sRGB images. We can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead.
  arXiv  Detail & Related papers  (2021-03-28T06:30:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.