ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency

• URL: http://arxiv.org/abs/2503.19283v1
• Date: Tue, 25 Mar 2025 02:29:39 GMT
• Title: ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency
• Authors: Yang Ren, Hai Jiang, Menglong Yang, Wei Li, Shuaicheng Liu,
• Abstract summary: RAW-to-sRGB mapping aims to generate DSLR-quality sRGB images from raw data captured by smartphone sensors.<n>ISPDiffuser is a diffusion-based framework that separates the RAW-to-sRGB mapping into detail reconstruction in grayscale space.<n>ISPDiffuser outperforms state-of-the-art competitors both quantitatively and visually.
• Score: 32.05482995863444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RAW-to-sRGB mapping, or the simulation of the traditional camera image signal processor (ISP), aims to generate DSLR-quality sRGB images from raw data captured by smartphone sensors. Despite achieving comparable results to sophisticated handcrafted camera ISP solutions, existing learning-based methods still struggle with detail disparity and color distortion. In this paper, we present ISPDiffuser, a diffusion-based decoupled framework that separates the RAW-to-sRGB mapping into detail reconstruction in grayscale space and color consistency mapping from grayscale to sRGB. Specifically, we propose a texture-aware diffusion model that leverages the generative ability of diffusion models to focus on local detail recovery, in which a texture enrichment loss is further proposed to prompt the diffusion model to generate more intricate texture details. Subsequently, we introduce a histogram-guided color consistency module that utilizes color histogram as guidance to learn precise color information for grayscale to sRGB color consistency mapping, with a color consistency loss designed to constrain the learned color information. Extensive experimental results show that the proposed ISPDiffuser outperforms state-of-the-art competitors both quantitatively and visually. The code is available at https://github.com/RenYangSCU/ISPDiffuser.

Related papers

• RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation [4.625376287612609]
  We propose a novel diffusion-based method for generating RAW images guided by RGB images. This approach yields high-fidelity RAW images, enabling the creation of camera-specific RAW datasets. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets, revealing its effectiveness for object detection in RAW imagery.
  arXiv  Detail & Related papers  (2024-11-20T09:40:12Z)
• Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain [27.1716081216131]
  Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images. We present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
  arXiv  Detail & Related papers  (2024-01-04T09:18:31Z)
• Image Demoireing in RAW and sRGB Domains [18.921026683632146]
  We develop Skip-Connection-based Demoireing Module (SCDM) with Gated Feedback Module (GFM) and Frequency Selection Module (FSM) We design a RGB Guided ISP (RGISP) to learn a device-dependent ISP, assisting the process of color recovery. Our RRID outperforms state-of-the-art approaches, in terms of the performance in moire pattern removal and color cast correction by 0.62dB in PSNR and 0.003 in SSIM.
  arXiv  Detail & Related papers  (2023-12-14T16:00:28Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Learning sRGB-to-Raw-RGB De-rendering with Content-Aware Metadata [46.28281823015191]
  We show how to improve the de-rendering results by jointly learning sampling and reconstruction. Our experiments show that our learned sampling can adapt to the image content to produce better raw reconstructions than existing methods.
  arXiv  Detail & Related papers  (2022-06-03T20:43:17Z)
• Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild [159.71025525493354]
  We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
  arXiv  Detail & Related papers  (2022-03-20T20:13:59Z)
• Model-Based Image Signal Processors via Learnable Dictionaries [6.766416093990318]
  Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP) Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping. We present a novel hybrid model-based and data-driven ISP that is both learnable and interpretable.
  arXiv  Detail & Related papers  (2022-01-10T08:36:10Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.