Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution

• URL: http://arxiv.org/abs/2411.10798v2
• Date: Thu, 21 Nov 2024 00:54:15 GMT
• Title: Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution
• Authors: Long Peng, Wenbo Li, Jiaming Guo, Xin Di, Haoze Sun, Yong Li, Renjing Pei, Yang Wang, Yang Cao, Zheng-Jun Zha,
• Abstract summary: Real-world image super-resolution (Real SR) aims to generate high-fidelity, detail-rich high-resolution (HR) images from low-resolution (LR) counterparts. Existing Real SR methods primarily focus on generating details from the LR RGB domain, often leading to a lack of richness or fidelity in fine details. We pioneer the use of details hidden in RAW data to complement existing RGB-only methods, yielding superior outputs.
• Score: 56.98910228239627
• License:
• Abstract: Real-world image super-resolution (Real SR) aims to generate high-fidelity, detail-rich high-resolution (HR) images from low-resolution (LR) counterparts. Existing Real SR methods primarily focus on generating details from the LR RGB domain, often leading to a lack of richness or fidelity in fine details. In this paper, we pioneer the use of details hidden in RAW data to complement existing RGB-only methods, yielding superior outputs. We argue that key image processing steps in Image Signal Processing, such as denoising and demosaicing, inherently result in the loss of fine details in LR images, making LR RAW a valuable information source. To validate this, we present RealSR-RAW, a comprehensive dataset comprising over 10,000 pairs with LR and HR RGB images, along with corresponding LR RAW, captured across multiple smartphones under varying focal lengths and diverse scenes. Additionally, we propose a novel, general RAW adapter to efficiently integrate LR RAW data into existing CNNs, Transformers, and Diffusion-based Real SR models by suppressing the noise contained in LR RAW and aligning its distribution. Extensive experiments demonstrate that incorporating RAW data significantly enhances detail recovery and improves Real SR performance across ten evaluation metrics, including both fidelity and perception-oriented metrics. Our findings open a new direction for the Real SR task, with the dataset and code will be made available to support future research.

Related papers

• RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation [4.625376287612609]
  We propose a novel diffusion-based method for generating RAW images guided by RGB images. This approach yields high-fidelity RAW images, enabling the creation of camera-specific RAW datasets. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets, revealing its effectiveness for object detection in RAW imagery.
  arXiv  Detail & Related papers  (2024-11-20T09:40:12Z)
• ClearSR: Latent Low-Resolution Image Embeddings Help Diffusion-Based Real-World Super Resolution Models See Clearer [68.72454974431749]
  We present ClearSR, a new method that can better take advantage of latent low-resolution image (LR) embeddings for diffusion-based real-world image super-resolution (Real-ISR) Our model can achieve better performance across multiple metrics on several test sets and generate more consistent SR results with LR images than existing methods.
  arXiv  Detail & Related papers  (2024-10-18T08:35:57Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Real-RawVSR: Real-World Raw Video Super-Resolution with a Benchmark Dataset [19.118790225253964]
  We construct a real-world raw video SR dataset and propose a corresponding SR method. We utilize two DSLR cameras and a beam-splitter to capture low-resolution (LR) and high-resolution (HR) raw videos with 2x, 3x, and 4x magnifications. Experimental results demonstrate that the proposed method outperforms benchmark real and synthetic video SR methods with either raw or sRGB inputs.
  arXiv  Detail & Related papers  (2022-09-26T07:33:31Z)
• Reference-based Image Super-Resolution with Deformable Attention Transformer [62.71769634254654]
  RefSR aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images. This paper proposes a deformable attention Transformer, namely DATSR, with multiple scales. Experiments demonstrate that our DATSR achieves state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-25T07:07:00Z)
• Towards Low Light Enhancement with RAW Images [101.35754364753409]
  We make the first benchmark effort to elaborate on the superiority of using RAW images in the low light enhancement. We develop a new evaluation framework, Factorized Enhancement Model (FEM), which decomposes the properties of RAW images into measurable factors. A RAW-guiding Exposure Enhancement Network (REENet) is developed, which makes trade-offs between the advantages and inaccessibility of RAW images in real applications.
  arXiv  Detail & Related papers  (2021-12-28T07:27:51Z)
• Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation [91.40265983636839]
  We propose a framework to learn SR from an arbitrary set of unpaired LR and HR images. We minimize the discrepancy between the generated data and real data while learning a degradation adaptive SR network. The proposed unpaired method achieves state-of-the-art SR results on real-world images, even in the datasets that favor the paired-learning methods more.
  arXiv  Detail & Related papers  (2020-01-08T05:17:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.