Real-RawVSR: Real-World Raw Video Super-Resolution with a Benchmark Dataset

• URL: http://arxiv.org/abs/2209.12475v1
• Date: Mon, 26 Sep 2022 07:33:31 GMT
• Title: Real-RawVSR: Real-World Raw Video Super-Resolution with a Benchmark Dataset
• Authors: Huanjing Yue, Zhiming Zhang, Jingyu Yang
• Abstract summary: We construct a real-world raw video SR dataset and propose a corresponding SR method. We utilize two DSLR cameras and a beam-splitter to capture low-resolution (LR) and high-resolution (HR) raw videos with 2x, 3x, and 4x magnifications. Experimental results demonstrate that the proposed method outperforms benchmark real and synthetic video SR methods with either raw or sRGB inputs.
• Score: 19.118790225253964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, real image super-resolution (SR) has achieved promising results due to the development of SR datasets and corresponding real SR methods. In contrast, the field of real video SR is lagging behind, especially for real raw videos. Considering the superiority of raw image SR over sRGB image SR, we construct a real-world raw video SR (Real-RawVSR) dataset and propose a corresponding SR method. We utilize two DSLR cameras and a beam-splitter to simultaneously capture low-resolution (LR) and high-resolution (HR) raw videos with 2x, 3x, and 4x magnifications. There are 450 video pairs in our dataset, with scenes varying from indoor to outdoor, and motions including camera and object movements. To our knowledge, this is the first real-world raw VSR dataset. Since the raw video is characterized by the Bayer pattern, we propose a two-branch network, which deals with both the packed RGGB sequence and the original Bayer pattern sequence, and the two branches are complementary to each other. After going through the proposed co-alignment, interaction, fusion, and reconstruction modules, we generate the corresponding HR sRGB sequence. Experimental results demonstrate that the proposed method outperforms benchmark real and synthetic video SR methods with either raw or sRGB inputs. Our code and dataset are available at https://github.com/zmzhang1998/Real-RawVSR.

Related papers

• Towards RAW Object Detection in Diverse Conditions [65.30190654593842]
  We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW. We distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain to assist RAW pre-training.
  arXiv  Detail & Related papers  (2024-11-24T01:23:04Z)
• RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation [4.625376287612609]
  We propose a novel diffusion-based method for generating RAW images guided by RGB images. This approach yields high-fidelity RAW images, enabling the creation of camera-specific RAW datasets. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets, revealing its effectiveness for object detection in RAW imagery.
  arXiv  Detail & Related papers  (2024-11-20T09:40:12Z)
• Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution [56.98910228239627]
  Real-world image super-resolution (Real SR) aims to generate high-fidelity, detail-rich high-resolution (HR) images from low-resolution (LR) counterparts. Existing Real SR methods primarily focus on generating details from the LR RGB domain, often leading to a lack of richness or fidelity in fine details. We pioneer the use of details hidden in RAW data to complement existing RGB-only methods, yielding superior outputs.
  arXiv  Detail & Related papers  (2024-11-16T13:29:50Z)
• BSRAW: Improving Blind RAW Image Super-Resolution [63.408484584265985]
  We tackle blind image super-resolution in the RAW domain. We design a realistic degradation pipeline tailored specifically for training models with raw sensor data. Our BSRAW models trained with our pipeline can upscale real-scene RAW images and improve their quality.
  arXiv  Detail & Related papers  (2023-12-24T14:17:28Z)
• RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image [36.17182977927645]
  High dynamic range (RGB) images capture much more intensity levels than standard ones. Current methods predominantly generate HDR images from 8-bit low dynamic range (LDR) s images that have been degraded by the camera processing pipeline. Unlike existing methods, the core idea of this work is to incorporate more informative Raw sensor data to generate HDR images.
  arXiv  Detail & Related papers  (2023-09-05T07:58:21Z)
• HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains [23.309488653045026]
  High dynamic range ( HDR) video reconstruction is attracting more and more attention due to the superior visual quality compared with those of low dynamic range (LDR) videos. There are still no real LDR- pairs for dynamic scenes due to the difficulty in capturing LDR- frames simultaneously. In this work, we propose to utilize a staggered sensor to capture two alternate exposure images simultaneously, which are then fused into an HDR frame in both raw and sRGB domains.
  arXiv  Detail & Related papers  (2023-04-10T11:59:03Z)
• Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution [65.20905703823965]
  Video super-resolution (VSR) aiming to reconstruct a high-resolution (HR) video from its low-resolution (LR) counterpart has made tremendous progress in recent years. It remains challenging to deploy existing VSR methods to real-world data with complex degradations. EAVSR takes the proposed multi-layer adaptive spatial transform network (MultiAdaSTN) to refine the offsets provided by the pre-trained optical flow estimation network.
  arXiv  Detail & Related papers  (2022-12-10T17:41:46Z)
• Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report [109.2135194765743]
  This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.
  arXiv  Detail & Related papers  (2022-10-20T10:43:53Z)
• Invertible Image Signal Processing [42.109752151834456]
  Invertible Image Signal Processing (InvISP) pipeline enables rendering visually appealing sRGB images. We can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead.
  arXiv  Detail & Related papers  (2021-03-28T06:30:15Z)
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution [69.2432352477966]
  Real image super-resolution(Real-SR) focus on the relationship between real-world high-resolution(HR) and low-resolution(LR) image. In this article, we propose a Dual-path Dynamic Enhancement Network(DDet) for Real-SR. Unlike conventional methods which stack up massive convolutional blocks for feature representation, we introduce a content-aware framework to study non-inherently aligned image pair.
  arXiv  Detail & Related papers  (2020-02-25T18:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.