Efficient HDR Reconstruction from Real-World Raw Images

• URL: http://arxiv.org/abs/2306.10311v5
• Date: Wed, 5 Jun 2024 07:44:13 GMT
• Title: Efficient HDR Reconstruction from Real-World Raw Images
• Authors: Qirui Yang, Yihao Liu, Qihua Chen, Huanjing Yue, Kun Li, Jingyu Yang,
• Abstract summary: High-definition screens on edge devices stimulate a strong demand for efficient high dynamic range ( HDR) algorithms. Many existing HDR methods either deliver unsatisfactory results or consume too much computational and memory resources. In this work, we discover an excellent opportunity for HDR reconstructing directly from raw images and investigating novel neural network structures.
• Score: 16.54071503000866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread usage of high-definition screens on edge devices stimulates a strong demand for efficient high dynamic range (HDR) algorithms. However, many existing HDR methods either deliver unsatisfactory results or consume too much computational and memory resources, hindering their application to high-resolution images (usually with more than 12 megapixels) in practice. In addition, existing HDR dataset collection methods often are labor-intensive. In this work, in a new aspect, we discover an excellent opportunity for HDR reconstructing directly from raw images and investigating novel neural network structures that benefit the deployment of mobile devices. Our key insights are threefold: (1) we develop a lightweight-efficient HDR model, RepUNet, using the structural re-parameterization technique to achieve fast and robust HDR; (2) we design a new computational raw HDR data formation pipeline and construct a real-world raw HDR dataset, RealRaw-HDR; (3) we propose a plug-and-play motion alignment loss to mitigate motion ghosting under limited bandwidth conditions. Our model contains less than 830K parameters and takes less than 3 ms to process an image of 4K resolution using one RTX 3090 GPU. While being highly efficient, our model also outperforms the state-of-the-art HDR methods in terms of PSNR, SSIM, and a color difference metric.

Related papers

• HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting [76.5908492298286]
  Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. We propose a new framework, High Dynamic Range Gaussian Splatting (-GS)
  arXiv  Detail & Related papers  (2024-05-24T00:46:58Z)
• Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry from Sparse Low Dynamic Range Panoramic Images [82.1477261107279]
  We propose the irradiance fields from sparse LDR panoramic images to increase the observation counts for faithful geometry recovery. Experiments demonstrate that the irradiance fields outperform state-of-the-art methods on both geometry recovery and HDR reconstruction.
  arXiv  Detail & Related papers  (2023-12-26T08:10:22Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image [36.17182977927645]
  High dynamic range (RGB) images capture much more intensity levels than standard ones. Current methods predominantly generate HDR images from 8-bit low dynamic range (LDR) s images that have been degraded by the camera processing pipeline. Unlike existing methods, the core idea of this work is to incorporate more informative Raw sensor data to generate HDR images.
  arXiv  Detail & Related papers  (2023-09-05T07:58:21Z)
• SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders [97.64072440883392]
  We propose a novel semi-supervised approach to realize few-shot HDR imaging via two stages of training, called SSHDR. Unlikely previous methods, directly recovering content and removing ghosts simultaneously, which is hard to achieve optimum. Experiments demonstrate that SSHDR outperforms state-of-the-art methods quantitatively and qualitatively within and across different datasets.
  arXiv  Detail & Related papers  (2023-04-14T03:42:51Z)
• HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains [23.309488653045026]
  High dynamic range ( HDR) video reconstruction is attracting more and more attention due to the superior visual quality compared with those of low dynamic range (LDR) videos. There are still no real LDR- pairs for dynamic scenes due to the difficulty in capturing LDR- frames simultaneously. In this work, we propose to utilize a staggered sensor to capture two alternate exposure images simultaneously, which are then fused into an HDR frame in both raw and sRGB domains.
  arXiv  Detail & Related papers  (2023-04-10T11:59:03Z)
• A Two-stage Deep Network for High Dynamic Range Image Reconstruction [0.883717274344425]
  This study tackles the challenges of single-shot LDR to HDR mapping by proposing a novel two-stage deep network. Notably, our proposed method aims to reconstruct an HDR image without knowing hardware information, including camera response function (CRF) and exposure settings.
  arXiv  Detail & Related papers  (2021-04-19T15:19:17Z)
• Beyond Visual Attractiveness: Physically Plausible Single Image HDR Reconstruction for Spherical Panoramas [60.24132321381606]
  We introduce the physical illuminance constraints to our single-shot HDR reconstruction framework. Our method can generate HDRs which are not only visually appealing but also physically plausible.
  arXiv  Detail & Related papers  (2021-03-24T01:51:19Z)
• HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions [62.44802076971331]
  We propose a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content.
  arXiv  Detail & Related papers  (2020-07-03T11:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.